June 2021, Volume 71, Issue 6

Research Article

Comparison of OSCE scores of surgical clinical education after face-to-face vs. blended learning methods among undergraduate medical students

Masood Jawaid  ( Adjunct Faculty, Jinnah Sindh Medical University, Karachi, Pakistan )
Lubna Baig  ( APPNA Institute of Public Health, Jinnah Sindh Medical University, Karachi, Pakistan )
Syed Moyn Aly  ( Department of Medical Education, Jinnah Sindh Medical University, Karachi, Pakistan )

Abstract

Objective: To compare the efficacy of blended learning with contemporary face-to-face teaching among medical students.

Methods: The quasi-experimental, cross-over study was conducted at the Dow University of Health Sciences, Karachi, from March to August 2014, and comprised third year medical students posted in the surgical units who were divided into two groups. In the first month, group A was taught by blended learning (BL), while group B was taught by face-to-face (f2f) teaching. Both groups were assessed using objective structured clinical examination. In the second month, the teaching method was flipped for the groups with new learning contents. Assessment tool remained the same. Data was analysed using SPSS 23.

Results: Thirty-one students completed OSCE in group A (BL) and 28 in group B (f2f). The mean score in group B was 78.01±13.29 compared to 85.12±13.77 in group A (p=0.49).

Conclusion: Blended surgical learning was found to be more effective compared to face-to-face teaching.

Keywords: Blended learning, Medical education, OSCE, Teaching strategies, Learning approach. (JPMA 71: 1535; 2021)

DOI: https://doi.org/10.47391/JPMA.571

 

Introduction

 

Clinical education involving patients and their problems is a core component of undergraduate medical education.1 It includes taking history from patients, examining them, developing a differential diagnosis and proceeding with the rest of the management. It brings students from the artificial setting of the classrooms into the real world of tertiary healthcare delivery.2 It results in increased student motivation and learning relevance, and helps in developing clinical expertise.3

Despite its potential strengths, clinical education has been criticised for its variability and haphazard nature. Clinical environment is usually opportunistic; one cannot ensure relevant clinical encounters. There is no guarantee when learners are present and prepared to learn, and also when sufficient patients with good clinical features are available.3 Other issues include lack of availability of teaching faculty due to their clinical engagements, refusal of patient consent and patient safety issues while teaching procedures like nasogastric (NG) intubation and  catheterisation.4 However, in many curricula, students are assessed on a standardised objective structured clinical examination (OSCE) test after completion of such unsystematic clinical rotations.

Most of the teaching in clinical rotations is face-to-face (F2F). Another approach is that of blended learning (BL) which is defined as a combination of traditional F2F and synchronous or asynchronous e-learning.5 This medium includes discussion forums, flash-based interactive videos, interactive case-based scenarios, static pages and quizzes to teach high cognitive clinical skills within a given time duration. In short, BL is the integration of online and F2F instructions to support meaningful interaction between teachers and learners regarding learning content.6 It ensures standardised learning experiences for all students. With the utilisation of modern technology, teachers can ensure that students achieve their learning objectives effectively and within the limited time.

Currently, there is scant literature on BL using OSCE to assess its effectiveness. One study used Community-Primary-Care-Preceptorship (CPCP), which is an elective BL programme comprising clinical and online components. Four interactive online modules were developed comprising office practice, challenging patient interviews, common primary care diagnosis, and prevention of diseases. At the end of the course, the students completed a generalised OSCE exam. Results indicated that CPCP students scored greater than the control group with statistically significant difference.7

The current study was planned to compare OSCE scores after BL and F2F methods.

 

Subjects and Methods

 

The quasi-experimental, cross-over study was conducted at the Dow University of Health Sciences (DUHS), Karachi, from March to August 2014. After approval from the institutional ethics review board, the sample size was calculated using OpenEpi v.2,5 with power of the test and confidence interval 95%, and other data based on a previous study.6

The sample was raised using non-probability purposive sampling technique from among third year medical students of either gender posted in the surgical units after taking informed consent. Students with less than 70% ward attendance and those who did not take the OSCE exam at the end of the rotation were excluded.

The subjects were divided into two groups by the departmental administration; F2F group was sent to Surgical Unit 1 and was trained using the traditional strategies, while the BL group was sent to Surgical Unit 2 and were exposed to BL (Figure 1).

The F2F group had clinical exposure to real patients, small group discussions (SGDs), simulated patient sessions where students took mock histories and performed mock examination for formative feedback, and theatre-case observations where students were taught scrubbing, gowning and gloving, and were also given a chance to assist in minor procedures.

The BL group students were exposed to additional strategies. An online course was designed based on the cognitive load theory of multimedia learning.6 This exposed the students to different diseases and examination findings and allowed for increased interaction time between students and faculty. This also helped in formative feedback.

The Sharable Content Object Reference Model (SCORM)8 was also developed for interactive flash learning contents for the BL group. It produced packages of online learning material which could be shared across diverse systems. For good learning experience, videos were uploaded as SCORM packages including question-answers, hotspot and clickable links (Figure 2).

Simulated cases developed with flash technology (Articulate Studio) were also developed for the BL group. They helped the students with clinical reasoning skills with prompt feedback for right and wrong options. These scenarios brought the students closer to real-life situations. Potential areas for additional training before facing real patients were identified. SCORM also compensated for lack of faculty time for effective feedback.

Threaded discussion forum was introduced for conversational modes of learning. Static pages with information about operation theatre (OT) instruments, NG, Foleys, Redivac drain etc. were supplied. Pictures of lumps and bumps, like lipoma, sebaceous cyst and carbuncle, with characteristic points about each were also displayed. Every week, students had to attempt 10 One-Best type multiple choice questions (MCQs) within 15 minutes. Students received feedback with explanations immediately. Scores of lessons from static pages, SCORM packages and quizzes were stored in a customised grade book for every student. Students maintained log-books about what they learned in the out-patient departments (OPDs), wards and OTs.

The topics taught to both the groups were the same. At the end of four weeks, since the topics were the same, one OSCE was administered to both the groups to assess difference in performance.

In order to be ethically fair to the entire cohort, the study was designed to expose both the groups to BL. Hence, after the OSCE was held at the end of four weeks, the initial F2F group was sent to Surgical 2 and vice versa. This time different surgery-related topics were taught, but the teaching methods were switched. Another OSCE was administered to both groups at the end of four weeks. OSCE marks from the two cohorts were combined for both the groups. Each OSCE station was of maximum 10 marks, with total maximum score of 140 for 14 stations.

OSCE stations were selected from a blueprint for content validity, and the 14 stations included were checked and validated by three surgical faculty members and one medical educationist. Validation was done to ensure that the items used in the OSCE stations were suitable for the content and also to ensure that they matched the objectives of the posting. Data was analysed using SPSS 23. Mean and standard deviation (SD) were calculated for quantitative responses, while frequencies and percentages were computed for qualitative output response. OSCE scores were compared for any significant difference with unpaired student t test. Statistical significance was taken at p<0.05. Cronbach’s alpha was used to evaluate internal consistency of the OSCE results. Difficulty and discrimination indices were used to find out any difference in post-hoc analysis result in the two groups.

 

Results

 

For total study subjects, 31 students completed the OSCE in the BL group and 28 in the F2F group [all student exposed to both type of learning in this cross over study]. Demographic details were noted (Table 1).

Total internal reliability coefficient of 14 OSCE stations was 0.915, with all stations contributing positively to the reliability (Table 2).

Overall, there was significant improvement in OSCE score (p=0.049) after BL method 85.12±13.77 compared to F2F 78.01±13.29 (Table 3).

 

Discussion

 

OSCE scores for the BL group were significantly higher than those for the F2F group.  This is one of the first studies from Pakistan about BL in surgical clinical education. Recent studies suggested that majority of the students use mobile technology to improve how they learn new material, and prefer taking classes that incorporate information technology (IT).9

The findings are in concordance with a systematic review10 which reported that learning of clinical skills via BL is “no less” than that by traditional methods. This is an important finding as knowledge improvement can be accomplished easily with web-based learning, while significant performance improvement is difficult to achieve with computer-based learning alone, and the importance of properly developed online modalities of teaching cannot be emphasised enough.

Local data regarding the importance of BL is scarce, and the current study can initiate a discourse on its usage. A multi-centre observational study11 reported that 51.6% faculty members perceived the importance of integration of technology in their teaching. Around 54% recognised themselves capable of integrating technology in teaching and learning. However, 54.5% participants asserted that their university lacked mentorship in the field of BL.

The current results are also in line with a study12 that used video case studies, discussion forum, workbooks with resource links as a teaching medium among social work students for reflective skills. Results showed that students’ presentations established in-depth theoretical understanding and improved reflective analysis. A study13 compared F2F and BL approaches in medical students and the results were not different from those of the current study. Another study14 reported that students’ satisfaction and learning improved with BL.

In contrast, one study15 reported no significant difference between its BL and F2F groups. A likely explanation could be that the pre-existing clinical curriculum made it tough for any added e-learning exposure to further develop the skills.

One study16 compared the effectiveness of BL with didactic learning (DL) approach of teaching evidence-based medicine (EBM) to medical students, and concluded that the two groups did not have significantly different competencies in EBM. However, BL was significantly more effective in improving students’ attitudes toward EBM and self-reported use of EBM in clinical practice.  However, emphasis should not be on technology alone; rather, the clinical teacher must regulate how technology will boost the teaching of any particular subject before using any particular medium.17-19

The current study has its limitations as the conclusions drawn are based on data from only one specialty and one institution. Besides, the date was collected about 6 years ago which could not be publicly shared owing to academic engagements.

 

Conclusion

 

BL was found to be more effective than F2F teaching methodology.

 

Disclosure: The text is based on a Masters of Health Profession Education (MHPE)) research project.

Conflict of Interest: None.

Source of Funding: None.

 

References

 

1.      Spencer J. ABC of learning and teaching in medicine: Learning and teaching in the clinical environment. BMJ 2003; 326: 591.

2.      Morris C. Facilitating learning in the workplace. [Online] [Cited 2018 July 7]. Available from: URL: http://www.faculty.londondeanery.ac.uk/e-learning

3.      Ericsson KA. Acquisition and Maintenance of Medical Expertise: A Perspective from the Expert- Performance Approach With Deliberate Practice. Acad Med 2015; 90: 1- 15

4.      Jedaar Z, Marrin C, Pugsley L. How to… Overcome barriers to effective work-based learning. Educ Prim Care 2009; 20: 477-9.

5.      Dean AGSK, Soe MM. OpenEpi: open source epidemiologic statistics for public health, version 2.3.1. [Online] [Cited 2018 Oct 8]. Available from: URL: www.openepi.com

6.      Liu Q1, Peng W, Zhang F, Hu R, Li Y, Yan W. The Effectiveness of Blended Learning in Health Professions: Systematic Review and Meta-Analysis. J Med Internet Res 2016; 18: 1- 18

7.      Garrison DR, Kanuka H. Blended learning: Uncovering its transformative potential in higher education. Internet High Educ 2004; 7: 95–105.

8.      Lewin L, Singh M, Bateman B, Glover P. Improving education in primary care: development of an online curriculum using the blended learning model. BMC Med Educ 2009; 9: 33

9.      Mayer RE, Moreno R. Nine ways to reduce cognitive load in multimedia learning. Educ Psychol 2003; 38: 43-52.

10.    McCutcheon K1, Lohan M, Traynor M, Martin D. A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education. J Adv Nurs 2015; 71: 255-70

11.    Law JK, Thome PA, Lindeman B, Jackson DC, Lidor AO. Student use and perceptions of mobile technology in clinical clerkships - Guidance for curriculum design. Am J Surg 2018; 215: 196-9.

12.    Cooner TS. Creating opportunities for students in large cohorts to reflect in and on practice: Lessons learnt from a formative evaluation of students' experiences of a technology‐enhanced blended learning design. Br J Educ Technol 2010; 41: 271-86.

13.    Makhdoom N, Khoshhal KI, Algaidi S, Heissam K, Zolaly MA. ‘Blended learning’as an effective teaching and learning strategy in clinical medicine: a comparative cross-sectional university-based study. J Taibah Univ Med Sci 2013; 8: 12-7. 

14.    Sajid MR, Laheji AF, Abothenain F, Salam Y, AlJayar D, Obeidat A. Can blended learning and the flipped classroom improve student learning and satisfaction in Saudi Arabia? Int J Med Educ 2016; 7: 281-5

15.    Carbonaro M, King S, Taylor E, Satzinger F, Snart F, Drummond J. Integration of e-learning technologies in an interprofessional health science course. Med Teach 2008; 30: 25-33.

16.    Ilic D, Nordin RB, Glasziou P, Tilson JK, Villanueva E. A randomised controlled trial of a blended learning education intervention for teaching evidence-based medicine. BMC Med Educ 2015; 15: 39.

17.    Rizvi NF, Gulzar S, Nicholas W, Nkoroi B. Barriers in adopting blended learning in a private university of Pakistan and East Africa: faculty members’ perspective. Mhealth. 2017; 3:18.

18.    Ocak MA, Topal AD. Blended learning in anatomy education: a study investigating medical students' perceptions. EURASIA J Math Sci Tech Ed 2015; 11: 647-83.

19.    Jeffrey LM, Milne J, Suddaby G, Higgins A. Blended learning: How teachers balance the blend of online and classroom components. Journal of Information Technology Education: Research 2014; 13:121-40.

 

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: