By Author
  By Title
  By Keywords

October 2013, Volume 63, Issue 10

Original Article

Assessing the Objective Structured Clinical Examination: Saudi family medicine undergraduate medical students’ perceptions of the tool

Hafsa Raheel  ( Department of Family and Community Medicine, College of Medicine, King Saud University, Dariyyah, Riyadh, Kingdom of Saudi Arabia. )
Naghma Naeem  ( Chair for Medical Education Research and Development, College of Medicine, King Saud University, Dariyyah, Riyadh, Kingdom of Saudi Arabia. )

Abstract

Objectives: To assess the perception and opinion of the medical students about Obejective Structured Clinical Examination.
Method: The descriptive study was conducted at King Saud University College of Medicine, Riyadh, Saudi Arabia, from March to May 2010. Total population sampling, which is a type of purposive sampling technique, was used. A structured pre-tested questionnaire was circulated among the undergraduate medical students doing Family Medicine clerkship immediately at the end of Objective Structured Clinical Examination.
Results: Of the 97 questionnaires distributed, 95(98%) were received. A number of positive aspects of the OSCE exam were identified by students such as 63(66%) felt that it provided practical and useful experience; 63(66%) perceived as having a positive impact on student learning; 62(65%) stated it is a standardised exam; 53(56%) felt that it should be used more often in undergraduate assessment; 54(57%) stated that it evaluated a wide variety of clinical skills; 50(53%) stated that the format allowed students to compensate for deficiency in some areas; 49(52%) were of the opinion that the exam was fair; 45(47%) preferred it over other forms of assessment; 39(41%) perceived exam scores to be truly reflective of competence in clinical skills; and 34(35%) felt that the format minimises chances of failure. Also, 46(48% found it to be stressful, and 38(40%) found it not easy.
Conclusions: Overall, the Objective Structured Clinical Examination was perceived very positively and welcomed by the undergraduate family medicine students. Certain negative perceptions such as stress and difficulty were also highlighted, which could be managed through better orientation and preparation of the students.
Keywords: Education, Students, Clinical examination, Perceptions. (JPMA 63: 1281; 2013).

Introduction

The Objective Structured Clinical Examination (OSCE) was first conceptualised in 1975.1 It is the method of choice for evaluation of learner\\\'s clinical competence.2,3 OSCE is used for teaching as well as for making high-stake decision regarding licensing of physicians.4 However, a review of reliability and validity research does not clearly demonstrate the superiority of OSCE testing.4 Many studies have reported less-than-ideal reliability scores for OSCEs because of reasons that include content specificity,5 student fatigue, personal bias, relatively high anxiety about the testing method, and memory lapse.6,7 Despite a long tradition of research relating to OSCE, there have been relatively fewer studies about the perceptions of the students about OSCE as an assessment tool and its educational impact. A study reported that faculty and examinee ratings of the OSCE experience were very positive and over a 9-year period, student performance improved, showing less variability and significantly fewer failed stations.8
An OSCE was recently introduced at the end of the clerkship in Family Medicine in 4th year of an undergraduate medical school in Saudi Arabia. To our knowledge, OSCE has not been used as an assessment tool at the 4th year undergraduate level in the field of Family Medicine, elsewhere in Saudi Arabia. Since assessment is a key driver of learning, it is important to explore the views of the learners about this assessment tool of clinical competence.
The objective of the current study was to explore the perceptions of undergraduate medical students about the OSCE.

Subjects and Methods

The descriptive study was conducted at King Saud University College of Medicine, Riyadh, Saudi Arabia, from March to May 2010. Total population sampling, which is a type of purposive sampling technique, was used. We opted for the technique as the population size was relatively small and non-inclusion of subjects might have resulted in significant loss of information. Since total population sampling involves all members within the population of interest, it was possible to get deep insights into the subjects under study.
A questionnaire was developed comprising items to explore both positive and negative aspects of the OSCE experience. The questionnaire was kept simple and short to maximize the response rate. A five-point rating scale was used, which was collapsed to three categories for the purpose of analysis. The questionnaire (Appendix) was piloted before actual administration to address any ambiguities.
The questionnaire was distributed immediately afte the OSCE at the end of the Family Medicine clerkship.

Results

A total of 95 completed questionnaires were received, back from the 97 distributed. The response rate, as such, was 98%. A number of positive aspects of the OSCE exam were identified by students such as 63(66%) felt that it provided practical and useful experience; 63(66%) perceived as having a positive impact on student learning; 62(65%) stated it is a standardised exam; 53(56%) felt that it should be used more often in undergraduate assessment; 54(57%) stated that it evaluated a wide variety of clinical skills; 50(53%) stated that the format allowed students to compensate for deficiency in some areas; 49(52%) were of the opinion that the exam was fair; 45(47%) preferred it over other forms of assessment; 39(41%) perceived exam scores to be truly reflective of competence in clinical skills; and 34(35%) felt that the format minimises chances of failure. Also, 46(48% found it to be stressful, and 38(40%) found it not easy (Table-1).


Discussion

King Saud University, Riyadh, is one of the biggest and the oldest universities in Saudia Arabia. Like other public-sector universities in the country, its undergraduate examination system comprises written examination based on multiple choice questions (MCQs) accompanied in some disciplines by oral examinations. Despite the fact that OSCE is a recognised and widely used tool of assessment in many medical schools globally, its application is mainly limited to postgraduate medical education in Saudi Arabia. There is recent evidence that OSCE has been used as a mode of assessment in the surgical and medical undergraduate examinations in Saudi Arabia.9,10 However, the King Saud University implemented this mode of assessment in the discipline of Family Medicine for undergraduate fourth year final examination for the first time.
The aim of the study was to obtain the opinion of medical students about various aspects of OSCE as students\\\' feedback is regarded as a key indicator for successful implementation of the OSCE and also provides feedback for improvement.11
The survey gathered the opinions of undergraduate medical students about a number of aspects regarding OSCE based on their actual experience with this tool. A high response rate of 98% was achieved, which makes bias unlikely in the study and provides sufficient evidence to base our conclusions. Majority of the students generally perceive OSCE as a positive experience.12 This was demonstrated by the positive responses regarding standardisation, fairness, practicality and usefulness of the exam. In a study of pharmacy students, an overwhelming proportion of the students (90%) agreed that the OSCE provided a useful and practical learning experience.13 The students also validated the authenticity of the OSCE by expressing that OSCE was reflective of their actual competence in clinical skills. Earlier studies also reported that most students felt that the OSCE accurately assessed their clinical skills.14,15 Students\\\' acceptance of the OSCE was demonstrated by their perception of OSCE as the preferred method of assessment of clinical skills. Majority of the students in the current study felt that OSCE had a positive impact on their learning and should be used more often in undergraduate examinations, They also felt that OSCE assessed a wide variety of skills and required different kinds of preparation than that required for other examinations. This perception, however, needs to be explored further by qualitative studies.
Besides, 35% of the students affirmed that the OSCE process minimised their chances of failing, but 47% were not sure on this count. This can be explained on the basis of their inexperience with this format of examination.
Students were unsure about the influence of personality, gender and ethnicity on scores in OSCE, and if the format provides opportunity for constructive feedback. These perceptions might be explained by the fact that one opportunity to experience OSCE might be inadequate to form an opinion about these aspects.
Also, 26% students felt that the examination was less stressful and 29% felt that it was easy compared with 46% and 38% respectively who felt otherwise. This perception of stress and difficulty may partly be attributed to the fact that the students were being exposed to this format of examination for the first time without proper orientation or training in the form of mock exams. Prior to the examination, the students were briefed on the way the exam would be conducted, time allocated to each station and requirements at each station.
It might also be due to fact that OSCE assesses competencies other than those assessed by traditional methods of assessment and, therefore, requires practice and feedback. It has been reported in earlier studies also that OSCE provokes anxiety among students.16,17
The study was conducted at a single medical college. As such its generalisability is limited. However, since the study is exploratory in nature, it does provide insights for further exploration.
Total population sampling used in the current study is a purposive sampling technique (i.e. non-probability sampling), making it impossible to make statistical generalisations about the population being studied. However, the use of total population sampling did make it possible to make analytical generalisations.
In addition, in-depth qualitative research methods could have have been used to gain further insights, which was not done in the current study but can be on future research agenda.

Conclusions

Overall, OSCE was perceived very positively and welcomed by the undergraduate Family Medicine students. Certain negative perceptions such as stress and difficulty were highlighted, which could be managed with better orientation and preparation of students, including training through mock examination with debriefing at end the of examination and provision of constructive feedback on their performance.
Future research efforts should include qualitative studies to investigate the reasons for negative perceptions such as stress and difficulty.

Acknowledgement

The study was supported by the College of Medicine Research Centre, Deanship of Scientific Research, King Saud University, Riyadh, Kingdom of Saudi Arabia.

References

1. Harden R, Stevenson M, Downie WW, Wilson GM. Clinical competence in using objective structured examination. Br Med J 1975; 1: 447-51.
2. Howley L. Performance assessment in medical education: where we\\\'ve been and where we\\\'re going. Eval Health Prof 2004; 27: 285-303.
3. Epstein R. Assessment in medical education. N Engl J Med 2007; 356: 387-96.
4. Turner JL, Danoski ME. Objective Structured Clinical Exams: a critical review. Fam Med 2008; 40: 574-8.
5. Van der Vleuten CPM, Swanson DB. Assessment of clinical skills with standardized patients: the state of the art. Teach Lrn Med 1990; 2: 58-76.
6. Rutala PJ, Witzke DB, Leko EO, Fulginiti JV, Taylor PJ. Student fatigue as a variable affecting perrformance in an objective standardized clinical examination. Acad Med 1990; 65 (9 Suppl): S53-4.
7. Tamblyn RM, Klass DJ, Schnabl GK, Kopelow ML. Sources of unreliability and bias in standardized patient rating. Teach Learn Med 1991; 3: 74-85.
8. Duerson MC, Romrell LJ, Stevens CB. Impacting faculty teaching and student performance: nine years\\\' experience with the objective structured clinical examination. Teach Learn Med 2000;12: 176-82.
9. Al-Naami MY, El-Tinay OF, Khairy GA, Mofti SS, Anjum MN. Improvement of psychometric properties of the objective structured clinical examination when assessing problem solving skills of surgical clerkship. Saudi Med J 2011; 32: 300-4.
10. Bakhsh TM, Sibiany AM, Al-Mashat FM, Meccawy AA, Al-Thubaity F. Comparison of students\\\' performance in the traditional oral clinical examination and the objective structured clinical examination. Saudi Med J 2009; 30: 555-7.
11. Khursheed I, Usman Y, Usman J. Students\\\' feedback of objectively structured clinical examination: a private medical college experience. J Pak Med Assoc 2007; 57: 148-150.
12. Larsen T, Jeppe-Jensen D.The introduction and perception of an OSCE with an element of self- and peer-assessment. Eur J Dent Educ 2008; 12: 2-7.
13. Awaisu A, Mohamed MH, Al-Efan QA. Perception of pharmacy students in Malaysia on the use of objective structured clinical examinations to evaluate competence. Am J Pharm Educ 2007; 71: 118.
14. Awaisu A, Abd Rahman NS, Nik Mohamed MH, Bux Rahman Bux SH, Mohamed Nazar NI. Malaysian pharmacy students\\\' assessment of an objective structured clinical examination (OSCE). Am J Pharm Educ 2010; 74: 34.
15. A-Latif A. An examination of the examinations: the reliability of the objective structured clinical examination and clinical examination. Med Teach 1992; 14: 179-83.
16. Allen R, Heard J, Savidge M, Bittengle J, Cantrell M, Huffmaster T. Surveying students\\\' attitudes during the OSCE. Adv Health Sci Educ 1998; 3: 197-206.
17. Troncon LE. Clinical skills assessment: limitations to the introduction of an "OSCE" (Objective Structured Clinical Examination) in a traditional Brazilian medical school. Sao Paulo Med J 2004; 122: 12-7.

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: