Welcome to Journal Of Pakistan Medical Association
February, 2018 >>

Evaluation of effectiveness of a paediatric simulation course in procedural skills for paediatric residents A pilot study

Abdullah AlShammari, Aman Inayah, Nasir Ali Afsar, Akram Nurhussen, Amna Siddiqui, Muhammad Lucman Anwer, Sadek Obeidat, Mohammed Khaled Bakro, Tawfik Samer Abu Assale, Muhammad Zafar  ( Alfaisal University, College of Medicine )

Eyad Almidani, Abdullah Alsonbul, Sami Alhaider, Ibrahim Bin Hussain, Emad Khadawardi  ( Department of Pediatrics King Faisal Specialist Hospital & Research Center, Riyadh, Saudi Arabia )

Abstract

Objective: To explore the effects of simulation training on paediatric residents' confidence and skills in managing advanced skills in critical care.
Methods: The study was conducted at Alfaisal University, Riyadh, Saudi Arabia, from March to June 2016, and comprised junior residents in paediatrics. All paediatric residents (years 1 and 2) were recruited into two workshops, held one week apart. The first workshop covered lumbar puncture/ cerebrospinal fluid interpretation, oral intubation, bone marrow aspiration, and critical airway management. The second workshop covered chest tube insertion, pleural tap, insertion of central line, and arthrocentesis. The participants were surveyed using a 5-point Likert scale survey pre- and post-course, assessing their confidence. Their practical skills were assessed using a pre-objective structured clinical examination on the same day and post-course objective structured clinical examination a week later on selected skills. The outcome measures were: (1) pre-/post-course confidence rating, and (2) pre-/post-course objective structured clinical examination results. Data was analysed using SPSS 20.
Results: Of the 16 participants, 8(50%) were boys and 8(50%) girls. Besides, 13(81%) residents were in year-1 and 3(19%) in year-2. Median post-course confidence level ranks for all the skills were higher (p<0.05). There was no improvement in mean pre-objective structured clinical examination scores (2.31±2.66/ 7.46±3.02) and post- objective structured clinical examination scores (22.54±4.39/ 31.85±6.90) in Year 1 residents (p<0.001).
Conclusion: Simulation course was significantly successful in improving residents' clinical skills and confidence in performing critical tasks.
Keywords: Paediatric simulation course, Effectiveness of simulation, Clinical competency, Paediatrics clinical training. (JPMA 68: 240; 2018)

Introduction

Patient safety has always been a topic of paramount importance in the clinical practice. However, despite best efforts in creating a culture for patient safety, 'medical errors' have always plagued the healthcare systems.
In a 1991 Harvard research, which is the benchmark study on medical practice, Brennan et al. reported that medical management-related adverse effects occurred in around 3.7% of all hospital admissions.1 These findings were supported by another report, which concluded that medical errors led to at least 44,000
deaths in the United States annually during the study period.2Similar numbers have been cited worldwide including New Zealand,3 Australia4 and Canada,5 Although studies on medical errors in Saudi Arabia are scarce,6-9 comparable or even higher statistics are expected due to reasons such as language barrier and cultural peculiarities.
In response to those studies medical curricula started to stress more on the importance of clinical competency rather than mere knowledge acquisition.10 However, the question of how to practise the clinical skills still remains an issue; on the one hand was the ideology of 'practice makes perfect', whereas on the other hand was the patient safety concerns.
This demand-and-supply dilemma led to the era of simulation in clinical practice.11 Simulation, simply put, is a "technique to replace or amplify real-patient experiences with guided experiences, artificially contrived that evokes or replicates substantial aspects of the real world in a fully interactive manner."12 In its essence it is not a new concept; it has been used since long in various fields such as aviation, military and industry. However, its widespread formal application in medicine is only a recent development.13
Despite being in its early stages of evolution, several studies have already shown that simulation has had a significant positive impact on clinical learning.14 However, until now the use and efficacy of simulation in individualised clinical fields, for example paediatrics, still remains an area of research and needs further development. A recent study suggested that paediatric acute care providers have limited exposure to critically ill patients and subsequently lack the skills to manage them.15 This is especially true for the junior trainees, who lack even further in exposure and experience.2,11,12,14,16,17 Simulation has the potential to fill this technical void. The current study was planned to examine the immediate effects of a paediatric simulation course to paediatric residents at different levels of training. We planned to measure and compare the levels of reported confidence before and after learning the skills; compare the level of expertise of selected skills before and after learning the skills; and to measure the feasibility and impact of using small-group approach while practising on simulators.
We hypothesised that a structured simulation-based training workshop in procedural skills for paediatric residents should increase the level of confidence and expertise in performing the clinical procedures.

Subjects and Methods

This study was conducted at Clinical Simulation Centre, Alfaisal University, Riyadh, Saudi Arabia, from March to June 2016, and comprised junior residents in paediatrics. It was conducted in collaboration with Paediatric Residency Programme under the Department of Paediatrics at King Faisal Specialist Hospital and Research Centre-Riyadh (KFSH&RC) to match the curriculum requirements set by the Saudi Commission for Health Specialties (SCHS).  KFSH&RC is a tertiary care centre and a teaching hospital in paediatrics. It is one of the few hospitals in Saudi Arabia that admit children with complex health issues. Therefore, many of the patients admitted are complicated cases that often require specific clinical procedures being carried out on them. The study was approved by the institutional Committee for Medical and Bioethics.
The current training course was developed to train junior paediatrics residents about different clinical procedures and techniques required under different clinical scenarios. All junior residents in paediatrics from KFSH&RC were enrolled. All of them were from junior training level, including year-1 and year-2. The course was conducted in two sessions, scheduled one week apart, offering a total of eight skills as given below. During the first session, the residents were trained in four techniques, namely: 1) lumbar puncture and cerebrospinal fluid (CSF) interpretation, 2) oral intubation, 3) bone marrow aspiration, and 4) critical airway management. During the second session, conducted a week later, they were trained for the remaining 4 techniques: 1) chest tube insertion, 2) pleural tap, 3) insertion of central venous pressure line, and 4) arthrocentesis. The residents were divided into 4 groups. The skills were set in a series of 4 stations. Each group was assigned to start at a given station and rotate to the
remaining stations in sequence, spending a uniformly fixed time period. All the residents were exposed to all the techniques at the stations and had ample time to practise the technique at the respective station, where the technique was demonstrated and then supervised by a consultant paediatrician/clinical skills team member. For each skill/task, a structured checklist was prepared and provided to all trainees for standardised training.
The residents were assessed by two methods. Firstly, by using a standardised survey that was designed to explore the reflection (confidence) of the residents in carrying out the tasks and procedures they were trained on before and immediately after the course. The survey was written in a narrative manner, depicting relevant clinical situations which would require the skills in the selected set of procedures. The answers were collected on a 5-point Likert scale. However, the classical strongly-disagree-to-strongly-agree scale were replaced with 5 statements of practical steps that the resident may opt to do in these clinical situations, while each statement translates to an increasing level of confidence, where "1" denoting being "least confident in carrying out the given procedure, and "5" being fully confident to do so. Face validity of the questionnaire was obtained by discussing among researchers, in-house educationists and clinical consultants.
The second method of assessment was by performing an objective structured clinical examination (OSCE). For the OSCE, first-year residents, being naïve to the skill, were examined in which they were asked to perform a task prior to attending the simulation course (pre-OSCE). Two stations were selected: intubation (for day-1 sessions) and arthrocentesis (for day-2 sessions). A week after the completion of the course, another OSCE (post-OSCE) was conducted. Thus, each first-year resident had a pre- and post-simulation course score. Standardised, peer-reviewed checklists were prepared for all stations by the consultant paediatricians.
The data was analysed using SPSS 20. Mean ± standard deviation (SD) was used for quantitative variables, and frequency and percentage were used for qualitative variables. Cronbach's alpha and factor analysis of the survey was carried out. Wilcoxon signed-rank test was used to compare pre- and post-course agreement levels (subject to normality of the data). Paired samples t-test was used to examine the change between pre- and post-course OSCE scores. Conventional common theme analysis approach was used to explore open-ended answers. P<0.05 was considered significant.

Results

Of the 16 participants, 8(50%) were boys and 8(50%) girls. There were 4(25%) participants in each group. Moreover, 13(81%) residents were in year-1 and 3(19%) in year-2. Also, 13(81%) participants had no prior exposure to the selected skills. An attempt was made to get subjective input and feedback regarding the skills which were important according to the participants as well as barriers, if any, encountered in their learning. The majority of participants agreed that the skills were relevant to their residency programme as follows: airway management 15(93.8%), lumbar puncture 14(87.5%), intubation 14(87.5%), bone marrow aspiration 10(62.5%), chest tube insertion 10(62.5%), pleural tap 10(62.5%), central line insertion 11(68.8%) and arthrocentesis 9(56.3%).
The participants also pointed out some barriers towards mastering those skills as follows: lack of simulation training sessions 11(68.8%), time constraint 7(43.8%), limited exposure 5(31.3%), and limited supervision 1(6.3%) All the participants said they were motivated by the course to learn the selected set of skills, whereas 15(93.8%) said the course met their expectations.
The median post-course confidence level ranks for lumbar puncture (median=3.00), intubation (median=4.00), bone marrow aspiration (median=2.50), chest tube insertion (median= 3.00), pleural tap (median=3.00), central line insertion (median=3.00), and arthrocentesis (median=3.00) were significantly higher than the median post-course confidence level ranks for the same skills (lumbar puncture (median=5.00), intubation (median=5.00), bone marrow aspiration (median=4.00), chest tube insertion (median= 4.00), pleural tap (median=4.00), central line insertion (median=4.00 ), and arthrocentesis (median=4.00).
However, no significant difference was found between pre- and post-course confidence level ranks for critical airway management (Table-1, Figure).





The survey questionnaire was tested for internal consistency through measurement of Cronbach's alpha. The alpha value was 0.73, thus "acceptable". We checked for the redundancy of the variables in the questionnaire and appropriateness for factor analysis by conducting principle component analysis through Bartlett's test of sphericity and Kaiser-Meyer-Olkin (KMO) test. Bartlett's test of sphericity was significant (p=0.001) suggesting that original correlation matrix was not an identity matrix. Although KMO statistic was 0.3 (<0.5, most likely due to small sample size), factor analysis yielded a typical screen plot. Thus, the redundancy of the variables was ruled out.
To have more realistic evaluations of the course, year-1 residents were administered an OSCE before and after the training, which showed significantly improved expertise in the relevant skills. Since the sample size was less than 30, the data was checked for normality; it was found that the data did not violate normal distribution (p<0.05). The improvement was given in terms of improved point score while performing the task, judged through a peer-reviewed checklist.
The qualitative responses were analysed using the conventional common theme analysis approach. The qualitative responses were divided under three subheadings, namely, (a) most liked aspects of the workshop, (b) most beneficial sessions in the course, and (c) reasons for taking the course. It was found that 7(43.8%) participants liked hands-on practice, 6(37.5%) said all sessions were beneficial and 5(31.3%) identified general skill development as a reason for taking the course (Table-2).



Discussion

In this pilot study, we reported our experience in conducting a paediatrics simulation course among residents. This course was designed as per the guidelines set by the scientific committee for Paediatric Residency Programme as laid down by the Saudi Commission for Health Specialties (SCHS). We found that the simulation-based approach used in this course was an effective method of learning to train residents who would otherwise have limited access to training on actual patients due to various factors discussed below. Among the residents who completed the course, there was an improved ability to execute the clinical skills, and increased confidence of residents in the performance of assigned skills.
According to our literature search, this study is first of its kind in the region. This is important to note that this region differs from other geographical regions in several key factors, such as being highly dependent on more traditional, tutor-oriented approach to medical education where an individual trainee would get potentially less hands-on experience. This implies that a majority of clinicians could acquire relevant clinical skills only in late residency or during fellowships. The same was also pointed out in the feedback about barriers by the course participants in our study.
SCHS, which is the governing body for medical training in Saudi Arabia, now strongly recommends to include simulation sessions in order to improve the skills of clinical residents. Thus our study allows testing the new guidelines set by the SCHS, which were designed to evaluate and encourage the systematic use of simulation in postgraduate training programmes.
The study of Falck et al. of 449 residents determined that none of their resident groups met their specified definition of technical competence when performing neonatal endotracheal intubations. They concluded that as opportunities to perform the skill become more limited, new methods of developing technical competency should be established.16
The value of using simulators in the field of clinical skills training cannot be undermined. A study led by Leone et al.17 that examined 5,051 successful neonatal intubations at a single academic centre determined that the median success rate for the procedure among level 1, 2, and 3 residents, and neonatal fellows were 33%, 40%, 40% and 68%, respectively. They further concluded that the poor success rate among junior residents was due to inadequate opportunities to perform the skill at their centre.
It is possible that clinical skill simulation may prove to be an appropriate means of attaining clinical skill competence in the presence of limited live training opportunities. Grantcharov et al. found that surgeons who've received virtual reality surgical simulation showed significantly greater performance in the operating room than surgeons who had not.14 Simulation can also lead to statistically significant increases in both confidence and competence.18
Since the study aimed to test the efficacy of the paediatric training course on residents, this could be best evaluated on residents who have had minimal or no previous exposure to the clinical skills being tested. Hence, all participants in our study were junior (year 1 and year 2) paediatric residents.
The results of the study showed a significant increase in the confidence of the course participants to perform particular clinical skills. The most significant increase in confidence was reported in the ability to perform lumbar puncture, intubation, bone marrow aspiration, chest tube insertion, pleural tap, central line insertion and arthrocentesis. However, there was no increase in confidence in performing critical airway management, despite that the pre-OSCE confidence level matches the median pre-OSCE confidence levels noted for the other skills preformed in this study. Even though only further studies can explore this, it may be due to relative lack of time and opportunity to fully grasp the skill, given the complexity of this particular skill. Finally, the study also showed that the majority of residents reported the course met their expectations, and they were highly satisfied.
The schedule constraints allowed only two clinical skills to be assessed in the pre- and post-OSCE, namely intubation and arthrocentesis. In the intubation station, student performance showed an average increase of 20.23 points out of a 28-point checklist, as well as an increase of 24.38 points out of a 44-point checklist on the arthrocentesis station. These results show that the course not only improved the junior residents' ability to perform a clinical skill, presumably, it also improved their ability to take a holistic approach when encountering a real patient. Such approach includes a proper introduction, preparation of materials, preparing the patient for the procedure, taking appropriate precautions, performing the skill, eliciting and reporting findings, and appropriately ending the encounter. However, the notion needs to be validated with further studies.
Several factors were suggested by the course participants as limitations to their ability to acquire and practise clinical skills in the hospital. Among the biggest barriers encountered were limited opportunity to practice in real-life situation, lack of simulators or an organised simulation centre as well as absence of structured simulation-based programme during residency to practise and gain confidence in a skill.
We must acknowledge some factors which limit the generalisation of this study. As discussed earlier, the intended focus of this study was junior paediatric residents, which limited our ability to interpret the relevance of our results across all levels of training and across other disciplines.
Furthermore, we found it to be particularly challenging to recruit residents for this study during their busy training. In the absence of previous research to indicate the potential and feasibility of a study such as ours, it was challenging to convince programme directors to participate in the study and allow their residents to dedicate time from their work-hours.
Given the results of this pilot study, we are encouraged to further expand it to include a larger sample size, wider variety of clinical skills in the pre-and post-OSCE, as well as include a long-term follow-up. This follow-up could include an assessment of the junior residents' retention of the skills learned during the course over a given period of time, their confidence in performing the skill in the long term, and the assessment of the relevance of taught skills in their actual clinical practice.
To develop the skills of residents, we strongly recommend that a simulation centre within the training hospital should be established, along with a structured simulation-based programme.
One limitation of the study was its small sample size; however, it should be appreciated that all available paediatric residents were included as the study population.

Conclusion

Simulation course was significantly successful in improving residents' clinical skills as well as confidence in performing critical tasks. The residents felt satisfied with the course.

Disclaimer: None.
Conflict of Interest: None.
Source of Funding: None.

References

1.  Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991; 324: 370-6.
2.  Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press (US); 2001. COMMITTEE ON QUALITY OF HEALTH CARE IN AMERICA. [Online] [Cited 2017 Jan 25]. Available from: URL: https://www.ncbi.nlm.nih.gov/books/NBK222278/.
3.  Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J 2002; 115: U271.
4.  Evans SM, Berry JG, Smith BJ, Esterman A, Selim P, O'Shaughnessy J, et al. Attitudes and barriers to incident reporting: a collaborative hospital study. Qual Saf Health Care 2006; 15: 39-43.
5.  Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004; 170: 1678-86.
6.  AlJarallah JS, AlRowaiss N. The pattern of medical errors and litigation against doctors in Saudi Arabia. J Family Community Med 2013; 20: 98-105.
7.  Alsaddique AA. Medical liability. The dilemma of litigations. Saudi Med J 2004; 25: 901-6.
8.  Al-Saeed AH. Medical liability litigation in Saudi Arabia. Saudi J Anaesth 2010; 4: 122-6.
9.  Al-Saeed AH. Status of medical liability claims in Saudi Arabia. Saudi J Anaesth 2007; 1: 4.
10.  Smith SR. AMEE Guide No. 14: Outcome-based education: Part 2-Planning, implementing and evaluating a competency-based curriculum. Med Teach 1999; 21: 15-22.
11.  Gaba DM. The future vision of simulation in health care. Qual Saf Health Care 2004; 13suppl 1: i2-10.
12.  Haidar E. Clinical simulation: a better way of learning? Nurs Manag (Harrow) 2009; 16: 22-3.
13.  Passiment M, Heather S, Huang G. Medical simulation in medical education: results of an AAMC survey. Washington, DC: Association of American Medical Colleges; 2011.
14.  Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004; 91: 146-50.
15.  Weinberg ER, Auerbach MA, Shah NB. The use of simulation for pediatric training and assessment. Curr Opin Pediatr 2009; 21: 282-7.
16.  Falck AJ, Escobedo MB, Baillargeon JG, Villard LG, Gunkel JH. Proficiency of pediatric residents in performing neonatal endotracheal intubation. Pediatrics 2003; 112: 1242-7.
17.  Leone TA, Rich W, Finer NN. Neonatal intubation: success of pediatric trainees. J Pediatr 2005; 146: 638-41.
18.  Augustine EM, Kahana M. Effect of procedure simulation workshops on resident procedural confidence and competence. J Grad Med Educ 2012; 4 :479-85.

 


NEWS AND EVENTS

ANIMAL-BASED STUDIES


Research articles conducted on animals, will not be considered for processing or publication in the JPMA.

FOR REVIEWERS

ANNOUNCEMENT

SUPPLEMENT

INDEX

INSTRUCTIONS TO AUTHORS

COMMITTEE ON PUBLICATION ETHICS


This journal is a member of and subscribes to the principles of the Committee on Publication Ethics.

Copyrights © 2015 JPMA- All rights reserved
Powered by: PakCyber