Ayesha Abubakar Mitha  ( Department of Riphah Academy of Research and Education, Ripah International University, Islamabad, Pakistan; )
Usman Mahboob  ( Institute of Health Professions Education and Research, Khyber Medical University, Peshawar, Pakistan. )
Tayyaba Rahman  ( Department of Riphah Academy of Research and Education, Ripah International University, Islamabad, Pakistan; )

Objective: To analyse practices of patient safety non-technical skills among postgraduate trainees to establish the utility of organised teaching programmes in this regard, and to determine the utilisation of different teaching modes.

 

Method: The multicentre, two-phase, cross-sectional study was conducted from February to August 2020 at 27 teaching hospitals across Pakistan which were part of the Patient Safety Friendly Hospital Initiative by the World Health Organisation. The Hospital Survey on Patient Safety Culture was modified with validated additional survey items before using it to gather relevant data. Data was analysed using SPSS 21.

 

Results: In the first phase, 42 additional survey items were developed with content validity ratio >0.66 and item content validity index >0.83. Cronbach’s alpha of the modified survey tool was 0.790. The second phase comprised 388 postgraduate trainees; 199(51.3%) males and 189(48.7%) females. Of them, 134(34.5%) were in the third year of training, 215(55.4%) fourth, and 39(10.1%) in the final year. Highest positive response was found for learning Teamwork 216(55.7%) and the lowest for Situational Awareness 62(15.9%). Leadership had highest good practice responses (subscales range: 77.6% to 76.6%) and Communication had the lowest (subscales range: 16.5% to 74.2%). Agreement on Informal Learning mode was the highest 268(69.1%) and the lowest was for Simulator Learning 63(16.2%). Besides, 274(70.6%) and 281(72.4%) subjects agreed on the supportive role of supervisors and hospital administrations. The correlation of the studied variables with the year of training was significant only for Teamwork (p=0.02) and Medication Safety skills (p=0.01).

 

Conclusion: Modified Hospital Survey on Patient Safety Culture could be used as a benchmark for evaluating patient safety teachings and practices. Significance of patient safety non-technical skills was established with limited evidence for the utility of organised teaching programmes.

 

Keywords: Patient safety non-technical skills, Postgraduate trainees, Low- and middle-income countries.

 

DOI: https://doi.org/10.47391/JPMA.2445

 

Introduction

 

Patient Safety (PS), a mandatory component of quality healthcare, is the prevention of patient-related adverse events resulting from the processes of healthcare delivery. Clinicians must acquire core concepts of PS, including technical as well as non-technical skills (NTSs), in order to improve their skills as clinicians and to improve the working environment of their institutions. PS NTS are the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and efficient task performance. Eight PS NTS domains were broadly identified through literature search, including Leadership, Adverse Event Reporting, Communication, Working Safely, Medication Safety, Situational Awareness, Teamwork and System-Based Practice skills. ¹-⁵, ⁶-⁸. Significant reduction from 18.5% to 13% in errors caused by healthcare staff was noticed following PS NTS courses in hospitals. ¹ Data to support the efficacy of these strategies is still deficient. ² Postgraduate trainees (PGTs) play a pivotal role in implementing PS measures, being in the forefront of dealing with patients, and must undergo PS NTS training. ² The challenge is how to implement PS NTS among doctors as these are not considered a mandatory requirement in hospitals the world over. ¹-⁷

There is a lack of comprehensive studies to evaluate PS NTS among junior doctors as there is no specific validated measuring tool. ⁸ Breakdown in multiple domains of these skills still remains a root cause of harm to patients as PS training is deficient in postgraduate curricula. ⁹ The role of supervisors to train PGTs on PS is also found deficient. ¹⁰ Organisational factors to influence the implementation of teaching PS NTSs to hospital staff are also not clear.

Studies on practices of PS NTS and their teaching programmes in hospitals of low- and middle-income countries (LMICs) are extremely deficient. ⁴ In Pakistan, limited hospital-based studies exist regarding the evaluation of PS NTS attitudes among doctors. ¹¹,¹² The World Health Organisation (WHO) has launched a PS Friendly Hospital Initiative (PSFHI) in LMICs which includes 27 postgraduate teaching hospitals in Pakistan. ¹³,¹⁴ No comprehensive report is available on the implementation of PS culture in these hospitals.

The current study was planned to fill the gap by analysing PS NTS practices among PGTs to establish the utility of organised teaching programmes in this regard, and to determine the utilisation of different teaching modes in this context.

 

Subjects and Methods

 

The multicentre, two-phase, cross-sectional study was conducted from February to August 2020. Hospital Survey on Patient Safety Culture (HSOPSC) v1.0 developed by the Agency for Healthcare Research and Quality (AHRQ), United States of America, and designed for healthcare settings was selected. ¹³,¹⁴ All the identified domains of PS NTS and teaching modalities were not assessed by the tool. ¹⁵ A modified AHRQ HSOPSC tool was required for survey on PS practices and teaching modalities among PGTs. After permission from AHRQ to use and modify HSOPSC v1.0 only with addition to the survey items, the additional items were constructed and were subsequently psychometrically validated by a 7-step process.

In step 1 new items were created. Following literature review, 46 additional survey items were constructed in line with Association for Medical Education in Europe (AMEE) Guide 87 guidelines. ¹⁶ To the existing domains of Teamwork, System-Based Practices and Communication skills, only items of prevalence of teaching programmes were added. To the domain of Adverse Event Reporting skill, five items were added. Leadership skill was updated with three items, Situational Awareness skill with seven items, Working Safely skill with eleven items, Medication Safety skill with four items, and Teaching Modalities section with eight items.

Step 2 comprised random interviews of participants on conceptualisation of items. Unprompted information was collected from two PGTs each from the departments of General Medicine, General Surgery, Paediatrics, Anaesthesiology and Obstetrics and Gynaecology from Combined Military Hospital Lahore.  There was lack of clarity only on items related to Situational Awareness skill.

Step 3 comprised improving item for clarity through repeated interviews of PGTs to achieve saturation. Items under Situational Awareness skill were modified. All relevant definitions were added, and interviews were repeated to achieve saturation on clarity of items.

Step 4 comprised expert view on relevance and essentiality of the additional items. All items were assessed by six experts, including three PS experts and three medical educationists through online Google Forms to establish content validity (CV).

In the fifth step, validity of additional items was quantified and the element of chance was reduced by using calculating Lawshe’s content validity ratio (CVR) and item content validity index (I-CVI) for each item. ¹⁷,¹⁸

Step 6 comprised pilot testing which was done with 53 subjects from among the targeted population using the modified survey tool to establish internal reliability.

In the final step of the first phase Cronbach’s alpha value of the modified survey tool was calculated along with mean inter item correlation values for additional domains and sections.

–The second phase of the study involved PS Culture Survey using the modified HSOPSC. Twenty-seven teaching hospitals across Pakistan which were part of the WHO PSFHI programme were approached. Eighteen hospitals responded and were sent the survey questionnaire. Only 09 hospitals met the eligibility criteria and responded with data.

A quantitative analytical cross-sectional survey was conducted on third, fourth and final year PGTs of the 09 PSFHI hospitals of Pakistan by Simple Random Sampling technique, after ERC approval of author’s university. Non-responders were excluded. The sample size was calculated using the formula:

          n= p (100 – p) z² 19

                                    E²

Socio-demographic data was noted, and responses of PGTs using a Likert scale were taken for PS NTS practices, teaching programmes, teaching modes, role of supervisors, and role of hospital administrations.

Cumulative percentages of the response rate were categorised as ‘good’ and ‘poor’, while teaching programmes were categorised as ‘organised’ and ‘not organized’.

Responses role of supervisors, role of hospital administrations and teaching modes were ‘disagree’, ‘agree’ and ‘neutral’.

Data was analysed using SPSS ²¹. Chi-square test of association was run between related categorical variables and correlation analysis was run using binary logistic regression. P<0.05 was considered statistically significant.

 

Results

 

In the first phase, 42 additional survey items were developed with CVR >0.66 and I-CVI >0.83 (Table 1).

 

 

Cronbach’s alpha of the modified survey tool was 0.790. Leadership skill and Medication Safety skill had poor mean inter item correlation scores but with acceptable I-CVI values of their subscales (Table 2).

 

 

The Internal reliability of the modified tool post-survey Showed improved Cronbach’s alpha value of 0.96.

The second phase comprised 388 postgraduate trainees; 199(51.3%) males and 189(48.7%) females. Of them, 134(34.5%) were in the third year of training, 215(55.4%) fourth, and 39(10.1%) in the final year.

In terms of PS initiatives being followed at their respective hospitals,314(80.9%) PGTs answered in the affirmative, 24(6.2%) in the negative, and 50(12.9%) said they were not sure. Also PS practice at their respective hospitals was graded excellent by 76(19.6%) PGTs, very good by 195(50.3%), acceptable by 97(25.0%), poor by 19(4.9%) and falling by 1(0.3%). Number of adverse events reported per year was none in 157(40.5%) cases, ¹-² in 132(34.0%), 3-5 in 94(24.2%), 6-10 in 3(0.8%) and 11-20 in 2 (0.5%) cases.

Cumulative responses were graded for all the eight domains (Table 3).

 

 

PS NTS teaching programmes were generally graded negatively (Figure).

 

 

Highest positive response was found for learning Teamwork 216(55.7%) and the lowest for Situational Awareness 62(15.9%). Leadership had highest good practice responses (range of subscales: 77.6% to 76.6%) and Communication had the lowest (range of subscales: 16.5% to 74.2%). Agreement on Informal Learning mode was the highest 268(69.1%), followed by 206(53.1%) on Courses and Workshops, 176(45.3%) on Bedside Teachings, 95(24.5%) on Small Group Discussions (SGDs), 68(17.5%) on Didactic Lectures and 63(16.2%) on Simulator Learning.

Besides, 274(70.6%) subjects agreed on the supportive role of supervisors, 291(75%) acknowledged that the supervisors did not push them to work faster under pressure to compromise PS, and 315(81.2%) agreed that the supervisors did not overlook PS problems.

Also, 281(72.4%) PGTs agreed that hospital managements provided a working climate to promote PS, 248(63.9%) agreed that hospital units coordinated well on PS, 254(65.5%) acknowledged that PS was a top priority for hospital managements, and 274(70.6%) PGTs agreed that shift changes in units were not problematic.

Association results for “Improvement of Patient Management through PS NTSs” and “Different Teaching Modes” showed Chi-square for Informal Learning process x²- 136.9,(p=0.001), Didactic lectures x²-93.6,(p=0.001), SGDs/tutorials x²-109.8  (p=0.001), Courses and Workshops x²-196.6(p=0.001), Bedside Teaching x² 167.2 (p=0.001) and for Simulators x²- 64.2 (p=0.001).

“Teaching and practices of Adverse Event Reporting skill” and “Frequency of Adverse Events Reported” also showed significant association (Table 4).

 

 

The association between “PS NTS Domain Teachings and Practices” and “Years of postgraduate training” had significant association only in the context of Teamwork (p=0.02) and Medication Safety skills (p=0.01). For the rest of the domains, the correlation with the year of training was not significant (p>0.05)

Binary logistic regression showed the impact of years of PGT on Teamwork skill to be significant (p=0.014); odds ratio [OR] 1.48; 95% confidence interval [CI] 0.970-2.263). The model explained 2.2% (Cox and Snell R square) and 2.9% (Nagelkerke R square) of variance in practicing Teamwork skills. It correctly classified 220(56.7%) of the cases.

The impact of years of PGT on Medication Safety skill was not significant (p=0.938, OR 1.017, 95% CI 0.658-1.574).

 

Discussion

 

The first PS NTS survey among PGTs was conducted in United Kingdom in 2013.8 In Pakistan, a similar study limited to only the Medicine department of a hospital was conducted in 2018. ¹² This current study is the first comprehensive research on teaching and practising of PS NTS among PGTs in Pakistan, and used the psychometrically validated modified HSOPSC tool. Low values of inter-item correlation in two domains indicate deficiencies in methodological rigour in designing items due to inadequate subject material. ¹³

Most of the PGTs agreed that PS initiatives were being followed in their hospitals 314(80.9%), and 255(65.7%) appreciated good systems and procedures to promote PS, with 241(62.1%) agreeing on active implementation of PS measures. According to 260(67%) PGTs, PS was never sacrificed by them to get more work done in their respective fields, implying that PS measures were being practiced by the PGTs to a significant level.

In a British study, 70% PGTs agreed on good practices of Communication skill. ⁸ It was ranked third in another study on PGTs in Pakistan. ¹² In the current study, PGTs exhibited mixed pattern in the Communication skill subscale. There was lack of communication with more authoritative senior staff members. Only 176(45.5%) PGTs agreed on organised teaching programmes, implying the need of improved teaching modalities.

In a study on PGTs in Pakistan, practices of Teamwork skill was ranked fourth. ¹² In a study in Turkey, the average positive response rate was 82% for all staff, and 75% in a study in Iran. ²⁰ In the current study, 309(79.6%) trainees agreed that the staff worked as a team in crisis, and 216(55.7%) expressed that they were learning Teamwork as the most taught PS NTS. Significant statistical correlation in improvement of Teamwork skill with increasing years of PG training was attempted, but could not be established.

System-based practices are not assessed in many studies as a PS skill. PGTs on average were found to be practising this skill, calling effectively on resources in the system to provide optimum healthcare facilities. Practices were by self-directed as only 135(34.8%) trainees agreed on organised teaching programmes.

In one study in Pakistan, the Situational Awareness skill was ranked first in health practice. ¹² In similar studies, it was needed to be enhanced to improve decision-making by trainees working in emergency departments. ⁶-¹⁰ Regarding the Working Safely skill, 227(58.5%) PGTs were coping well with their working hours. Both domains have not been studied much in Asian countries. The PGTs exhibited good practices of both these skills in all subscales. In one study, only 57.4% PGTs reviewed patient’s medical history before surgery. ²¹ In the current study, 341(87.9%) PGTs reviewed patients’ medical records before procedures. Regarding ethical behaviour towards patients and personal reflection to recognise deficiencies in PS measures, 177(45.6%) PGTs declared that their supervisors discussed practices of ethical behaviour towards patients, and 212(54.6%) acknowledged practising reflection. Working Safely skill also enhances measures for infection prevention and hand hygiene measures for which good practice of hand hygiene was adopted as 284(73.2%) PGTs confirmed it, and effective infection prevention and control measures were confirmed by 283(72.9%). However, only 62(16%) and 78(20.1%) PGTs confirmed the presence of teaching programmes for both skills, respectively, implying that informal learning process was the practised learning mode.

One study identified the Leadership skill as the fundamental basis of PS culture. ²² Weightage of fifth position was given in a previous study. ¹² Although 107(45.4%) PGTs declared having no training programme, they were practicing a leader’s role by ensuring the required standards of patient management.

For Medication Safety skill, purpose and dosage of prescribing medicines were explained to patients by only 233(67.8%) and 258(66.5 %) PGTs respectively. Over-dosage risks were either not explained or poorly explained by only 137(43.5%) PGTs. Only 123(31.7%) PGTs declared that teaching programmes were present to teach the skill. Significant statistical correlation of its practices with years of PG training was attempted, but could not be established.

In the present survey, 315(81.2%) PGTs learnt from he errors committed by team members. Only 85(21.9%) emphasised on teaching programmes. As a result, only 157(40.5%) PGTs reported no adverse event, 132(34.0%) reported 1-2 events, and 94(24.2%) reported 3-5 events per year. Significant association was established between frequency of Adverse Event Reporting with practicing and teaching of the skill.

In the United States, only one-quarter of medical schools taught PS with limited hours. ²² In a systematic review on LMICs, no teaching programmes were organised for PS NTS9 and majority of healthcare institutions were found to be in the planning stages of implementing the PS culture in their respective hospitals. ²² The PGTs were mostly learning PS NTS as part of the hidden curriculum depending on individual, unit and system factors related to their respective work areas, their supervisors and the focus of hospital administration. In the current study, 239(61.6%) PGTs declared no planned teaching programmes, implying that minimum initiatives were being taken by hospitals. Significant statistical association was found for all teaching modes with improvement in patient management which merits their maximum utilisation.

Deficient faculty training contributes to poor implementation of PS culture. ²² In a study in Iran, average positive role of supervisors was acknowledged by 76% hospital staff. In Turkey, it was 52.1%.¹⁵ Positive role of supervisors was identified in the current study, with 315(81.2%) PGTs acknowledging that their supervisors did not overlook PS problems, a possible contributory factor for PGTs’ good PS NTS practices.

In a British study, the positive role of management to improve PS was acknowledged by 46% PGTs⁸ and a systematic review concluded 72% average positive response by hospital staff. ¹⁵ In Iran, it was 65% and in Turkey it was 54.7%.¹⁵ In the current study, 281(72.4%) PGTs agreed that the hospital administrations provided a conducive working climate to promote PS. Overall, 195(50.3%) PGTs graded their hospital as “Very Good” for facilitating a PS culture, 97(25%) graded them “Acceptable” and 76(19.6%) graded them as “Excellent”. The WHO PS guide shared with selected hospitals to implement PS culture is a tremendous initiative and must be utilised as a set of guidelines to augment the process. PS survey tools increase awareness among participants¹³ with positive responses.

In terms of the limitations of the current study, theoretical deficiencies in literature on elements of PS NTS domains affected the development of survey items. Due to lack of quantitative studies conducted on PGTs for PS NTSs, there was limited statistical data for direct comparison. Weak statistical association between PS NTS practices with years of training merits further inferential analysis of associations and correlations among PS NTS variables.

 

Conclusion

 

The modified survey tool may be used as a benchmark for future evaluation of the PS culture. The findings can be used as a baseline set of knowledge for PS curriculum planning. A collaborative effort by medical educationists, supervisors and hospital management is needed to augment workplace PS NTS with rigorous training programmes utilising all the identified modes of training.

 

Acknowledgement: We are grateful to patient safety (PS) experts for validating the additional survey items, participating postgraduate trainees (PGTs) and administrations of Agha Khan, Holy Family, Pakistan Institute of Medical Sciences, Khan Research Laboratories, Combined Military Hospital Lahore, Lahore General, Services hospital, Shalamar hospital and Khyber teaching hospitals.

 

Disclaimer: The text is based on a thesis for Masters in Health Professions Education.

 

Conflict of interest: None.

 

Source of Funding: None.

 

References

 

1.       Walton M, Harrison R, Burgess A, Foster K. Workplace training for senior trainees: a systematic review and narrative synthesis of current approaches to promote patient safety. Postgrad Med J 2015;91:579-87. doi: 10.1136/postgradmedj-2014-133130.

2.       Dieckmann P, Glavin R, Hartvigsen Grønholm Jepsen RM, Krage R. Non-Technical Skills Bingo-a game to facilitate the learning of complex concepts. Adv Simul (Lond) 2016;1:e23. doi: 10.1186/s41077-016-0024-z.

3.       Frank JR, Snell L, Sherbino J, eds. CanMEDS 2015 Physician Competency Framework. Ottawa, Canada: Royal College of Physicians and Surgeons of Canada; 2015.

4.       Ounounou E, Aydin A, Brunckhorst O, Khan MS, Dasgupta P, Ahmed K. Nontechnical Skills in Surgery: A Systematic Review of Current Training Modalities. J Surg Educ 2019;76:14-24. doi: 10.1016/j.jsurg.2018.05.017.

5.       Kirkman MA, Sevdalis N, Arora S, Baker P, Vincent C, Ahmed M. The outcomes of recent patient safety education interventions for trainee physicians and medical students: a systematic review. BMJ Open 2015;5:e007705. doi: 10.1136/bmjopen-2015-007705.

6.       Scott J, Revera Morales D, McRitchie A, Riviello R, Smink D, Yule S. Non-technical skills and health care provision in low- and middle-income countries: a systematic review. Med Educ 2016;50:441-55. doi: 10.1111/medu.12939.

7.       White N. Understanding the role of non-technical skills in patient safety. Nurs Stand 2012;26:43-8. doi: 10.7748/ns2012.02.26.26.43. c8972.

8.       Durani P, Dias J, Singh HP, Taub N. Junior doctors and patient safety: evaluating knowledge, attitudes and perception of safety climate. BMJ Qual Saf 2013;22:65-71. doi: 10.1136/bmjqs-2012-001009..

9.       Cooper S, Porter J, Peach L. Measuring situation awareness in emergency settings: a systematic review of tools and outcomes. Open Access Emerg Med 2013;6:e1-7. doi: 10.2147/OAEM.S53679.

10.    Mundschenk MB, Odom EB, Ghosh TD, Kleiber GM, Yee A, Patel KB, et al. Are Residents Prepared for Surgical Cases? Implications in Patient Safety and Education. J Surg Educ 2018;75:403-8. doi: 10.1016/j.jsurg.2017.07.001.

11.    Bari A, Jabeen U, Bano I, Rathore AW. Patient safety awareness among postgraduate students and nurses in a tertiary health care facility. Pak J Med Sci 2017;33:1059-64. doi: 10.12669/pjms.335. 13780.

12.    Majeed N, Mahboob U. Doctors’ experiences and awareness of non-technical skills, a way to the development of a behavioral marker system for patient management. Health Professions Educator Journal 2018;1:38-44. DOI: 10.53708/hpej.v1i1.30

13.    Alsalem G, Bowie P, Morrison J. Assessing safety climate in acute hospital settings: a systematic review of the adequacy of the psychometric properties of survey measurement tools. BMC Health Serv Res 2018;18:353. doi: 10.1186/s12913-018-3167-x.

14.    The Health Foundation. Evidence scan: Measuring safety culture. London, UK: The Health Foundation; 2011.

15.    Fleming M. Patient safety culture measurement and improvement: a "how to" guide. Healthc Q 2005;8:14-9. doi: 10.12927/hcq.2005. 17656.

16.    Artino AR Jr, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE Guide No. 87. Med Teach 2014;36:463-74. doi: 10.3109/0142159X.2014.889814.

17.    Gilbert GE, Prion S. Making Sense of Methods andMeasurement: Lawshe’s ContentValidity Index. Clin Simul Nurs 2016;12:530-1. doi: 10.1016/j.ecns.2016.08.002

18.    Rodrigues IB, Adachi JD, Beattie KA, MacDermid JC. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskelet Disord 2017;18:540. doi: 10.1186/s12891-017-1914-5.

19.    Creswell JW. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 3rd ed. California, USA: SAGE Publications, Inc, 2008; pp 205-7.

20.    Damayanti RA, Bachtiar A. Outcome of patient safety culture using the hospital survey on patient safety culture (hsopsc) in asia: a systematic review with meta analysis. In: Proceedings of the International Conference on Applied Science and Health No. 4. Thailand, Indonesia: 2019; pp 849-63.

21.    Hull L, Arora S, Aggarwal R, Darzi A, Vincent C, Sevdalis N. The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 2012;214:214-30. doi: 10.1016/ j.jamcollsurg.2011.10.016.

22.    Ginsburg LR, Dhingra-Kumar N, Donaldson LJ. What stage are low-income and middle-income countries (LMICs) at with patient safety curriculum implementation and what are the barriers to implementation? A two-stage cross-sectional study. BMJ Open 2017;7:e016110. doi: 10.1136/bmjopen-2017-016110.