Objectives: To assess the environment of postgraduate fellowship training in teaching hospitals of an urban centre.
Methods: The cross-sectional study was conducted at one public-sector and two private-sector teaching hospitals in Karachi from December 2014 to June 2015. Data was collected by using a modified version of Postgraduate Hospital Educational Environment Measure, a validated questionnaire, for which clinical residents were selected through convenience sampling. Data was analyzed using SPSS 16.
Results: Of the 302 participants, 168(55.6%) were males and 134(44.4%) were females. The overall mean age of the respondents was 28.46±3.03 years. The internal reliability of the questionnaire was good with a Cronbach's alpha of 0.92. The overall mean score of 93.96±20.79suggested more positive than negative perception with room for improvement. After adjusting for all important socio-demographic and residency co-variates, residency in a private hospital was positively associated with Postgraduate Hospital Educational Environment Measure score (p<0.01) compared to residency in public hospitals.
Conclusion: There is an urging need to standardise postgraduate training in terms of teaching, autonomy and social support in public and private hospitals of Karachi.
Keywords: PHEEM, Postgraduate training, Residency, Quality. (JPMA 67: 171; 2017)
Provision of quality environment in training junior doctors is emerging as a subject of great interest with proliferation of new teaching hospitals and postgraduate programmes, especially in the developing countries. As a result, training requirements have increased in modern times and now involve vast areas of management, team work, supervision, social support and research. Understanding the environment of these programmes is imperative to manage them effectively and improve their performance.1,2 Moreover, a good learning and teaching environment is also directly related to good professional performance of junior doctors.3 There is rich evidence on the benefits of improving the educational and environmental aspects of hospital training in improving performance of junior doctors.4-7 Therefore, good planning for clinical teaching is essential, otherwise grave medical errors may result.8
Globally, postgraduate medical training programmes have been standardised and are kept under a tight check to provide an acceptable standard of active training to its residents (trainees). In Pakistan, postgraduate training for various disciplines of medicine and surgery is done in accredited tertiary care hospitals for a period of 4-5 years depending on the specialty after passing an exam and registering with the College of Physicians and Surgeons of Pakistan (CPSP). Few standards are in place related to accrediting hospitals as being fit to offer training and mandatory trainings are given to supervisors in areas of educational planning and evaluation, assessment of competence, supervisory skills and research.9 However, a proactive supervised evaluation of training standards is not in effect in Pakistan and there is variation in the proficiency and excellence of the training programmes in different hospitals throughout Pakistan.10 To our knowledge, no study had been done in the country based on perceptions of resident trainees to assess the education environment of postgraduate training using a standardised validated tool. This study was planned to assess the education environment of postgraduate fellowship training and to identify differences in public-sector and private-sector tertiary care hospitals.
Subjects and Methods
The cross-sectional study was conducted in one public-sector and two private-sector hospitals in Karachi from December 2014 to June 2015. The list of residents was obtained from the administration of the relevant hospitals and all the residents from each discipline were approached. We expected to achieve a sample size of 191 calculated at 99% confidence level, 5% margin of error and expected variance of 721 taken from a previous study.11 However, our sample size was much larger after convenience sampling technique was employed.
The data was collected using a modified version of Postgraduate Hospital Educational Environment Measure (PHEEM) which has been evaluated as a reliable tool to measure the educational environment of postgraduate training programmes12-14 and has been used in several countries to assess clinical training.11,15,16 The 40-item questionnaire developed by Roff S.17 comprehensively evaluates the learning environment in hospital settings for residents by measuring perceptions of trainees in three domains: perceptions of autonomy, teaching and social support. Reponses are recorded on five-point agreement Likert scale and each item is given a score from 0-4. Responses generate a maximum score of 160 with scores of 0-40 meaning very poor, 41-80 portraying plenty of problems, 81-120 explaining more positive than negative with room for improvement, and 121-160 as excellent. In order to pinpoint more specific strengths and weaknesses based on individual items, a mean score of 3.5 or above is considered a real positive result. Any item with a mean of 2 or less is examined more closely as they indicate problem areas. Items with a mean between 2 and 3 are aspects of the climate that could be enhanced. Items in the questionnaire were modified to match the local applicability. The following questions were modified:
In question no. 7, "There is racism in this post" was changed to "there is ethnic or religious discrimination in this post".
In question no. 9, "There is an informative Junior Doctors Handbook" was changed to "There is a document on structured training specifying core competencies to be acquired at different levels".
In question no. 11, "I am bleeped inappropriately" was changed to "I am called inappropriately" (as all resident do not have pagers in all hospitals).
In question no. 17, "My hours conform to new deal" was changed to "My working hours are appropriate" (as there is no resident work time directive in Pakistan).
The data was collected by medical students. They were thoroughly trained about the objectives of the study and the questionnaire. Field testing of the final questionnaire was done before starting the formal data collection. The questionnaire was modified based on the feedback of the students after pilot testing. Data collection process was supervised by principal investigator throughout the data collection process. Forms were checked for completeness. Data was entered twice and consistency between the two data sets was checked.
Ethical approval was obtained from the Institutional Review Board of Jinnah Sindh Medical University. Official Permission was taken from administrations of all the hospitals where the data was collected. The names of hospitals have been kept confidential. All participants gave informed written consent prior to getting enrolled in the study.
The data was entered on SPSS 16.0 and cross-validated by random checking. Reliability of responses was checked using Cronbach's alpha. Frequencies and percentages were calculated for categorical variables, while means and standard deviations were calculated for continuous variables. Overall PHEEM score and score of its three domains (autonomy, teaching and social support) was categorised into three levels with different cut-off scores and their frequencies and percentages were calculated. After checking for normality of distribution, overall PHEEM scores and scores of its three domains were summarised as mean±standard deviation.
After checking for normality of distribution, Independent T-test was used to compare the overall mean scores of PHEEM and each of its items between public and private hospitals. Generalised Linear Regression analysis was used to examine the unadjusted and adjusted association between independent variables, including gender, type of hospital, hometown, residency specialty, household monthly income and dependent variable that is, PHEEM scores. Results were reported as Beta Coefficients with 95% confidence intervals (CI). P<0.05 was considered significant.
Of the 350 individuals invited for participation, 320(91.4%) consented, but 18(5.6%) of the entries had to be excluded due to incomplete information. The final sample stood at 302(94.4%) participants. The overall reliability of PHEEM score was high with Cronbach's alpha being 0.917 (Table 1).
There were 168(55.6%) males and 134(44.4%) females with an overall mean age of 28.46±3.03 years. Overall, 112(37%) residents were married; 171(56.6%) had family income less than Pak rupees (PKR) 100,000; 131(43.4%) had above PKR100,000; and 139(56%) had their hometown outside Karachi (Table 2).
PHEEM scores of 82(27%) residents were less than 81; 195(64.6%) perceived their residency more positive than negative with room for improvement; 25(8.6%) had excellent perception about their training; 83(27.8%) had negative perceptions about autonomy;84(27.5%) had negative perceptions about teaching; and 127(42.1%) perceived their workplace to be unpleasant in terms of social support (Figure).
The overall mean score of 93.96±20.79suggested more positive than negative perception with room for improvement. Overall mean PHEEM score was significantly higher for the private hospitals compared to the public hospital (p<0.01). Mean scores of majority of items was greater than 2 except 3 items including having access to document on structured training, being warned inappropriately, and adequacy of catering facilities while on call.
Pertaining to items related to perceptions on autonomy, residents in public hospitals reported significantly higher mean scores for working hours being appropriate (p<0.01), workload being fine (p<0.01)and not being warned inappropriately (p<0.01). On the contrary, private hospitals reported significantly higher scores for having an induction programme (p=0.02), appropriate level of responsibility(p<0.01), clear clinical protocols (p<0.01), opportunity to provide continuity of care (p<0.01), culture of mutual respect (p=0.02) and being ready to be a consultant (p<0.01).
Relating to items on perceptions for teaching, residents in private hospitals reported significantly higher scores for quality of clinical supervision (p<0.01), communication skills (p=0.01), enthusiasm (p<0.01), provision of regular feedback (p<0.01), organisation (p<0.01), accessibility (p<0.01) and encouraging residents to be independent learners (p<0.01). Residents in private hospitals also reported significantly higher mean score on few aspects of social support including less gender discrimination (p<0.01), more collaboration with other doctors (p<0.01), physical safety (p=<0.01) and counselling opportunities for junior doctors (p=0.03) (Table 3).
After adjusting for all important socio-demographic co-variates, residency in private hospitals showed positive association with PHEEM score (Beta = 6.79; 95% CI 6.35, 7.24;p<0.01) compared to residency in public hospitals. Similarly, PHEEM score was also positively associated with residents who were female and actually belonged to areas outside Karachi (p<0.05%). It was negatively associated with residents whose household monthly income was above PKR 100,000 (p<0.05). While doing residency in paediatric wards showed positive association with PHEEM score, doing residency in gynaecology and obstetrics showed negative association (p<0.05).
This is the first multi-centre study with adequate sample size which has used a modified version of PHEEM to assess the education and environment of postgraduate medical training in Pakistan. Globally, PHEEM has been rated as a tool with excellent reliability having Cronbach's alpha ranging from 0.84 to 0.95.13,16-18 High PHEEM scores have also showed an association with better knowledge and exam performance.3 The reliability of the PHEEM adapted to local context of Pakistan also showed excellent Cronbach's alpha of 0.917. However, its three-dimensional nature is still a question mark with one study reporting it to be one-dimensional and other two extracting five factors explaining the variation.13,16,18
The overall mean score suggested more positive than negative perception with room for improvement (Mean=93.96 ± 20.79). Similar mean scores ranging from 82.64to 102 have been reported from studies in developed and developing countries.11,13,15,19 Mean scores of majority of items was between 2 and 3, suggestive of environment requiring enhancement and improvement. Low scores were particularly identified for items calling for improvements in standardising the training and ensuring adequate facilities and infrastructure for residents.
More than one-fourth of the residents reporting a score of <81 suggests that there are variations in perceptions of residents from different backgrounds working in different environments. PHEEM score was found to be negatively associated with female residents. However, when adjusted for other co-variates, it turned positive. This may be due to the confounding element of residency in gynaecology and obstetrics which was initially positive, but after adjustment had negative association with PHEEM score. A study in Saudi Arabia, however, found that females were less likely to report better scores.11 However, the study doesn't provide adjusted association of PHEEM scores with gender and residency department.
Residents with hometown outside Karachi reported better PHEEM scores possibly due to the fact that training environment in urban hospitals presumably is better than peri-urban hospitals. This is contrary to a finding in a study in a developed country like Australia where no significant difference in PHEEM scores was reported in urban and rural settings.20 In comparison to residents in medicine and surgery, paediatric residents reported better scores while gynaecology and obstetrics residents reported significantly less scores. This explains that there is variability in perception about training not only between hospitals but also within different disciplines and departments within the hospitals.
Residency in private hospitals also showed positive association with PHEEM score. Significant differences between different types of hospital have also been reported from studies in Saudi Arabia and Japan.15,21 While generally residents of private hospitals had more positive perceptions on different aspects of autonomy, teaching and social support, residents in public hospitals only reported better score for working hours and workload. A similar study in Pakistan has reported that average working hours in private hospitals are higher compared to public hospitals.22
Among different items pertaining to autonomy, the residents of private hospitals reported higher scores for having an effective induction programme, appropriate level of responsibility, clear clinical protocols, opportunity to provide continuity of care and readiness of residents to be a consultant. This is suggestive of the fact that training programmes are more organised in private hospitals from induction to training protocols. Residents in private hospitals also rated teaching related factors highly including quality of clinical supervision, communication skills, enthusiasm, and provision of regular feedback, accessibility and encouraging residents to be independent learners. Greater competency, enthusiasm and teaching skills among non-university hospitals have also been reported in a study from Japan.15
Residents in private hospitals also reported significantly higher mean score on few aspects of social support, including less gender discrimination, more collaboration with other doctors, physical safety and counselling opportunities for junior doctors. These findings are indicative of better infrastructure and organisational policies in the private sector.
PHEEM is a strong tool, allowing the stakeholders to reflect the strengths and weaknesses of their residency programme.
Our study has a few limitations. Only one public-sector hospital was compared with two private-sector hospitals due to refusal of permission to collect data by one public-sector hospital. It was not logistically possible to include any other public-sector hospital due to resource constraints. Moreover, this study only provides a picture of a few major training hospitals of Karachi. It is likely that carrying out such an assessment in teaching hospitals of smaller cities may yield different results. Also, PHEEM is a self-reporting tool. This tool may be used to carry out a multi-centre large-scale study also representing smaller cities. The tool may also be modified further to suit the local context in line with CPSP guidelines.9 The modified tool may then be used for periodic monitoring of postgraduate education environment in all the settings.
Overall, postgraduate environment in three hospitals of Karachi was reported to be satisfactory by residents with room for improvement in all aspects of autonomy, teaching and social support. There is a need to standardise postgraduate training with periodic monitoring to account for the significant variability among different hospitals in terms of training conditions, protocols and infrastructure.
We are grateful to all the study subjects.
Disclaimer: The study was presented at AEME '16 Conference at Peshawar on March 5, 2016.
Conflict of Interest: None.
Source of Funding: None.
1. Genn JM. AMEE Medical Education Guide No. 23 (Part 1): Curriculum, environment, climate, quality and change in medical education-a unifying perspective. Med Teach 2001; 23: 337-44.
2. Harden R. The learning environment and the curriculum. Med Teach 2001; 23: 335-6.
3. Shimizu T, Tsugawa Y, Tanoue Y, Konishi R, Nishizaki Y, Kishimoto M, et al. The hospital educational environment and performance of residents in the General Medicine In-Training Examination: a multicenter study in Japan. Int J Gen Med 2013; 6: 637-40.
4. Miller A, Archer J. Impact of workplace based assessment on doctors' education and performance: a systematic review. BMJ 2010; 341: c5064.
5. Nøhr S, Ipsen M. The 3-hour meeting concept: an organisational approach for involvement of junior doctors in education. Ugeskr Laeger 2008; 170: 3523-4.
6. Thangaratinam S, Yanamandra S, Deb S, Coomarasamy A. Specialist training in obstetrics and gynaecology: A survey on work-life balance and stress among trainees in UK. J Obstet Gynaecol 2006; 26: 302-4.
7. Sheehan D, Wilkinson TJ. Maximising the clinical learning of junior doctors: applying educational theory to practice. Med Teach 2007; 29: 827-9.
8. Spencer J. Learning and teaching in the clinical environment. BMJ 2003; 326: 591.
9. College of Physicians and Surgeons of Pakistan. [online] [cited 2013 July 20]. Available from: URL: http://www.cpsp.edu.pk/ index.php?code=dHJhaW5pbmdfZ3VpZGVsaW5lc3xzdXBlcnZpc29yZS5waHB8MHwwfDA=.
10. Graduate Doctors & Students of AllamaIqbal Medical College, Lahore. Postgraduation and Specialization in Pakistan. [online] 2000 [cited 2013 July 20]. Available from: URL: http://www.iqbalians.com/links/fcps.asp.
11. Al-Marshad S, Alotaibi G. Evaluation of Clinical Educational Environment at King Fahad Hospital of Dammam University Using the Postgraduate Hospital Education Environment Measure (PHEEM) Inventory. Educ Med J 2011; 3: e6-e14.
12. Vieira JE. The postgraduate hospital educational environment measure (PHEEM) questionnaire identifies quality of instruction as a key factor predicting academic achievement. Clinics 2008; 63: 741-6.
13. Riquelme A, Herrera C, Aranis C, Oporto J, Padilla O. Psychometric analyses and internal consistency of the PHEEM questionnaire to measure the clinical learning environment in the clerkship of a Medical School in Chile. Med Teach 2009; 31: e221-e5.
14. Roff S, McAleer S, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach 2005; 27: 326-31.
15. Tokuda Y, Goto E, Otaki J, Jacobs J, Omata F, Shapiro M, et al. Educational environment of university and non-university hospitals in Japan. Int J Med Educ 2010; 1: 10-4.
16. Gooneratne I, Munasinghe S, Siriwardena C, Olupeliyawa A, Karunathilake I. Assessment of psychometric properties of a modified PHEEM questionnaire. Ann Acad Med Singapore 2008; 37: 993-7.
17. Koutsogiannou P, Dimoliatis ID, Mavridis D, Bellos S, Karathanos V, Jelastopulu E. Validation of the Postgraduate Hospital Educational Environment Measure (PHEEM) in a sample of 731 Greek residents. BMC Res Notes 2015; 8: 734
18. Shokoohi S, Emami AH, Mohammadi A, Ahmadi S, Mojtahedzadeh R. Psychometric properties of the Postgraduate Hospital Educational Environment Measure in an Iranian hospital setting. Med Educ Online 2014; 19: 10.
19. Flaherty G, Connolly R, O'Brien T. Measurement of the Postgraduate Educational Environment of Junior Doctors Training in Medicine at an Irish University Teaching Hospital. Ir J Med Sci 2016; 185: 565-71
20. Gough J, Bullen M, Donath S. PHEEM 'downunder'. Med Teach 2010; 32: 161-3.
21. BuAli WH, Khan AS, Al-Qahtani MH, Aldossary S. Evaluation of hospital-learning environment for pediatric residency in eastern region of Saudi Arabia. J Educ Eval Health Prof 2015;12: 14
22. Khan AA, Shaikh S, Ahmed Z, Zafar M, Anjum MU, Tahir A, et al. Quality of post graduate medical training in public and private tertiary care hospitals of Karachi. J Postgrad Med Inst 2014; 28: 13-8.
This journal is a member of and subscribes to the principles of the Committee on Publication Ethics.