Junaid Sarfraz Khan ( Department of Examinations, University of Health Sciences, Lahore. )
Saima Tabasum ( Department of Examinations, University of Health Sciences, Lahore. )
Usman Khalil Yousafzai ( University of Management and Technology, Johar Town, Lahore. )
Mehreen Fatima ( Department of Examinations, University of Health Sciences, Lahore. )
Objectives: To validate DREEM in medical education environment of Punjab, Pakistan.
Methods: The DREEM questionnaire was anonymously collected from Final year Baccalaureate of Medicine; Baccalaureate of Surgery students in the private and public medical colleges affiliated with the University of Health Sciences, Lahore. Data was analyzed using Principal Component Analysis with Varimax Rotation.
Results: The response rate was 84.14 %. The average DREEM score was 125. Confirmatory and Exploratory Factor Analysis was applied under the conditions of eigenvalues>1 and loadings > 0.3. In CONFIRMATORY FACTOR ANALYSIS, Five components were extracted accounting for 40.10% of variance and in EXPLORATORY FACTOR ANALYSIS, Ten components were extracted accounting for 52.33% of variance. Total 50 items had internal consistency reliability of 0.91 (Cronbach\\\'s Alpha). The value of Spearman-Brown was 0.868 showing the reliability of the analysis. In both analyses the subscales produced were sensible but the mismatch from the original was largely due to the English-Pakistan contextual and cultural differences.
Conclusion: DREEM is a generic instrument that will do well with regional modifications to suit individual, contextual and cultural settings.
Keywords: Validation, Dundee Ready Educational Environment Measure, Cerebral ischaemia, Reperfusion, Pakistan (JPMA 61:885; 2011).
In addition to the documented curriculum, students and teachers both become aware of the \\\'educational environment\\\' or \\\'climate\\\' of the institution.1 Several research groups over the years have attempted to identify and quantify the presence and impact of rather intangible aspects of a learning environment: the climate, or atmosphere, or ethos, tone or ambience, the culture or personality of the institution.1,2
Roff et al. 1997,3 developed and validated a generic, multicultural, multidimensional instrument for measuring the Learning Environment in Medical and Allied institutions that is being used worldwide and has been translated in over a dozen languages. Getulio et al. 2005,4 described the Dundee Ready Education Environment Measure (DREEM) as a robust instrument designed to measure student\\\'s perceptions about dimensions of the Learning Environment. It is based on a five point Likert scale measuring five dimensions of the environment, namely, \\\'Perceptions of Learning\\\' (PoL), \\\'Perceptions of teachers\\\' (PoT), \\\'Perceptions of Atmosphere\\\' (PoA), \\\'Academic Self-Perception\\\' (ASP) and \\\'Social Self-Perceptions\\\' (SSP). Higher the total and dimensional scores, the better the environment. Roff 2005,5 described that on an average the DREEM score varied from 78 to 139 out of 200 and the questionnaire had the ability to identify the strengths and weaknesses of particular institutions.
In our study, we set out to validate DREEM in our unique culturally-specific medical education environment in Punjab, Pakistan, something that had not been done in the past. The Population was limited to all the students of the Final Year (Fifth Year) Baccalaureate of Medicine; Baccalaureate of Surgery (M.B, B.S) in the Medical Colleges affiliated with the University of Health Sciences (UHS), Lahore in 2008. The reason for including final year M.B, B.S students was the experience of all the years of education environment. Approval from the Ethics Committee, UHS was obtained. The research study was conducted in SIX Public and TWO Private Medical Colleges. There were 1612 Students in Final Year M.B,B.S. in these institutions, 1406 in Public and 206 in private sector.
Sample size as calculated as 10 samples per item, using one of the rule of Factor Analysis. Multi staged sampling was done. Firstly population was divided into Private and Public Medical Colleges and secondly each medical college was treated as different stratum and then simple random sampling was done from each medical college.
A pilot study was conducted to estimate the volume of sample. DREEM questionnaire was collected anonymously from the students. The given data was analyzed using Non-parametric tests in Microsoft Excel 2003 and Statistical Package for Social Sciences (SPSS) v.16 using Principal Component Analysis with Varimax Rotation. A p<0.05 was considered significant.
Items were neither deleted nor added, their original (randomly arranged) order was not changed and the subscales were not rearranged. Standard instrument validation methodology was followed.6,7 To see whether different factor clustering operated in our data, we also performed exploratory factor analysis (EFA) under the conditions of eigenvalues >1 and loadings > 0.3. Questions whose deletion increased the overall alpha, and those that loaded less than 0.3 on the expected factor, loaded on two factors or were grouped within an unexpected factor were thoroughly examined.
Four Hundred and Ninety Eight questionnaires were distributed and 419 (219 females, 193 males and 7 undeclared) responded (84.14%; 88.48% in the private sector and 79.22% in the public sector). The overall average age was 23.5±2.51 years (24.1±2.61 in private and 23.0±2.09 in public), with comparable male, female age (23.9±2.16 and 23.2±2.27 respectively). The average DREEM score was 125±2.05 (137±2.01 in private and 115±2.07 in public).
There were 87 missing responses in 29 questionnaires. In each of these 29 questionnaires, the number of missing responses was less than 10%. The majority of these missing responses, 37% of all were for question No. 31, where the students perhaps did not really understand the meaning of \\\'Empathy\\\'. These missing responses were tabulated as \\\'Uncertain\\\'. There were in total 35 multiple responses in eleven questionnaires and in each case the higher response was accepted.
The 20863 non-missing values ranged from \\\'strongly agree\\\' 4679 (22.43%), \\\'agree\\\' 7895 (37.84%), \\\'uncertain\\\' 3253 (15.59%), disagree 3358 (16.09%), to \\\'strongly disagree\\\' 1678 (8.04%).
The value of Kaiser-Meyer-Olkin (KMO) was 0.929, where Bartlett\\\'s Test of Sphericity was significant (p-value <0.05). In Exploratory Factor Analysis (EFA), ten components were extracted with Kaiser Eigenvalues greater than 1 accounting for 52.33% of variance. Total 50 items (excluded missing values) had internal consistency reliability of 0.91 (Cronbach\\\'s Alpha). The value of Spearman-Brown was 0.868 showing the reliability of the analysis.
The results of EFA are presented in Table.
No significant difference was found between the DREEM scores of Private and Public Medical Colleges (P < 0.05).
The aim of this study was to validate DREEM in our unique cultural medical educational environment. The internal consistency reliability of 0.91 (Cronbach\\\'s Alpha) in our study is much higher than the 0.70 or 0.80 thresholds generally considered acceptable for scales.7 Even though the original DREEM was validated in our regional context the Cronbach\\\'s Alpha was similar to published studies of the DREEM translations in other languages as of Primparyon et al9: 0.91; Mayya & Roff:10 0.92; de Oliveira Filho:11 0.93; Riquelme et al. 12: 0.91.
The original DREEM described only five subscales.3 In our study the exploratory factor analysis yielded ten subscales. Similarly, the confirmatory factor analysis with five subscales contained items differently than the original subscales. In both analyses the subscales produced were sensible but the mismatch from the original was largely due to the English-Pakistan cultural differences. This is consistent with other validation studies with or without translation of original English DREEM, in which the instrument was found to be reliable but a mismatch in item-loading into original components.13 This perhaps, challenges the \\\'culture-free\\\' claim of the instrument.1
No statistically significant differences were identified between the Private and the Public Medical Institutions. The combined average DREEM Score of 125 was comparable with other studies reported from elsewhere.5 The large size of our sample and a high response rate enhances the validity of this study. The fact that no differences were identified between the two sectors reinforces the claim that the DREEM is a sensitive instrument capable of profiling an institution\\\'s strengths and weaknesses as perceived by a particular cohort of students at that point in time.5
Sue Roff claims that the instrument can be modified to increase its sensitivity and reliability for region-specific \\\'Profiling\\\'.5 The fact that the original factorial structure was not maintained in exploratory factor analysis is supported by others.14
The majority of the missing values were in item no. \\\'31\\\'. It is possible that the students had difficulty in putting the term "empathy" into their cultural perspective. It would therefore be sensible to provide translations of difficult terms and terminologies to help respondents clearly apply each item to their individual settings.8
DREEM is a valid, reliable and sensitive instrument capable of measuring the educational environment worldwide. It is a generic instrument that will do well with regional modifications to suit individual, contextual and cultural settings.
1.Roff S, McAleer S. What is Educational Climate? Med Teacher 2001; 23: 333-4.
2.Roff S, McAleer S, Ifere OS, Bhattacharya S. A global diagnostic tool for measuring educational environment: comparing Nigeria and Nepal. Med Teacher 2001; 23: 378-82.
3.Roff S, McAleer S, Harden RM, Al-Qahtani M, Ahmed AU, Deza H, et al. Development and validation of the Dundee Ready Education Environment Measure (DREEM). Med Teacher 1997; 19: 295-9.
4.Getu´ LR, de Oliveira F, Joaquim EV, Leonardo S. Psychometric properties of the Dundee Ready Educational Environment Measure (DREEM) applied to medical residents. Med Teacher 2005; 27: 343-7.
5.Roff S. The Dundee Ready Educational Environment Measure (DREEM) - a generic instrument for measuring students\\\' perceptions of undergraduate health professions curricula. Med Teacher 2005; 27: 322-5.
6.Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. England: John Willey & Sons, 2000.
7.Streiner DL, Norman GR. Health Measurement Scales - a practical guide to their development and use. 4th ed. Oxford: Oxford Univ Press, 2008.
8.Dimoliatis IDK, Vasilaki E, Anastassopoulos P, Ioannidis JPA, Roff S. Validation of the Greek Translation of the Dundee Ready Education Environment Measure (DREEM). 2010; 23: 1-16.
9.Primparyon P, Roff S, McAleer S, Poonchai B, Pemba S. Educational environment, student approaches to learning and academic achievement in a Thai nursing school. Med Teacher 2000; 22: 359-65.
10.Mayya S, Roff S. Students\\\' Perceptions of Educational Environment: A Comparison of Academic Achievers and Under-Achievers at Kasturba Medical College, India. Educ Health 2004; 17: 280-91.
11.de Oliveira Filho GR, Vieira JE, Schonhorst L. Psychometric properties of the Dundee Ready Educational Environment Measure (DREEM) applied to medical residents. Med Teacher 2005; 27: 343-7.
12.Riquelme A, Oporto M, Oporto J, Mendez J, Viviani P, Salech F, Chianale J, Moreno R, Sanchez I. Measuring Students\\\' Perceptions of the Educational Climate of the New Curriculum at the Pontificia Universidad Catolica de Chile: Performance of the Spanish Translation of the Dundee Ready Education Environment Measure (DREEM). Education for Health. (Online) 2009 (Cited 2011 March 20). Available from URL: http://www.educationforhealth.net/articles/subviewnew.asp?ArticleID=112.
13.Varma R, Tiyagi E, Gupta J. 2005. Determining the quality of educational climate across multiple undergraduate teaching sites using the DREEM inventory. BMC Med Educ 2005; 5: 8.
14.Mok MMC, Flynn M. Establishing longitudinal factorial construct validity of the quality of school life scale for secondary students. J Appl Meas 2002; 3: 400-20.