Background: In educational hospitals, in addition to providing care, emergency medicine department plays an important role in providing education and increasing the quality of care and education. This study was an endeavor to evaluate the performance of emergency medicine department in different areas such as education, service provision and interaction with other departments.
Methods: This was a descriptive cross-sectional study performed as a 360-degree evaluation. The stakeholders taking part in this evaluation included residents, interns, attendants, administrators, nurses, and patients. Particular questionnaires for each group was developed based on the data achieved in the focus groups. Structured interviews were used to collect data from patients. Patients\' medical files were investigated in order to extract performance indices of emergency medicine department. Data was analyzed using descriptive statistics, chi-square, and t-tests by SPSS.
Results: Based on the achieved data, patients\' waiting time for the first visit of the emergency physician was 13.18 minutes in average. The mean of outpatients\' waiting time (29.82 minutes) was higher than that of patients requiring hospitalization. In most cases (79%), similar first and final diagnoses were made. In addition, 93.5% of patients hospitalized in the emergency ward were highly satisfied with services received from the specialists while 6.5%were averagely satisfied. The highest performance scores to the emergency medicine department were given by the interns with the mean of 4.24 out of 5 performance.
Conclusion: The results of the emergency department evaluation by different groups as well as indices showing the performance of this department revealed its overall performance in educational, clinical, and interactional areas as good. The findings of this study can be applied in Alzahra Hospital to promote the positive points and amend the negative points.
Keywords: 360-Degree Evaluation, Performance Assessment, Emergency Medicine (JPMA 62: S-13; 2012).
Emergency medicine is a clinical specialty which evaluates, resuscitates, diagnoses, and provides care for emergency patients. This specialty field encompasses basic sciences information, clinical knowledge, and required skills for diagnosis, decision making, and treatment of the patients referring to emergency wards. This field also manages emergency medicine departments and unexpected events in hospitals. The specialists in this field should also be able to act as an effective member in medical team along with other specialists and also with other medical teams in pre-hospital emergency. With regard to the features of emergency ward and the value of time in delivering care to emergency patients, there is a need to specialist physicians who are familiar with urgent cases and are able to manage crisis in emergency wards.1-5 The specialists of this field are settled in emergency wards and provide services related to their specialty.6
Considering this field as a new established specialty in Iranian hospitals, it makes sense to expect some problems to occur at the beginning, just like many other changes which may primarily lead to some negative reactions. Still, the question for many stakeholders would be if the establishment of emergency medicine departments could improve the quality of services and medical education in emergency wards or not. In other words, the emergency medicine departments need to be evaluated. In fact, if a system is to provide high quality health care services to people, it should consider the viewpoints of people studying in such situation along with those receiving the services.7-10 In addition,trying to evaluate the performance of a department using the viewpoints of different stake holders may have high costs but it will be cost effective in long-term.11,12
In response to the existing problems in emergency wards, such as the increasing rate of hospital discharge with personal authorization, problems related to patient discharge or transfer, lack of immediate treatment of emergency patients, lack of immediate access to emergency medicine specialists, the department of emergency medicine was established in Alzahra Hospital, Isfahan, Iran. However, due to the interference of tasks with other departments and occurrence of cases which had led to dissatisfaction among some people and departments, the performance evaluation of the emergency department could clarify some issues. On the other hand, evaluation process is one of the essential elements of each program. It is used to judge the value of people\'s achievements or the success of activities. In other words, every active program should be evaluated continuously. Therefore, formative assessment is an important part of running educational programs. As a result, weaknesses of the program will be identified and addressed or eliminated before widespread implementation. In fact, the final aim of performing formative evaluations is modification and expansion of the program. Therefore, in order to improve the performance of emergency medicine department in Alzahra Hospital at the beginning of its establishment and especially its interaction with other departments, we decided to evaluate the performance of this department based on viewpoints of different stakeholders in education, treatment, and some other domains.
This study was conducted to evaluate the performance of emergency medicine department in Alzahra Hospital using the 360-degree evaluation method. This method mostly estimates interpersonal and communicational abilities, professional behaviors, and some viewpoints about patient care, and working manners of individuals. Therefore, rating forms are usually filled out by experts, colleagues of the same or lower levels, and patients and their relatives. In most evaluations of such kind, a questionnaire is used to obtain data about individuals\' manner of performance in different areas such as team work, communication, patient care, and decision making.
In our study, an evaluation committee initially configured the evaluation process of the study. During a number of sessions, the committee members determined the stakeholders of emergency medicine program and their significance in evaluation of this program. The stakeholders included faculty members, clinical specialists of different departments of the hospital, residents, nurses, interns and patients. Focus group sessions were run separately to develop a questionnaire for each stakeholder. There were similar items in these questionnaires, so, the items covered in these questionnaires were determined. These domains included interaction with other stakeholders (interactional), providing care and treatment (therapeutic), providing education for different learners (educational), and consequences of establishment of emergency medicine department in the emergency medicine ward of Alzahra Hospital. The validity and reliability of the questionnaires were confirmed by field experts and Cronbach\'s alpha, respectively. Each questionnaire included a cover letter explaining about the aim of the study and how to fill the items about the performance of the emergency medicine department, and items asking about demographic data. The questionnaires were delivered to the study subjects in person. Two weeks were considered for returning the questionnaires and maximum effort was put to reclaim the questionnaires.
Document investigation and semi-structured interviews were used in order to study the indices representing the performance of emergency medicine members. The indices included patients\' waiting time, similarity of primary and final diagnosis, and duration of patients\' hospitalization. Therefore, a form was designed to gather such data from patients\' files and other documents. Moreover, interviews with hospital administrators and members of the emergency medicine department were conducted to complete the required information.
In order to study the viewpoints of patients about services they received in the emergency ward, structured interviews using a pre-designed questionnaire were conducted. After collecting information, the achieved data was entered to SPSS and then analyzed using descriptive statistics, t-test, and chi-square test.
The stakeholders of the program were considered to be faculty members of all departments (n=23), residents (n=32), nurses (n=36), interns (n=36), and patients (n=31).The majority of the studied faculty members and patients were male (95.7% and 61.3%, respectively) while most of the studied interns, nurses, and residents were female (62.9%, 83.3%, and 53.1%, respectively).The mean score of total performance of the emergency medicine teachers was 2.80 (out of 5) according to residents of other departments. The highest mean score given by these residents belonged to interactional performance of the emergency medicine teachers (2.85±0.74). The highest mean performance score of emergency medicine teachers was given by interns (4.35±0.53). The highest mean score of performance given by faculty members of other departments belonged to the clinical performance of the emergency medicine teachers (3.19±0.59). The highest mean score given by nurses belonged to interactional performance of the emergency medicine (3.87±0.59). These results are summarized in Table-1.
The empty cells show that there were no items in that domain in the related questionnaire.
The viewpoints of patients about services they received at the emergency medicine department are summarized in Table-2.
In all the mentioned items, the mean performance scores of the emergency medicine teachers was higher than three. The highest score (mean=4.83) belonged to respectful behavior of the emergency medicine teachers toward patients and their accompaniers.
The findings obtained by investigating the existing documents and files are presented in Tables-3 and 4.
Table-4 shows the similarity of initial and final diagnoses. In the majority of cases (79%) similar initial and final diagnoses were made. Duration of patients\' hospitalization in the emergency ward was 257.98 minutes in average.
This study was performed to identify the existing problems, and make some modifications in the emergency medicine program, if necessary. We decided to evaluate the performance of this group in education, treatment, and other domains using the 360-degree evaluation method. We chose this evaluation method because it enables us to comprehensively assess the performance of the department in all domains. Moreover, since a clinical teacher has different roles, all aspects of his performance must be evaluated.5 The emergency ward is normally stressful due to the nature of emergency patients. It is also usually very busy and crowded. Therefore, the performance of clinical teachers in such situations, as a teacher and as a care provider, would be of special importance.15 In his role as a teacher, a clinical educator must be familiar with principles of education, teaching and learning strategies, and evaluation in clinical settings. On the other hand, this clinical teacher needs to provide health care to patients. Therefore, in addition to getting familiar with patient care principles, other professional qualifications such as interaction with patients, students, and colleagues would be highlighted. Thus, all these items should be considered for a comprehensive evaluation of the performance of a clinical teacher.13,14
The interns and nurses in this study confirmed the performance of the emergency medicine department staff to be great in interaction with patients and others. Residents of other departments gave the lowest scores to the interactional performance of emergency medicine department. Nurses are highly interacted with members of the emergency medicine ward in Alzahra Hospital and could provide relevant opinions about the performance of the department. The effective interaction between nursing personnel and physicians in the emergency ward leads to further quality improvement of the services. This is an indisputable professional requirement for personnel of this ward which proves the necessity to enhance communicational and interactional skills.12,15
The highest performance scores were given by interns and nurses. The reason might be the fact that emergency ward is an environment in which interns encounter patients and earn experiences which prepares them for their future career. Therefore, their satisfaction with clinical teachers of the emergency medicine department enhances. The lowest clinical performance scores to the emergency department members were given by the residents of other departments. The reason could be different approaches mentioned in references of emergency medicine department compared to other departments. In fact, since each department prefers its own references, such diversities might have resulted in contemplations such as failure to comply to standards of care.
In addition, interns assessed the educational performance of emergency medicine teachers as very high. Residents of other departments however, have not appraised it much. Since the majority of educational activities of emergency medicine members are related to interns, in this case, their viewpoint about the educational performance could be prior to others\' viewpoints. The total performance of the emergency medicine teachers enjoyed the highest mean scores respectively by interns, nurses, teachers, and finally residents. In general, the results showed that interns had given high scores and residents had given low scores to the performance of the emergency medicine teachers.
Residents and nurses absolutely agreed that establishment of emergency medicine department has imposed extra financial pressure to patients and increased the charges of hospitalization for emergency patients. The reason might be the codes documented in patient files by the emergency medicine which increase hospitalization charges in the emergency ward of Alzahra Hospital. The answer to this ambiguity has been mentioned in administrative responses.
Unlike the teachers, most residents claimed that emergency medicine department refers patients without initial diagnosis. Investigating the documents showed the majority of files (79%) to enjoy similar primary and final diagnoses. On the other hand, residents concurred that most of the actions made by residents of other departments in the emergency ward are recorded by the emergency medicine teachers as the performance of emergency medicine department. The teachers also confirmed this claim. All groups including residents, teachers, nurses, and interns agreed that presence of the emergency medicine department in Alzahra Hospital to be necessary.
Based on the findings of this study, 93.5% of patients were highly satisfied with the quality of services offered by the emergency medicine department. All patients with previous reference to Alzahra emergency ward confirmed that the quality of services was improved compared to their previous experiences. The highest mean score (4.83±0.45) belonged to respectful manner of emergency medicine teachers with patients and their accompaniers. The viewpoints of patients about most of the items in the questionnaire were positive. All these items demonstrated the quality improvement in service offer in Alzahra emergency ward after the establishment of emergency medicine department.
According to Table-3, the mean waiting time for the first visit of the emergency medicine physician was 13.18 minutes. In currently existing references, the waiting time for receiving services resulting in physician prescription is between 0 to 120 minutes considering the vital signs of the patient. Jabbari et al. reported this period as 35.82 minutes.15 This index also reveals the progress in service providing in the emergency ward. Undoubtedly, the reduction of patients\' waiting time in the emergency ward affects many aspects of health care services. The results of the current study determined the strengths of the emergency medicine department as increasing patients\' satisfaction with emergency services, paying quick attention to patients requiring emergency actions, creating peace of mind in patients due to the presence of specialist in the emergency ward, reducing patients\' waiting time, increasing the financial income of the hospital, and improving hospital and treatment output. These results also revealed some weak points such as augmentation of financial pressure on patient, increase of hospitalization charges for emergency patients, enhancement in accepting patients with complicated conditions referred by other health care centers which were rejected by other departments due to logical reasons. The results of the study could be applied for further planning in implementation of the emergency medicine program in Alzahra Hospital. Further studies are recommended to determine the effects of implementing of the suggested strategies in this research such as developing guidelines, running communicational skills courses for different departments, and evaluating other departments as well.
We should be thankful to the administrators of Alzahra Hospital, department of emergency medicine, nursing personnel of Alzahra emergency ward, Chancellery of Education and Research of Isfahan University of Medical Sciences, Isfahan Medical Education Development Center, and others who helped us in running this project.
1. Ministry of Health and Medical Education. Curriculum and regulations for Emergency medicine specialty. (Online) (Cited 2011 Feb 10). Available from URL: http://cgme.behdasht.gov.ir/index.aspx?siteid=264&pageid=22758. [in Persian].
2. Garcia EA, Likourezos A, Ramsay C, Hoffman S, Niles C, Pearl-Davis M, et al. Evaluation of emergency medicine community educational program. West J Emerg Med 2010; 11: 416-8.
3. Bawden J, Manouchehri N, Villa-Roel C, Grafstein E, Rowe BH. Important returns on investment: an evaluation of a national research grants competition in emergency medicine. CJEM 2010; 12: 33-8.
4. Moharari RS, Soleymani HA, Nejati A, Rezaeefar A, Khashayar P, Meysamie AP. Evaluation of morning report in an emergency medicine department. Emerg Med J 2010; 27: 32-6.
5. Colletti JE, Flottemesch TJ, O\'Connell TA, Ankel FK, Asplin BR. Developing a standardized faculty evaluation in an emergency medicine residency. J Emerg Med 2010; 39: 662-8.
6. Eastaugh S. Hospital nurse productivity. J Health Care Finance 2002; 29: 13-22.
7. McNeese-Smith DK. Staff Nurse view of their productivity and non productivity. Health Care Manage Rev 2001; 26: 7-19.
8. Witzke DB, Sanders AB. The development and evaluation of a geriatric emergency medicine curriculum. The SAEM Geriatric Emergency Medicine Task Force. Acad Emerg Med 1997; 4: 219-22.
9. Valenzuela TD, Criss EA, Spaite DW, Meislin HW. Evaluation of EMS management training offered during emergency medicine residency training. Ann Emerg Med 1989; 18: 812-4.
10. Binder L, Nelson B, Smith D, Glass B, Haynes J, Wainscott M. Development, implementation, and evaluation of a medical Spanish curriculum for an emergency medicine residency program. J Emerg Med 1988; 6: 439-41.
11. Beeson MS, Wilkinson LF. Evaluation of clinical performances of emergency medicine residents using standardized patients. Acad Med 1999; 74: 202.
12. Woloschuk W, Gorsche R, Kay D, Hnatuik J, Kohlhammer M, Betzner M. General emergency medicine skills (GEMS) training for rural physicians: evaluation of a pilot project. Cah Sociol Demogr Med 2007; 47: 491-510.
13. Colletti JE, Flottemesch TJ, O\'Connell TA, Ankel FK, Asplin BR. Developing a standardized faculty evaluation in an emergency medicine residency. J Emerg Med 2010; 39: 662-8.
14. Mattu A. Foreword. Emergency medicine and patient evaluation. Emerg Med Clin North Am 2010; 28: xiii-xxiv.
15. Jabbari A, Ahmadi O, Jafarian M, Khorasani E, Ghafary M, Majlesi M. Investigating waiting time for service and presenting proper strategies in Alzahra Emergency in 2009, [Dissertation]. Isfahan, Iran: School of Management and Medical Information, Isfahan University of Medical Sciences; 2010.