Zaeem-ul-Haq ( Johns Hopkins University Center for Communication Programs, Islamabad )
Farrukh Qureshi ( World Health Organization, EMRO, Cairo, Egypt )
Assad Hafeez ( National Program for Family Planning and Primary Health Care, Islamabad )
Shamsa Zafar ( Islamic International Medical Complex, Islamabad )
Bile Khalif Mohamud ( World Health Organization, Islamabad, Pakistan )
David P Southall ( Childhealth Advocacy International, Nottingham, UK )
To assess the motivational level and use of skills amongst the graduates of training courses on maternal and newborn healthcare in emergency settings. Methods:
Postal survey involving Doctors and Nurses from public sector hospitals who attended training courses on Essential Surgical Skills with emphasis on Emergency Maternal and Child Health.Results:
Ninety percent of respondents reported the use of acquired skills and the structured Airway, Breathing, Circulation (ABC) approach in handling emergencies. Instances were described where this approach helped to improve practice, simplify treatment and save lives in all age groups, especially mothers and newborns. Lack of equipment and lack of support from higher levels were the common barriers yet many graduates managed to obtain equipment and 81% of respondents reported that these training courses had resulted in better availability or use of supplies at their facilities. Efforts were made by the graduates for institutionalizing the trainings and discussions (88%) were found to be the most common mode followed by formal teaching of the skills (66%). Hand washing was the most commonly used skill followed by the use of bag valve and mask resuscitation in the newborn at birth. Those catering for neonates expressed their confidence in obstetric procedures, while obstetricians described themselves as being self-assured now in neonatal resuscitation. Conclusions:
The introduction of a structured training programme in a resource-constrained healthcare system has improved the emergency management of patients. ESS-EMCH trainings should be incorporated into the district health systems to enhance emergency care handling capacities of healthcare staff, to help them contribute to achieve the MDGs and also for sustainable improvement in maternal, neonatal and child healthcare (JPMA 59:22; 2009).
There has been considerable discussion recently on whether some countries will be able to achieve targets set under Millennium Development Goals (MDGs) 4 and 5.1,2
These MDGs symbolize the pledge to improve maternal and child health.3
Meanwhile more than 500,000 women continue to die each year during pregnancy and childbirth4
and more than four million babies suffer the same within 28 days of their birth.5
Most of these deaths occur in the resource-constrained countries of Sub-Saharan Africa and South Asia.6,7
Emphasis has been given on the provision of universal access to clinical services in addition to preventive interventions, in order to try and reduce these unethical figures.8
At the same time, the quality of the clinical services provided is crucial. An important strategy is effective training to improve the skills and morale of the professionals involved in delivering clinical care at all levels.9
Effectiveness of training is sometimes assessed by obtaining the feedback from participants, or from analysis of pre and post-training tests.10
There is a scarcity of evidence on the intermediate and long-term effectiveness of training, especially when undertaken in poorly resourced countries.
The Essential Surgical Skills with emphasis on Emergency Maternal and Child Health (ESS-EMCH) training programme is being conducted by Childhealth Advocacy International (CAI) in Pakistan and The Gambia.11
The Advanced Life Support Group-UK (ALSG-UK) and World Health Organization provide technical support for the training and the programme has been endorsed as a first ALSG-UK certified training course targeted specifically for developing countries.12
Being the first of its kind in a developing country, the programme has set out to assess the impact of its training programme in Pakistan.
This intermediary impact evaluation, conducted in the first quarter of 2007, assessed the changes in knowledge, attitudes and practice of the trainees. Here findings from a postal survey conducted to assess the motivational level and use of skills amongst the trainees in their practice settings are presented. This survey was conducted on those health professionals who attended the five-day ESS-EMCH courses during January 2004- December 2006.
Subjects and Methods
Modular ESS-EMCH training programmes have been developed to enhance the competency of healthcare providers of all levels involved in treating emergencies in pregnancy, infancy and childhood in resource poor countries. In Pakistan, doctors and nurses were invited through local health authorities to attend the courses held in selected districts of all four provinces. Participants learnt the skills necessary to save lives during the "golden hours" of emergency situations. Using a structured (ABC) approach, they were taught how to maintain the airway, breathing and circulation and save lives with the help of both manual and pharmacological interventions in a step-wise and structured manner.
After becoming an ALSG-certified EMCH worker, they were expected to use these skills at their workplace. A total of 234 health providers became accreditted workers following 5 day ESS-EMCH courses held during the period 2004-2006.
To assess their motivational level in the post-training phase and the extent to which they were using the skills learned, a postal survey was conducted. This survey was part of an intermediate impact evaluation of the ESS-ECMH programme designed considering Kirkpatrick's four levels of programme evaluation criteria.13,14
This present survey corresponded to level 3 of Kirkpatrick's evaluation framework. All health workers who attended the 5-day ESS-EMCH course were considered as the study population. A total of 184 course participants were included in the initial contact list out of the 234 attendees after discarding those who had left the country or had changed their addresses as available in the data bank. The final number included 152 doctors and 32 nurses belonging to primary, secondary and tertiary level public sector hospitals.
The local ethics committee of CAI's operational research wing approved this study.
A semi-structured, 15-point questionnaire in the English language was devised in consultation with the course team and a behaviour change communication specialist to assess attitudes and behaviour change among the practitioners. Five dimensions of this change were captured through responses to the questionnaire. The first two dealt with self confidence and self efficacy regarding their ability to perform and the second documented the actual performance levels. The third part of the questionnaire addressed the barriers perceived or faced by the trainees at their workplace regarding implementation of the learning. The final two parts captured respondents' efforts towards institutionalization of the trainings and their experience in terms of patient outcomes by making use of skills (Table 1). [(T1)] The questionnaire was pre-tested in Islamabad before beginning the survey. The final version was mailed to all identified course participants along with a covering letter and a stamped self-addressed envelope for return mail. The letter described the purpose of evaluation which was also explained on a telephone call contact made one week after mailing the packet. The participants were asked to send back their responses within one month. A reminder telephone call was made after three weeks had passed to those who had not complied up to that time and a further time period of two more weeks was given to improve compliance.
Participants mailed completed questionnaires back to the CAI Pakistan office in Islamabad. None of the questionnaires received were improperly completed, hence no form was rejected. Completed forms were manually edited and coded for computerized data entry. Statistical software SPSS, version 10, was selected for complete data management, that is from data entry to data cleaning and analysis. Correctness of entered data was ascertained by verifying entries of 10 percent of randomly selected forms.
The postal questionnaire was responded to by 100 participants, showing a response rate of 54%. The response was divided between doctors and nurses (81:19) in the same proportion as was their attendance as course participants. The responses were received from all geographical locations, with the province of Punjab having the highest number of responses (28%, n=28), followed by NWFP (27%, n=27). Participants from all ten courses responded. The quality of responses was variable with some making just yes/no answers while others gave a detailed account of their experience. On average, 75% of respondents also provided subjective explanations to queries in the questionnaire. Adverse comments about the courses were usually provided by those in non-clinical or academic positions. Among the responders, 41% had received additional maternal and child health training during the previous two years. There were no differences in observations between those who had attended single or multiple training in maternal and child health.
Most of the respondents answered in the affirmative to questions regarding the use of skills acquired, with 95% reporting the frequency of use in the range of 'occasional use' to 'almost daily' (Figure 1). Ninety percent reported using the taught structured ABC approach in their practice. [(Fig1)] Instances were described where a structured approach helped to improve practice, simplify treatment and save lives in all age groups, especially in pregnancy and in newborn infants. One paediatrician working in a refugee camp commented as follows: "I have conducted five deliveries in a tent and that was possible for me only after attending this course." A doctor responsible for obstetric services wrote:
"By the structured approach of newborn resuscitation, my practice has much improved. I have actually saved 4 babies by using structured approach, especially by using ambu bag in a right way at the right time."
Almost 48% of respondents faced barriers in the implementation of what they had learnt. Questions were asked on the five most important barriers perceived or actually faced. Lack of equipment and lack of support from higher levels were the two most commonly encountered barriers (Figure 2). However, there were also respondents [(Fig2)] who used their professional position and advocacy skills to acquire the basic emergency equipment revealed as essential by their training. Overall, 81% of respondents reported that these training courses had resulted in better availability or use of supplies at their facilities.
Questions also captured the respondents' efforts towards institutionalization of their training. Discussions (88%) were the most common mode of dissemination followed by formal teaching of the skills (66%) (Table 1).
Most participants suggested extending the programme into the curricula and training of doctors and paramedics at the institutional level.
One participant wrote: "I am convinced that I can bring a change by convincing people that life-support (structured approach) can save lives." Another commented: "The training courses are helpful not only for specialist doctors but also for paramedics and nursing staff, so should be extended to other medical and paramedical staff."
Outcomes were assessed through questions on their ability to treat emergencies. Those caring for neonates expressed their confidence in obstetric procedures, while obstetricians described themselves as being self-assured now in neonatal resuscitation. Instances were described where the use of skills had saved lives or addressed disease or trauma related complications in 85% of responses (Table 1). Figure 3 describes the most frequently [(Fig3)] used skills reported. Hand washing was the most commonly used skill followed by the use of bag valve and mask resuscitation in the newborn at birth. One doctor stated: "I did (long saphenous vein) cut-downs and chest drains personally in the earthquake (struck) areas." Another wrote: "A patient was delivered at home, and was brought to my clinic with a vaginal and cervical tear …she was nearly dead…her level of consciousness was also gone (altered). She was my first patient whose life I saved very confidently, without any fear."
This study provides evidence that the first time introduction of a structured training programme in a resource-constrained healthcare system has improved the emergency management of patients. Graduates are applying the knowledge and skills learnt during the course in their clinical practice and their confidence in handling acute maternal, neonatal and childhood emergencies has improved. The skills taught are being utilized both by doctors and nurses showing the usefulness of training for both groups of professionals. In a country with high maternal, neonatal and infant mortality, it is heartening to note that the pregnant mothers, newborns and children are the primary beneficiaries of these structured life-support courses.
Participants' views about the institutionalization of this learning and attempts to overcome hurdles within the systems of their facility are also encouraging. They were not given a 'training of trainers' course (available as part of the ALSG's training within the ESS-EMCH programme), yet the sharing of knowledge at their facilities through discussions and formal teaching represents a real motivational change. Similarly, more than two thirds reported improved availability and use of emergency drugs and supplies at their facility despite the fact that about half faced some barriers on the way. In a country where perceived economical and hierarchical barriers are strong and first care level workers don't enjoy decision making positions, this overcoming of barriers denotes a significant change of attitude among the trainees.
The limitation of this study is that it assesses only the intermediary outcomes of the training component of ESS-EMCH that is learning or behaviour change. Further evaluations are needed to identify confounding factors and to determine the long-term effects of this programme on improving maternal, neonatal and child health indicators. We are also aware of the limitations of a possible reporting bias from postal surveys; however this approach may have helped to avoid bias due to interviewer-respondent interactions that can occur with direct interviewing.
There is a scarcity of data on the evaluation of training courses especially from poorly resourced countries. Those conducted in the developed countries have predominantly assessed the immediate effects. In a detailed systematic review of 217 evaluation studies from the developed world, undertaken by the Learning and Teaching Support Network, only 8% had a follow-up component, the rest usually being made at a single time-point, post intervention or from before and after studies.10
Similarly a review by Jordan found course evaluations in the developed countries being focused on knowledge, attitudes and personal development rather than on practice.15
Such evaluations consider single outcome measures, which might serve as outcome indicators of the training, but not necessarily translate into 'proxy' data of effectiveness in practice. We aimed to have an objective account of the impact of the ESS-EMCH courses in practice through this evaluation in Pakistan. The training has been received well by different cadres of health workers, has been able to mobilize an attitude change of participants in handling emergencies, and the skills are being used frequently in practice. Importantly, it shows that the ESS EMCH training courses, which are practical and "hands-on", are successful in changing the attitudes and practice of health workers in health systems having limited resources.
Health providers dealing with most of the obstetric, neonatal and child emergencies in poorly resourced countries are non-specialists. The main focus of ESS-EMCH trainings is on these 'generalists' including general practitioners and nurses. The equivalent utilization of taught skills by all cadres of health providers reiterates the programme's philosophy, which is to impart emergency resuscitation skills irrespective of the existing knowledge and occupational status. The findings suggest that "hands-on skills based" training should be a regular component of all pre and in-service Continuing Medical Education (CME).
Pakistan is among countries where poor maternal, neonatal and child health indicators make a significant contribution towards the overall dismal healthcare situation. The Bellagio child survival series reported Pakistan to be amongst the countries having the highest ratio of perinatal and neonatal mortality.6
Causes include birth asphyxia, preterm delivery, tetanus and sepsis.16
Previously conducted national studies also present similar etiological factors.17
The ESS-EMCH trainings have resulted in improved management of these obstetric, neonatal and child health emergency situations in practice. Our findings support the further expansion and large-scale implementation of these training programs in Pakistan and other poorly resourced countries. We recommend that ESS-EMCH trainings should be incorporated into the district health systems to enhance emergency care handling capacities of healthcare staff, to help them contribute to achieve the Millenium Development Goals and also for the achievement of sustainable improvements in maternal, neonatal and child healthcare.
"The authors are grateful to Childhealth Advocacy International, Ministry of Health Government of Pakistan, World Health Organization and Advance Life Support Group U.K for supporting these trainings and their evaluation. A comprehensive report of evaluation of these trainings was compiled and put up on the web: http://www.childadvocacyinternational.co.uk/projects/emch_manual.htm#evaluation
The report has been removed from the website for technical reasons."
1. Evans DB, Adam T, Edejer TT, Lim SS, Cassels A, Evans TG; WHO Choosing Interventions that are Cost Effective (CHOICE) Millennium Development Goals Team. Time to reassess strategies for improving health in developing countries. BMJ 2005; 331: 1133-6.
2. United Nations. The Millennium Development Goals Report. United Nations, New York [online] 2007. Available from: URL: http://www.un.org/millenniumgoals
3. United Nations Millennium Declaration; Resolution 55/2 adopted by the UN General Assembly. New York: September 2000.
4. WHO, UN Children's Fund, UN Population Fund. Maternal mortality in 2000: estimates developed by WHO, UNICEF, UNFPA. Geneva: World Health Organization, 2004.
5. Zupan J, Aahman E. Perinatal mortality for the year 2000: estimates developed by WHO. Geneva: World Health Organization, 2005.
6. Black RE, Morris SS, Bryce J. Where and why are 10 million children dying every year? Lancet 2003; 361:2226-34.
7. Ronsmans C, Graham WJ; Lancet Maternal Survival Series steering group. Maternal mortality: who, when, where and why. Lancet 2006; 368: 1189-200.
8. Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, Mathai M, et al. Cost effectiveness analysis of strategies for maternal and neonatal health in developing countries. BMJ 2005;331:1107.
9. Koblinsky M, Matthews Z, Hussein J, Mavalankar D, Mridha MK, Anwar I, et al. Going to scale with professional skilled care. Lancet 2006; 368:1377-86.
10. Freeth D, Hammick M, Koppel I, et al. A critical review of evaluations of interprofessional education. London: Learning and Teaching Support Network for Health Sciences and Practice; 2002.
11. Hafeez A., Zafar S, Rifaq F, et al. Achieving health millennium development goals through capacity building of health workers-ESSEMCH Strategy. Pak Paedr J 2006: 30:3-9.
12. Advanced Life Support Group [homepage of Advance Life Support Group]. Manchester: Registered charity. [online] 2007 [cited 2007 Aug 12]. Available from: URL: http://www.alsg.org/index.php?id=475
13. Bates R. A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence. Eval Program Plann 2004;27:341-7.
14. McCallum M, Curran-Smith J, Wojnar D, Williamson S. Evaluating the impact of organizational learning initiatives. J Nurses Staff Dev 2002;18:177-83.
15. Jordan S. Educational input and patient outcomes: Exploring the gap. J Adv Nurs 2000; 31:461-71.
16. Lawn JE, Cousens S, Zupan J ; Lancet Neonatal Survival Steering Team. 4 million neonatal deaths: when? Where? Why? Lancet 2005; 365: 891-900
17. Jalil F. Perinatal health in Pakistan: a review of current situation. Acta Pædiatr 2004; 93:1273-9.