Muhammad Amir Khan ( Association for Social Development, Islamabad, Pakistan. )
Muhammad Arif Munir ( Association for Social Development, Islamabad, Pakistan. )
Shirin Anil ( College of Medicine, King Saud University, Riyadh, KSA; Public Health Solutions Pakistan, Lahore, Pakistan. )
Maqsood Ahmad ( Association for Social Development, Islamabad, Pakistan. )
John Walley ( Nuffield Center for International Health, University of Leeds, UK. )
Ejaz Qadeer ( National Tuberculosis control Program Pakistan. )
Karam Shah ( World Health Organization for EMRO. )
Nida Khan ( Association for Social Development, Islamabad, Pakistan. )
Athar Nazir ( Association for Social Development, Islamabad, Pakistan. )
Objective: To develop and evaluate a more structured process for effective tuberculosis control monitoring.
Methods: The quasi-experimental exploratory study was conducted from April 2007 to January 2008 in the Punjab province of Pakistan. Eight intervention districts were compared with eight control districts. Intervention consisted of managers using performance monitoring guidelines and tools for monitoring meetings at the facility and district levels. Proportion of tuberculosis suspects among outpatients, registered confirmed cases and patients\\\' default rate were monitored. Semi-structured interviews were done to assess the experience of the participants.
Results: The proportion of TB suspects among outpatient attendees was significantly higher in the intervention districts (95% confidence interval 1.6-1.8%). The pre-registration default also showed difference (p=0.07). The case finding during 9 months of the intervention showed 96.3% increase compared to the 9 months of the preceding year.
Conclusion: The new process was effective in improving tuberculosis case finding. The process may be used to improve tuberculosis monitoring systems and other such healthcare services.
Keywords: Tuberculosis, Delivery of healthcare, National health programmes, District hospitals, Monitoring. (JPMA 66: 418; 2016).
Pakistan ranks fifth amongst the 22 tuberculosis (TB) "high burden" countries with 410,000 incidents of TB reported in 2012.1 The government of Pakistan endorsed Directly Observed Treatment — Short (DOTS) course for controlling TB in 1995, which started being implementated in 2001 after the Ministry of Health declared TB a national emergency that year.2 Implementation of DOTS in Pakistan relies on a countrywide network of 11,445 diagnostic centres and 500 treatment centres in 134 districts.2 This decentralised delivery of TB care has improved patient access to these services. However, quality control of TB care services does pose an immense managerial and technical challenge for the programme.
Strengthening of health systems is pivotal to TB control.3 Effective supervision and monitoring at facility, district and province levels has been one of the National TB Control Programme (NTP)\\\'s main strategies to ensure the quality of countrywide TB care implementation as outlined in the Stop TB strategy of the World Health Organisation (WHO).4 The NTP has already started organising district, province and national level quarterly monitoring meetings. However, the lack of clear objectives and a structured process for organising and linking these monitoring meetings at these three levels has led to sub-optimal outputs. Facility-level monitoring is almost non-existent.
The current study was planned to assist the NTP in strengthening TB control by developing context-adapted and well-structured peer review processes for the three levels (facility, district, provincial) of programme monitoring and supervision. The study aimed at refining the existing monitoring guidelines and tools used by programme managers for possible scale-up in other parts of the country and to disseminate the experience internationally. If found effective, taking TB as an example, such a structured monitoring approach could be replicated for other communicable and non-communicable disease healthcare programmes in developing countries.
Subjects and Methods
The quasi-experimental exploratory study was conducted from April 2007 to January 2008 in the Punjab province of Pakistan.
It comprised four phases (Figure-1) and had both quantitative and qualitative components. Approval was obtained from the NTP, and informed consent was obtained from the Provincial Tuberculosis Control Programme (PTP), Punjab, the District Health Offices (DHOs) and from the participants. Confidentiality of all patients was maintained as the data was extracted from the TB recording and registering forms of the NTP that do not contain information about identity of the patients.
Phase 1 comprised developing the intervention monitoring guidelines and tools which was done using a technical working group process that involved the NTP. Feasibility and utility indicators were agreed upon to help evaluate these guidelines and tools.
In phase 2, the monitoring guidelines and tools were implemented at facility, district and provincial levels.
At the facility level, diagnostic centres in each district were grouped into clusters, and each cluster was composed of three to five diagnostic centres. The cluster meetings rotated every month at each diagnostic centre. The participants were TB Facilitators and were given a fixed travel cost of around US$3.5 per person. The District TB Coordinator (DTC) peer-reviewed all the cluster meetings according to a structured process. Each cluster meeting had two main sessions: 1) joint review of the participating facilities, reviewing the input availability, case management practices and indicator analysis of respective facilities; and, 2) progress review and action plan of the diagnostic centre. The entire monitoring exercise for each facility was documented, with a copy kept at the respective facility as well as at the office of the DTC.
District level monitoring meetings were held every quarter (one quarter = three months), and were attended by staff from all the diagnostic centres. The meeting had two main sessions: 1) quarterly reporting session, peer-reviewed by the DTC, attended by DOTS facilitators from all diagnostic centres in a district; and 2) the review and plan session, supervised by the Executive District Officer (Health) and facilitated by the DTC, attended by in-charge Medical Officers (MOs) from all diagnostic centres in a district. Situation analysis and action plan for each diagnostic centre was summarised in table form by the DTC and was presented to the PTP.
Province-level monitoring meetings were held every quarter, chaired by the PTP manager, and attended by DTCs from all the districts. Three groups (10-11 districts each), facilitated by senior TB Control Programme staff, would work in three separate rooms. Each DTC brought the quarterly reports for their respective district. The DTC summarised case finding and treatment outcomes, resource availability, and implementation gaps in the district. A summary table was prepared and submitted to the NTP, which included the actions for each district in the province.
In phase 3, the intervention was implemented in eight districts. The experiences and the indicators in the intervention districts were compared to eight control districts. Districts where TB-DOTS was already implemented, mainly by the support of Fund for Innovative DOTS Expansion Through Local Initiatives to Stop TB (FIDELIS) and District Health Offices (DHO) of which agreed to participate in the study, were included. Districts in which DHOs did not agree to participate were excluded. Equal criteria of selection were followed for the intervention and control districts from single province (Punjab) to minimise the chances of selection bias. All the districts selected participated, and, hence, the reason for non-participation was not applicable.
As this was a health services operational research, in which monitoring and evaluation tools were being piloted for potential scale-up in the rest of the country, sample size calculation was not required. Nevertheless, we calculated post-hoc power. Based on the difference of prevalence of TB suspects found in the study (1.7%), taking average of 50,000 outpatients department (OPD) cases per district in eight districts in each intervention and control arm and 5% level of significance, post-hoc power was found to be 80.7% at intra-class correlation (ICC) of 0.0095.
Quantitative data was gathered from the records and registers maintained at respective facilities, districts and the province. The data tools consisted of TB recording and reporting tools of NTP, which collect the data on TB patients regarding diagnosis of TB, and treatment outcomes of success and failure. The recording and reporting tools are available on the NTP website.5 Data was collected by DOTS Facilitators in all the intervention and control districts who were given standardised training to do so in accordance with PTP and NTP protocols. This decreased the chance of information bias. Data on all the selected variables were present in the TB registers and we did not face the challenge of missing data.
Qualitative research methods included record review, participant observation, and semi-structured interviews with 12 DOTS Facilitators, 12 doctors, 4 District TB Coordinators and 2 programme officials and 4 other staff following their informed consent. Qualitative data collection tool consisted of questions about opinions on the feasibility and utility of the adapted monitoring guidelines and tools, like logistic and administrative difficulties, use of indicators and data extraction tools for monitoring, and case management gaps identified and efforts to overcome the same.
Quantitative data was entered into Microsoft Excel and analysed using SPSS 17. For quantitative analysis, outcomes of interest were percentage of TB suspects among outpatients (prevalence of TB suspects), prevalence of sputum smear positive TB cases, TB case finding, and pre-registration default of sputum smear positive TB cases, exposure being the implementation of monitoring and evaluation tools in the districts, and the confounders being age and gender. Other potential confounders could have been intervention related to TB case detection in the districts included in this study during the same time period of the study. No other intervention was conducted in the public sector for TB case detection in the study districts during the study period to the best of our knowledge. Mean and standard deviation (SD) values were calculated for continuous variable like age. Frequency and percentages were calculated for categorical variables. Difference of proportion and its 95% confidence interval (CI) by Wilson\\\'s method was calculated. Chi-square test was conducted and p<0.05 was considered significant.
Qualitative data was transcribed and categorised into themes. We compared the intervention and control districts, as well as the findings in the intervention district during the 9-month study duration with those found during the same period in the preceding year.
In phase 4, we held monthly meetings with the NTP and research field staff to share ongoing experiences, agree on modifications, and to address the gaps in guidelines and tools. It also included participant observation of randomly selected facility and district level meetings by trained researchers not otherwise involved in the event.
A total of 662,249 patients with a mean age of 25.2±2.3 years attended the OPD in the intervention districts, and 650,719 with a mean age of 25.5±3.1 years in the control districts. Overall, 668,947(51%) were males (Table-1). The TB suspect identification as a percentage of outpatient attendees was 1.7% (n=11,417) (95% CI: 1.6 -1.8%) higher in the intervention districts (p<0.001). Laboratory data revealed that 10.8% (n=1,725) of the examined TB suspects in intervention districts were sputum smear positive, compared to 9.8% (n=447) in the control districts; a difference of 1.0% (95% CI: 0.04 - 1.96%) (p=0.05).
In intervention districts, the case finding over nine months showed 96.3% increase compared with the same nine months of the preceding year (p<0.001) (Figure-2).
The pre-registration default showed statistically significant difference between intervention and control districts, being 2.5% (95% CI: 0.48 - 5.2%) (p=0.007). Both intervention and control districts showed a decrease over time in the proportion of pre-registration defaults. Defaults were more in the control districts (Figure-3).
Limiting the facility level meeting to include between three to five facilities in a cluster was found to be very acceptable to the facility staff, as well as practical for the programme and the district.
"The cluster approach has made the supervision and support of TB care at health facilities feasible for me, because now I can cover the whole district in four days."
The DOTS Facilitators as well as DTC did not report any significant administrative or logistic problem in their regularly attending the meeting. Rotating the venue was found feasible and acceptable to the participants. The DTCs were formally oriented on the guidelines for managing the cluster meetings in the district. The DOTS Facilitators reported having acquired the required skills mainly through learning-by-doing, with on-the-spot guidance during the cluster meetings. The cluster meeting averaged 2 hours for reviewing and planning TB related activities for three to five facilities.
At district level meetings, participation of facility staff was found to be 91% in 24 quarterly meetings over nine months. DOTS Facilitators and MOs supported their separate structured meetings:
"Previously we (doctor and paramedic) used to go on the same day to attend the district level monitoring event. Now we go on two different days, which makes our absence from the health centre less problematic for the patients".
According to an EDO, "With the previous unstructured approach the whole one-day event was haphazard and less-productive. However, the structured approach made it possible for us to review the progress facility-by-facility and take decisions accordingly."
The PTP staff got oriented on the guidelines for managing and facilitating the structured provincial event mainly by reading and doing. The provincial programme was found able to arrange the required space and audio-visual equipment for the event. The provincial meeting averaged 4.5 hours.
The initially selected "input" and "case management" indicators have continuously been reviewed and revised, in the light of ongoing practical experiences during the three quarters. The "Input" indicators covered the availability of trained staff, drugs, print and other materials, based on data brought from each respective facility. And the identified gaps at each facility were addressed through combination of onsite replenishment and plan for subsequent distribution. The "case management" covered the diagnosis, treatment and follow-up practices for each registered TB patient, based on compiled data from patient records at the respective facilities. Indicator analysis covered the case finding and intermediary outcomes for each facility, based on compiled facility data recorded in the respective monitoring form. Corrective actions, where needed, were taken or planned accordingly.
The DTCs found the data requirement for the finally-selected indicators feasible and useful for their monitoring the arrangements and performance of DOTS at each respective facility.
"The set of indicators used in the facility-level monitoring has enabled us to identify the input and case management gaps and respond in-time".
The DOTS Facilitators, trained by the programme with supervisory support of DTC, became skilful in extraction and compilation of the data on the input and case-management indicators. The review of cluster meeting records showed that staff errors in data compiling on selected indicators became minimal as they got more familiar with the forms and the processes.
"Errors in data compiling for the facility presentation, by DOTS Facilitators, became minimal as they got more familiar with the tables and graph."
Same was observed at the district and the provincial meetings:
"A computerised format for entering the district data and getting the required outputs for the district presentations made the process efficient and error-free."
Case management gaps at the facilities included the TB suspect referral for Acid Fast Bacillus (AFB) testing, pre-registration default, patient registration as per national guidelines, treatment support, absentee identification and retrieval, and validation of treatment outcomes. The identified and documented case management gaps were addressed by the facility managers during a specially organised meeting or the next district-level quarterly meeting. The district level structured monitoring meeting helped in addressing the material as well as case management gaps at facility level.
"In many facilities the referral of TB suspects for AFB microscopy improved after Executive District Officer (Health) started asking the facility staff about their relatively low suspect identification."
The province-level structured monitoring meeting was constructive in addressing the material as well as case management gaps at district level.
"In many districts suspect identification, absentee retrieval and material management improved, after discussion in the provincial meeting".
In two of the eight intervention districts, where malaria implementation capacity was also strengthened, a joint TB-malaria facility-level monitoring meeting was piloted. The initial two hours of TB progress review and plan was followed by a one-hour progress review and plan of malaria case management arrangements and practices. In these districts, the authorities designated the DTCs as Roll Back Malaria Focal Persons for their respective districts. The same health facilities were working as diagnostic centres for TB and microscopy centres for malaria. The district management and the facility staff found the joint facility-level monitoring of TB and malaria feasible and useful.
In the control districts, the interviews with the facility staff revealed that: 1) none of the facilities had regular monitoring visits of the district-level supervisor, 2) only one facility reported a single visit of the district supervisor in which some relevant records were reviewed in unsystematic manner, 3) no action was taken or planned on input-gaps and/or deviation in case management practices, if noted, and 4) no record was maintained on the supervisor\\\'s findings and recommendations during the visit. The interview with the DTC also revealed a similar situation i.e. their inability to visit most of the diagnostic centres on a regular basis, lack of clear guidelines about what to do during the supervisory visit, lack of documentation of the findings/agreed actions, and lack of follow-up to the facility visit, if made.
Observations from the district meetings in the control districts revealed: 1) three types of staff from each facility (i.e. DOTS Facilitator, Medical Officer and laboratory person) coming to the same venue caused too much crowd physically, 2) the respective roles of a doctor and a DOTS Facilitator from a facility were not clear, 3) the main output was limited to the preparation of quarterly reports, 4) unsystematic review of indicators and no presentation of summary tables, 5) no objective decision(s) made to improve facility performance on specific indicators, 6) minimal involvement of the district authority in the proceedings, and 7) no record-keeping of the visit was done.
The intervention districts showed better results than their control counterparts, implying an association between the structured monitoring events and the improvement in TB case management practice. In many countries, a TB suspect register is being piloted as a means to monitor TB suspect management.6,7 This approach has workload implications,8 especially for the healthcare staff working in general health services, where context is different from that of a specialized TB centre. This monitoring intervention in Pakistan assessed a simplified alternative approach to monitor the practices related to TB suspect identification and their referral for AFB testing.
The more effective suspect AFB testing in intervention districts, compared to the control districts, indicates the potential usefulness of structured monitoring in improved suspect examination. Measuring the suspect-outpatient ratio and comparing the results with better-performing facilities of the same type, helps to identify the facilities with poor suspect identification and laboratory testing practices.9 This is then followed by "interaction" with the facility staff to understand and address the reasons for low suspect referral for AFB testing.9 The early implementation experience shows that with little added workload, the gaps in suspect referral have been identified and addressed in the intervention districts. Increased suspect referral in intervention districts was as per the national guidelines.10 The rise in the case finding in the intervention districts during the study duration compared to the same duration last year indicates the potential positive impact of structured monitoring.
Early diagnosis and prompt treatment of TB patients is the key to tuberculosis prevention and control.11 Pre-registration default, also referred to as \\\'pre-treatment loss to follow-up\\\' or \\\'initial default\\\', means that patient was diagnosed to have smear-positive TB, but did not start anti-tuberculosis chemotherapy.12 This indicator reflects the failing of the health system in effectively controlling the spread of TB.12 Lower the pre-registration default, better the control efforts of the NTP and vice versa. The current study showed pre-registration default to be less in the intervention districts compared to the control ones. This indicates effective control efforts due to the structured monitoring intervention.
A DTC, through cluster meetings, can regularly cover 12 - 20 diagnostic centres in 3 or 4 days. This is time-efficient, as the DTC is a general public health manager (rather than a full-time TB person), with multiple other managerial responsibilities in the district. Three to four days per month of supervisor\\\'s field visit seem justified and feasible, under routine programme conditions, if this helps to enhance the TB case management processes at the respective facilities.
The entire intervention was designed in such a way that outputs from one meeting fed into higher-level meetings. The diagnostic centre level monitoring meetings focused mainly on managing individual TB cases, whereas the district meetings focussed on tracking output and outcome indicators for each diagnostic centre. The compiled diagnostic facility data was used in the district meetings, and the outputs of these district meetings fed into the provincial review meetings. The use of cluster meetings in some districts was used to jointly monitor more than one disease intervention, such as malaria. Globally there has been an increasing interest in designing integrated disease activities to enable health facilities and health personnel to efficiently manage multiple interventions.13,14 This preliminary experience of linking together the monitoring meetings at facility, district and provincial levels needs further evaluation and scale-up before it can be replicated in other programmes.
The fact that a bias could have crept in due to the non-randomised quasi-experimental design of the study. It was addressed by selecting the intervention and control districts from the same province where similar demographic and ethnicity would be expected. This was verified by similar age and gender distribution in the intervention and control districts. Information bias was taken care of by giving standard training to all the DOTS Facilitators according to NTP and PTP protocols to collect data in a standardised way from the TB registers.
The guidelines developed for structured monitoring in this research project have been scaled up in the control districts, as well as other districts of Pakistan. It is also being implemented for TB control in Afghanistan through WHO Eastern Mediterranean Regional Office.
Structured monitoring of healthcare services using performance monitoring meetings at facility, district and provincial levels was found to be effective in the study area. It was also found to be operationally feasible and acceptable by the healthcare staff at all levels of TB care provision.
We are grateful to the National and Provincial TB Control Programmes and their staff. We are also grateful to the technical working group of the NTP for their expert comments in review and refinement of guidelines, tools and training materials. We are also grateful to the district health offices for cooperation and support in the implementation of guidelines in the intervention and control districts. Thanks are also due to ASD Regional Coordinators and their support staff.
Funding: The research and development work has been supported by UK aid, through COMDIS-HSD based at the Nuffield Center for International Health and Development, University of Leeds, whereas the implementation has mainly been through National Tuberculosis control Programme and FIDELIS support.
1. World Health Organization.Global tuberculosis report 2013. [online] 2013 [cited 2014 Mar 15]. Available from: URL: apps.who.int/iris/bitstream/10665/91355/1/9789241564656_eng.pdf
2. Syed AH, Eric D, Sarwat KS, Pauline S, Sher SK, Sang JK, et al. Third-Party Evaluation of the National Tuberculosis Control Program. [online] 2010 [cited 2014 Mar 15]. Available from: URL: http://www.trfpakistan.org/LinkClick.aspx?fileticket=hwdjnPs6j30%3D&tabid=2402.
3. Atun R, Weil DE, Eang MT, Mwakyusa D. Health-system strengthening and tuberculosis control. Lancet. 2010; 375:2169-78.
4. World Health Organization. Global tuberculosis control: epidemiology, strategy, financing: WHO report 2009. Geneva; 2009.
5. National Tuberculosis control Program. Revised recording & reporting tools of NTP Pakistan. [online] 2014 [cited 2015 July 30]. Available from: URL: http://www.ntp.gov.pk/resource.php
6. Botha E, den Boon S, Lawrence KA, Reuter H, Verver S, Lombard CJ, et al. From suspect to patient: tuberculosis diagnosis and treatment initiation in health facilities in South Africa. Int J Tuberc Lung Dis. 2008; 12:936-41.
7. Mwanyika H, Lubinski D, Anderson R, Chester K, Makame M, Steele M, et al. Rational Systems Design for Health Information Systems in Low-Income Countries: An Enterprise Architecture Approach. J Enterp arch. 2012; 7: 60-9.
8. Blok L, van den Hof S, Mfinanga SG, Kahwa A, Ngadaya E, Oey L, et al. Measuring workload for tuberculosis service provision at primary care level: a methodology. Hum Resour Health. 2012; 10:11.
9. Wahyuni CU, Budiono, Rahariyani LD, Sulistyowati M, Rachmawati T, Djuwari, et al. Obstacles for optimal tuberculosis case detection in primary health centers (PHC) in Sidoarjo district, East Java, Indonesia. BMC Health Serv Res. 2007; 7:135.
10. National Tuberculosis control Program .Situation Analysis, Public-Private Partnership Models, Operational and Monitoring & Evaluation Guidelines for National TB Control Program Pakistan. Islamabad; 2006.
11. Harries AD, Rusen ID, Chiang CY, Hinderaker SG, Enarson DA Registering initial defaulters and reporting on their treatment outcomes [Unresolved issues]. Int J Tuberc Lung Dis. 2009; 13:801-3.
12. MacPherson P, Houben RM, Glynn JR, Corbett EL, Kranzer K. Pre-treatment loss to follow-up in tuberculosis patients in low-and lower-middle-income countries and high-burden countries: a systematic review and meta-analysis. Bull World Health Organ. 2014; 92:126-38.
13. Unger JP, De Paepe P, Green A. A code of best practice for disease control programs to avoid damaging health care services in developing countries. Int J Health Plann Manage. 2003; 18 Suppl 1:S27-39.
14. Yamey G. Scaling up global health interventions: a proposed framework for success. PLoS Med. 2011; 8:e1001049.