Sadia Ahsin  ( Physiology, Foundation University Medical College, Islamabad, Pakistan. )
Gul e Naghma Saeed  ( Physiology, Foundation University Medical College, Islamabad, Pakistan. )

Objective: To assess the operational efficiency of the ethical review committee of a medical college.

Methods: This study was conducted at the Foundation University Medical College, Islamabad, Pakistan, from 2012 to 2014. On the basis of literature review, methods for assessment of various features of ethical review committee were studied. A constitution-practice-outcome measurement model for evaluation of ethical review committee assessment process was developed. Data submitted to ethical review committee since its constitution was extracted and quantitatively analysed. 

Results: The ethical review committee comprised 14 members, including 4(28.6%) permanent and 10(71.4%) rotating clinical, basic sciences and non-medical members. As many as 45 research protocols were submitted, with submission frequency of 8(17.8%), 12(26.7%) and 25(55.5%) per year respectively, and issued ethical approval certificates within a mean duration of 7.2±3.2 days from the time of first submission to final notification. Issues looked into were according to World Health Organisation guidelines. Standard review was done on 29(64.4%) studies and expedited on 16(35.5%). In addition, 24(53.3%) protocols needed resubmission. Only 2(4.4%) protocols were not approved. The number of issues raised for resubmission was 71. Main reasons for resubmission were found to be incomplete documents 26(36.6%), invalid informed consent forms 12(16.9%) and negligence in maintaining confidentiality of study participants 9(12.7%).  

Conclusion: Ethical review committee with its limited resources was fulfilling its founding objectives as depicted by constitution-practice-outcome model.

Keywords: Ethical review committee, Self-audit. (JPMA 67: 42; 2017)

 

Introduction

According to international guidelines on research with human participants, ethical committee review is its fundamental requirement. The Council for International Organisations of Medical Sciences (CIOMS) states that "all proposals to conduct research involving human subjects must be submitted for review and approval to one or more independent ethical and scientific review committees".¹ Other regulatory bodies like the International Conference on Harmonisation (ICH), ² the Council of Europe³ and the United Nations Educational, Scientific and Cultural Organisation (UNESCO)⁴ have also issued similar guidelines. According to these guidelines, all research involving human subjects should be carried out in accordance with the fundamental ethical principles of respect, beneficence and justice. The basic purpose of an ethics review committee (ERC) is to protect the rights of the human participants involved in research. Institutes the world over are increasingly allocating considerable resources to establishing or strengthening research ethics committees. In Pakistan, the practice is a little less well established. While the medical colleges and research centres are establishing ERCs, they are doing so on their own in response to mandatory requirement of ethical clearance for international publication and on-ground establishment of ERC for securing foreign funding and grants. There is no oversight body on governmental level.

In 2004, the government of Pakistan established the National Bioethics Committee (NBC) under the Ministry of Health with the mandate to promote and facilitate ethical health services delivery, health research and to be an umbrella body for the ERC in various organisations and institutions. It has two subcommittees, i.e. the medical ethics committee (MEC) and the research ethics committee (REC).⁵ Unfortunately, there has been no policy announcement so far for registration, accreditation or regulating processes or an oversight by the NBC.⁶

The Pakistan Medical and Dental Council (PMDC) also has its code of ethics of practice for medical and dental practitioners including a section on research ethics, stating that it "endorses" the Declaration of Helsinki and that this code shall be "binding on all medical and dental practitioners." However, the PMDC has no mandate to regulate human subject research in the country.⁷

ERC at the Foundation University Medical College (FUMC) was established in 2011 to safeguard rights of study participants, ensure high quality research as per requirements of the Higher Education Commission (HEC) of Pakistan, and meet compulsory international criteria for publication. It started with initiative of institute\\\'s own senior faculty members who had research experience and could provide independent advice to researchers and professionals on the extent to which their proposals for research comply with the ethical guidelines recognised by the CIOMS and the World Medical Association (WMA) for researchers and reviewers.^1,8

The secretariat office of the ERC is located at the premises of the FUMC, where it carries out its administrative affairs since its constitution in 2011. One of the major functions of the ERC secretariat is to receive the research protocols, arrange ERC meetings, carry out the documentation of decisions, issue notification to researchers and record keeping. Globally, regular audit of standard operating procedures and working of ERC is recommended for quality assurance and for better future outcomes. Assessment of ERC performance can be done through development of various self-assessment tools and critical feedback from researchers.⁹

On the basis of literature review,^9-13 various methods for assessment of various features of ERC were considered. The self-assessment models for evaluating ERC performance reported in literature were found to be too extensive and elaborate for a budding ERC like the one at this institute where there are lesser number of research protocols requiring exhaustive reviews. 

Therefore, a modified, to-the-point and relevant assessment tool was indigenously developed on the basis of constitution-practice-outcome (CPO) model which can also be used by other developing ERCs of the region.

It was decided to undertake first internal audit since ERC inception to improve its standard. The current study was carried out to ensure quality control of the ERC and to assess its operational efficiency. The two major purposes of this study were to assess the performance of ethics committee by using the CPO model as a measurement tool, and to identify common ethical issues found in submitted research protocols.

 

Materials and Methods

This study was conducted at the FUMC, Islamabad, Pakistan, from 2012 to 2014. Permission for the study was obtained from the secretariat of ERC. Data was manually extracted and quantitatively analysed using the CPO model (Table-1)

and entered on a proforma. The objective of evaluation of constitution was to record its composition, members\\\' qualification and the number of rotations. For practice evaluation, the aim was to document the mode of protocol submission and the method of review, checklist of ethical issues which were discussed, the number of meetings and members required for expedited evaluation (i.e. evaluation within two weeks when research procedures present no more than minimal harm to the research participants or communities and are submitted with expedited review request) and regular review process (within one month), target time for evaluation of regular and expedited reviews  and the number of meetings per month in 3 years. For the outcomes evaluation, the number of protocols reviewed through both expedited and regular process, types of studies reviewed, number of resubmissions, approval time (target time as laid down in the standard operating procedures (SOPs) is less than 2 weeks for expedited review and less than a month for regular review), the number of non-approved protocols, pending decisions, method of communication with researchers and main ethical issues raised by the ERC were recorded. 

 

Results

Evaluation of the ERC\\\'s constitution showed that it comprised of 14 members, including 4(28.6%) permanent and 10(71.4%) rotating clinical, basic sciences and non-medical members. Only  physicians had training regarding clinical ethics, 2(14.3%) were trained through diploma course in medical ethics from Centre of Biomedical Ethics and Culture at the Sindh Institute of Urology and Transplantation (SIUT), Karachi, Pakistan, 6(42.9%) were trained through bioethics workshops, while non-medical members did not have any training beyond general observation and common sense. Moreover, 11(78.6%) members were from the FUMC, 1(7.1%) member, a social scientist, was from the Foundation University Rawalpindi Campus (FURC), an affiliated institute of the parent institute, i.e. the Foundation University Islamabad (FUI), while 2(14.3%) members, a lawyer and a religious scholar, were not associated with the parent university in any other capacity. All qualifying clinical and basic medical faculty was required to serve a one-year term on ERC board. As any temporary member completed one year of rotation tenure in ERC, he or she could voluntarily continue or leave the committee (Table-2)

Evaluation of ERC\\\'s practice showed that application form for ethical review was available on the FUMC\\\'s website. Research protocols along with application form were submitted to ERC secretariat manually. Online submission option was available as well. There was no specific time and date restriction for submission of protocols. After submission of application, the secretary ERC verified the completion of documents attached with application and ERC members were notified to meet within two weeks along with a copy of protocol. In case of expedited review, a meeting was called within one week of submission with a minimum of three permanent members. The list of issues to be deliberated was conveyed to the members at the time of their nomination to the ERC and was also brought by the secretary ERC in the meeting for further verification. It included scientific value, risk and benefit, study design, sampling process, data collection procedure, participants consent and the right to withdraw from study, management of research-related adverse effects, cost-effectiveness, confidentiality, conflict of interest, study documents, budgeting, local and international references, research funding and compensation. Protocols requiring resubmission and objections raised thereof were notified through letters from secretariat within one week. Target time as per institutes requirement was less than 2 weeks for expedited review and less than a month for regular review. Amended and resubmitted protocols were re-sent to 2 of the permanent members and were given ethical approval within one week if they qualified.  Minutes of the meeting were recorded and kept in secretariat. There was an option for researchers to clarify their point of view to ERC members if required. Members met biannually for discussion of general progress, nomination of new members when present member\\\'s on-board tenure was completed and any problems related to management of ERC. Permanent members also conducted workshops on bioethics for faculty on a regular basis (Table-3)

 

Evaluation of ERC\\\'s outcome showed that a total of 45 research protocols were submitted during the study period. Of them, 44(97.8%) were submitted manually. Expedited review was done for 16(35.5%) protocols and 29(64.4%) went through regular review process. Moreover, 30(66.7%) protocols were clinical studies including clinical trials and cross-sectional studies and 15(33.3%) were non clinical/descriptive studies. No animal studies were submitted, although the protocol for animal studies was present. Minimum number of reviewers was 3 for expedited review (Table-3). 

The mean time from the first submission to final notification through official letters was found to be 7.2±3.2 days in case of expedited review and 15±4.5 days in case of regular review process. There were 19(42.2%) protocols with one resubmission, 2(4.4%) with multiple resubmissions, whereas 2(4.4%) protocols were not approved. The number of issues raised for resubmission was 71. Of them, main reasons for resubmission were found to be incomplete documents 26(36.6%), invalid informed consent 12(16.9%) and negligence in maintaining confidentiality of study participants 9(12.7%). Resubmitted protocols where amendments had been made were given ethical approval within 3±1.2 days on average through certificate. The option of personal interview to clarify their point of view was not used by researchers (Table-4)

 

 

Discussion

By overseeing the research protocols, ERCs ensure that progression and addition of knowledge to life sciences is not at the cost of human dignity and safety. In the similar spirit of accountability, the need of overseeing and auditing the workings of the ethical research committees themselves is felt to ensure that they are faithfully carrying out the task that is entrusted to them. The evaluation of the ERC demonstrates the degree of effectiveness of ethics committees regarding research participants, researchers and research management.¹⁴ Self- audit of ERC at the FUMC to evaluate and review the established SOPs is one such effort, so that maximum facilitation to the researchers is coupled with the efficiency and optimum ethical practice.

Evaluation tools based on the ERC review process have been developed to assess whether its working has made an impact on human research protection or not. Several independent institutes like Stanford University¹⁰ and Boston University Medical Campus¹¹ have developed their own evaluation checklists or metrics based on institutional review board (IRB) policies, SOPs and feedback from the researchers and reviewers. The University of Missouri has even included the identification of barriers to effective compliance in its ERC performance audit.¹² Jong et al. in their paper published in 2012 outlined a comprehensive procedure to evaluate research proposal by means of two repertoires: a repertoire of rules and a repertoire of production. In the repertoire of rules the REC applies rules, weighs scientific value and burden to the participants, whereas the repertoire of production includes checking documents and forms and advising researchers on how to improve their proposals.¹⁵

The results of the present study were based on the CPO model developed and used at ERC-FUMC, which mainly assessed performance concerning composition, review process and the outcomes (in terms of research protocols being processed in stipulated time and the observations made).

Since ERC-FUMC is a newly established office, the self-evaluation process is focusing on the composition of the ERC, operating procedures and their smooth functioning and quantitative assessment of outcomes. It is pertinent to mention here that ERC under study is a joint one for both the medical college and the affiliated teaching hospital. Similarly, it looks into scientific merit as well as the ethical aspect of the protocols.

The first parameter to be assessed was the composition and general administrative hierarchy of the ERC under review. It was recognised that the laid-down policy about the number, type and qualification of the members was being followed but shortage of staff and time scarcity was a constant complaint. According to the guidelines given in the Griffith University\\\'s Human Research Ethics Committee Manual, the representation of each segment of society should be  ensured; their members include male and female laypersons, a pastor, a lawyer, a psychologist, a social worker, a medical graduate with research background, a non-medical academic staff member, representatives of (cultural, religious) minorities, a nominee from ethical research advisors pool with the deputy vice-chancellor heading the committee. ¹⁶

However, experiences shared in an Indian editorial caution us against merely filling the quota seats of the members stipulated. As pointed out, the non-medical persons on the committee are usually in awe of the medical persons and speak little. Often the laypersons are lacking in bioethical training and do not contribute effectively to the discourse.¹⁷ In a study reported from India in 2009, composition and role of ethics committees was discussed.¹⁸ Nearly half of their participants had medical backgrounds, 15% were social scientists and 33% were from social/behavioural science, community representative, social worker, lawyer, etc. Only 29% had formal training in ethics and all of them were from medical background. There was a lack of enthusiastic participation of ERC members with a legal background and those representing the general community. Lack of community representation in this study is similar to our ERC composition. Lack of training is also identified in a 2015 study from Pakistan, which reviewed the status of ERC at medical colleges in Khyber-Pakhtunkhwa (KPK). While there was no degree or diploma holder in their survey participants, a total of six doctors had attended workshops and thus were knowledgeable about medical ethics.¹⁹

In the same study, it was mentioned that although nine medical colleges reported monitoring of the ongoing studies for adherence to the approved protocol, only two presented proof in the form of periodic meetings on a regular basis for researchers to present their progress, and appointing members to monitor the ethical part of the research. Some IRBs have reported failure to maintain requisite composition and quorum of the review committee due to high turnover and excessive workload of the members. ¹⁴ The ERC-FUMC has not encountered such structural or functional problems so far but that may be in part because the workload is not excessive as it caters to just one medical college and its associated teaching hospital. 

For the practice assessment, SOPs were found to be clearly laid down and documented at ERC-FUMC. The aim of SOPs is to ensure a quality and consistent ethical review mechanism and to guide auditors by providing a checklist. A 2007 case study from Africa pointed out that the IRBs often failed to maintain adequate written SOPs.²⁰ In the study from KPK, record keeping was identified as a problem area with majority of medical colleges having no track record about its review boards.19 In contrast, following the protocol and faithful record keeping was observed in ERC-FUMC. However, the record of researchers\\\' compliance in field conduction of research was not kept. 

For outcome evaluation, the key indicator proposed in the literature is the average processing time which indicates the efficiency of an ERC.¹² Several university review boards give out timeline targets in their standard operating procedures and set duration targets for protocol reviews — ranging from 30 to 60 days.²⁰ At ERC-FUMC, total time between first submission to final ethical clearance notification was found to be 7.2 days in case of expedite review and 15 days in case of regular review process. Amended and resubmitted protocols were given ethical approval within one week. Adams et al. reported approximate duration of 60 days at ERC at Faculty of Tropical Medicine (FTM), Mahidol University, Bangkok, Thailand.¹⁴

In the current study, major observations made included incomplete study documents (36.6%), inadequate participants consent, right to withdraw from study (16.9%) and failure to protect the confidentiality of the participants (12.7%). Thus it can be seen that researcher’s ignorance, or maybe their impatience, of the laid-down submission requirements, was the main cause of amendment requests. Other major reasons for resubmission were related to the autonomy of the research participants. These findings are corroborated by other studies as well. For example, in a study conducted in Spain in 2011 that analysed the experience of the REC for the Carlos III Health Institute in the ethical assessment of research proposals involving human subjects, most of the comments were aimed at improving informed consent and procedures to ensure confidentiality (57.6%), 18.9% observations related to the principles of beneficence and non-malfeasance and the rest involved incomplete or incorrect documentation or requests for additional information.²¹

However, in the international scenario, the ERC self-audit has moved beyond mere procedural review. In a review article by Carl H. Coleman and Marie-Charlotte Bouësseau, the authors were concerned that the  focus on the procedures distracts ERC members from the actual purpose of the ERC, namely the protection of research participants.²² They argue that any evaluation of the ERC should include its effectiveness as the watchdog of the ethics as well. The authors suggest that the evaluation of the outcomes should include assessment of the actual impact of ERC review on the research participants, like their understanding and decision-making about participation and risks and benefits of the research, whether ERC review actually reduced the risks involved and whether it promoted more community responsive research. The audit process under study did not include any of the suggested parameters, since ERC reviewed is in the stage of formulation and it was justifiably felt that composition, SOPs and documentations should be our priority for self-audit so that procedural hiccups may be first identified and ironed out before undertaking more ambitious reviews.

A very relevant quality assurance component of the ERC review procedure and unfortunately one of the hardest is on ground reality check as urged by Coleman and Bouësseau.²² ERC must from time to time conduct on ground checks to see whether the ERC recommendations for participants\\\' safety measures and informed consent procedures are being followed in letter and spirit as well. This shortcoming was identified during our ERC performance review. Although the committee members tried their best to point out the procedural inadequacies in methodology, sampling and informed consent forms, yet the SOPs did not mandate any field visits to oversee actual research conduction to see if the  ERC recommendations were being followed or not. At present, a shortage of staff and concomitant duties of the serving committee members make any such requirement difficult to implement. Similar constraints for on-ground monitoring of the researchers were also experienced by other ERCs in this region as stated in a qualitative study of institutional ethics committees in New Delhi by Rohini Kandhari in 2013.²²

However, the inclusion of this component by the authors Adams et al. in their study for systematic analysis of the efficiency of ERCs\\\' functions indicates that field reality check is going to be a prominent feature of future ERC workings. ¹⁴

As far as this first review was concerned, it was a successful venture as it clearly charted the work done so far and it helped organise and file all the official records so that achievement of the goals and any shortfalls can now be noted at glance. Addressing the shortfalls observed is the next logical step to improve the working and thus cement the trust of the researchers on the institutional ERC.

ERC performance can be improved by incorporating critical feedback from the researchers, and by conducting onsite visits for monitoring of clinical trials.

Conclusion

CPO model was found to be an effective tool to evaluate optimal functioning of a budding ERC. The ERC at the FUMC with its limited resources was fulfilling its founding objectives. Main ethical issues found were negligence in obtaining informed consent and in maintaining confidentiality of study participants.  

 

Disclaimer: The abstract was presented at Association for Excellence in Medical Education (AEME) Conference 2015 held in March and the abstract was published in special issue of the Foundation University Medical Journal.

Conflict of Interest: None.

Source of Funding: None.

 

References

1.Council for International Organizations of Medical Sciences. International ethical guidelines for biomedical research involving human subjects. Bull Med ethics. 2002; (182): 17.

2.International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use: Guideline for Good Clinical Practice. [Online] [Cited 2015 March 30]. Available from URL: http://www.ich.org/ LOB/media/MEDIA482.pdf.

3.Council of Europe: Additional Protocol to the Convention on Human Rights and Biomedicine Concerning Biomedical Research. [Online] [Cited 2015 March 30]. Available from URL: http://conventions.coe.int/treaty/en/treaties/html/195.htm

4.UNESCO: Universal Declaration on Bioethics and Human Rights, art. 19. [Online] [Cited 2015 March 30]. Available from URL: http:/ / portal.unesco.org/ en/ ev.php-URL_ID=31058&URL_DO=DO_TOPIC & URL_SECTION=201.html.

5.National Bioethics Committee (NBC)[Internet]. Islamabad:Pakistan Medical and Research Council [Online] 2004 [Cited 2016 May 24]. Available from URL: http://www.pmrc.org.pk/NBC%20Guidelines.htm.

6.Sherin A. Research ethics committees (RECs) and monitoring of biomedical research in Pakistan. Khyber Med Univ J 2013; s(3): 121-2.

7.Jafarey AM, Iqbal SP, Hassan M. Ethical review in Pakistan: the credibility gap. J Pak Med Assoc 2012; 62: 1355.

8.World MA. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. Int J bioethics. 2004; 15: 124.

9.Lantero DA, Schmelz JO, Longfield JN. Using Metrics to Make an Impact in a Human Research Protection Program. J Clin Res Best Practice. 2011; 7: 1-5.

10.Research Compliance Office (RCO), Stanford University (2014) Survey and Metrics. [Online] [Cited 2014 Oct 14]. Available from URL: http://humansubjects.stanford.edu/new/cqi/surveys_ metrics.html. 

11.Clinical Research Resources Office (CRRO), Boston University (nd) Regulatory Tools and Resources. [Online] [Cited 2014 Oct 14]. Available from URL: http://www.bumc.bu.edu/crro/regulatory/ tools/. 

12.Institution Review Board, University of Missouri-Columbia (2008) Standard Operating Procedure: Assessments/Audits. [Online] [Cited 2014 Oct 14]. Available from URL: http://research.missouri. edu/policies/files/irb_assessments_final.pdf. 

13.Mayo Clinic Human Research Protection Program (2012) Roles, Qualifications and Evaluation of IRB Members. [Online] [Cited 2014 Dec 15]. Available from URL: http://www.mayo.edu/research/ documents/9-roles-qualification-and-eval-of-irb-memberspdf/DOC-10027103.]  

14.Adams P, Kaewkungwal J, Limphattharacharoen C, Prakobtham S, Pengsaa K, Khusmith S. Is Your Ethics Committee Efficient? Using "IRB Metrics" as a Self-Assessment Tool for Continuous Improvement at the Faculty of Tropical Medicine, Mahidol University, Thailand. PloS one. 2014; 9: e113356.

15.de Jong JP, van Zwieten MC, Willems DL. Ethical review from the inside: repertoires of evaluation in Research Ethics Committee meetings. Sociology of Health & Illness. 2012; 34: 1039-52.

16.Griffith University Australia Constitution for Ethical review committee. [Online] [Cited 2014 Dec 15]. Available from: URL:https://www.griffith.edu.au/__data/assets/pdf_file/0004/387094/HREC-Constitution-Updated-Apr-2013.pdf.

17.Thomas G. Institutional ethics committees: critical gaps. Indian J Med Ethics. 2011;8:200.

18.Brahme R, Mehendale S. Profile and role of the members of ethics committees in hospitals and research organisations in Pune, India. Indian J Med Ethics. 2009; 6: 78-84.

19.Abrar S, Ronis KA, Khan S, Siraj S, Safdar W, Khalid Y, et al. Status of ethical review boards at all medical colleges of Khyber Pakhtunkhwa. Journal of Ayub Medical College Abbottabad. 2015; 27: 411-4.

20.Kass NE, Hyder AA, Ajuwon A, Appiah-Poku J, Barsdorf N, Elsayed DE, et al. The structure and function of research ethics committees in Africa: A case study. PLoS Med. 2007; 4:e3.

21.Martín-Arribas MC, Rodríguez-Lozano I, Arias-Díaz J. Ethical review of research protocols: experience of a research ethics committee. Revista Española de Cardiología (English Edition). 2012;65:525- 9.

22.Coleman CH, Bouësseau M-C. How do we know that research ethics committees are really working? The neglected role of outcomes assessment in research ethics review. BMC Medical Ethics. 2008; 9: 6.

23.Kandhari R. Justice in jeopardy: a qualitative study of institutional ethics committees in New Delhi. Indian J Med Ethics. 2013; 10: 176-83.