By Author
  By Title
  By Keywords

January 2021, Volume 71, Issue 1

Protocol

Peer-taught virtual research workshops for surgical residents: Protocol for a novel and sustainable solution to improving surgical research in Pakistan

Authors: Russell Seth Martins  ( 4th Year Medical Student, Aga Khan University, Karachi, Pakistan. )
Ronika Devi Ukrani  ( Medical College, Aga Khan University Hospital, Karachi, Pakistan )
Mohummad Hassan Raza Raja  ( 2nd year Medical Student, Department of Biological Biomedical Sciences Aga Khan University, Karachi )
Maria Khan  ( Medical Student, Aga Khan University, Karachi, Pakistan. )
Haissan Iftikhar  ( Aga Khan University Hospital, Karachi. )
Syeda Sadia Fatima  ( Department of Biological and Biomedical Sciences, The Aga Khan University Karachi, Pakistan. )
Saulat Hasnain Fatimi  ( Department of Surgery, The Aga Khan University, Karachi, Pakistan. )

Abstract

Objectives: To generate a protocol describing the methodology for a study assessing the effectiveness of a comprehensive series of peer-taught online research workshops for surgical residents across Pakistan in terms of improving research-related knowledge and skills, and long-term research involvement and output.

Methods: The quasi-experimental study will consist of a series of six online research workshops conducted over Zoom for surgical residents across Pakistan. An online sign-up form will be circulated on social media to current surgical residents throughout Pakistan. Curricular content covered in the workshops will be developed in collaboration with experienced research faculty at the Departments of Surgery and Community Health Sciences at the Aga Khan University Hospital, Karachi. Facilitators of the workshops will be current surgical residents with a solid track record of research involvement at the hospital. Improvement in research-related knowledge and skills will be assessed through pre- and post-tests for each workshop. A post-series feedback form will assess satisfaction with the organisation, content relevance, peer-teaching and online delivery of the workshops. Long-term research involvement and output will be assessed by follow-up surveys at 6 months and 1 year post-workshop. All data-collection will be done via Google Forms. For ethical purposes, informed consent will be taken from participants prior to enrolment; data will be collected using a unique identifier number to maintain anonymity; and the only incentive provided to participants will be a Certificate of Completion for the research workshop series.

Result: We expect that surgical trainees participating in the series of six research workshops will demonstrate a highly significant percentage improvement (p<0.001) in the pre- and post-tests for each of the six core research competencies taught. On post-series feedback, we expect the majority of participants (>80%) to rate the overall organisation, effectiveness of online mode of instruction, and relevance to surgical training of the workshops as excellent/good. We also expect that most participants (>80%) would strongly agree/agree that peer-education is an effective model of teaching in research workshops. Finally, we expect statistically significant improvement (p<0.05) in research involvement and output in terms of publications as assessed in the 6 months and 1year post-workshop follow-up surveys.

Conclusion: In light of the ever-growing need for academic surgeons in Pakistan, this research protocol details a comprehensive strategy for research capacity-building among surgical trainees across the country. By equipping trainees with the knowledge and skills to conduct high-quality research, virtual research workshops provide a novel, grassroots-level and sustainable solution for addressing the surgical research crisis in Pakistan.

Keywords: Healthcare research, Medical research, Distance learning, Peer education, Surgery, Developing countries.

 

Introduction

 

Surgical research is an indispensable element of surgical healthcare in the era of evidence-based practice. However, surgical research in Pakistan leaves a lot to be desired in terms of both quality and quantity, with improvement still a long way off.1,2 Deficiencies in the landscape of surgical research in Pakistan are evident in the lack of reliable estimates of surgeries, surgical disease and unmet need for surgery in Pakistan.1 Additionally, the unique epidemiological characteristics of Pakistan's population may render international findings ungeneralisable in the local context.1 This emphasises the need for rigorous research to originate from Pakistan itself. Pakistan's research output pales in comparison with developed countries, like the United States of America (USA),3-6 with surgical research accounting for less than 20% of the total health-related research from Pakistan.7 This disparity compared to the developed countries stems from several factors, including a lack of financial support, a shortage of surgeon-scientists and academic surgeons, and inadequate research training and opportunities.8

Research training during surgical residency helps shape trainees' future careers, with many residency programmes in the US offering research competencies alongside surgical training in the form of Surgeon Scientist Programmes.9 Surgeons with academic capabilities are able to identify and innovate solutions to complex problems, observe, interpret and improve surgical outcomes, and disseminate expertise to the next generation of surgeons.10 In Pakistan, although surgical trainees are required to complete a dissertation as a prerequisite for the exit exam for the Fellowship of the College of Physicians and Surgeons Pakistan (FCPS) certification,11 this is not enough to create an organic research culture within the residency programmes. Research training for surgical residents is fruitful in improving long-term research output,12 and has also been suggested as a solution for strengthening surgical research in Pakistan.1

Research workshops (RWs) are a proven strategy to improve research capabilities,8 and are a common feature of international surgical conferences13 and surgical residency programmes in developed countries.14-17 RWs are usually delivered by surgical faculty with a wealth of research experience.14-17 However, recent focus on the effectiveness and sustainability of surgical residents as peer-teachers18 offers possibilities of enhancing the experience of RWs by incorporating peer-teaching in their delivery, as has been done for RWs teaching academic manuscript writing.19 Peer-education, where an individual with advanced knowledge on a particular topic, called the peer-teacher, teaches colleagues at a similar training level, called peer-learners, is widely recognised as an effective teaching-learning method.20 Lastly, the growing implementation of online learning in surgical residencies, a silver lining of the ongoing coronavirus disease-2019 (COVID-19) pandemic,21 allows for the upscaling of RWs to online platforms to cater to wider audiences.

Thus, the organisation of peer-taught RWs for surgical residents across the country has emerged as a novel and sustainable way to alleviate the surgical research crisis faced by Pakistan. To date, no study evaluating the effectiveness of comprehensive RWs for surgical trainees at a large scale across Pakistan has been done. The current protocol was planned to describe in detail the methodology for conducting a comprehensive series of RWs for surgical residents across Pakistan, to assess the effectiveness of online peer-taught RWs in improving knowledge and skills in core research competencies amongst surgical residents; the satisfaction of surgical residents with various aspects of the RWs, including overall organisation, content relevance, peer-teaching, and online delivery; and the long-term effectiveness of RWs for surgical residents in terms of research involvement and output at 6 months and 1 year post-intervention.

 

Methods

 

The RW series and this study will be organised and conducted by the Research and Development (R&D) Wing of the Society for Promoting Innovation in Education (SPIE) at the Aga Khan University (AKU), Karachi. SPIE is a society at AKU comprising students and trainees of health sciences, and is involved in innovation, education and research in the academic and public health sectors.

Study Design: A quasi-experimental study using a pre- and post-test design will be conducted to evaluate the effectiveness of the RW series in improving research-related knowledge and skills of surgical residents in Pakistan. In addition, a feedback form will be filled by all participants at the conclusion of the RW series, and a follow-up survey will be sent to all participants at 6 months and 1 year post-RWs. An overview of the research design is in place (Figure-1).

Study Setting: The RW series will be conducted over video conferencing software Zoom in conjunction with the International Business Machines (IBM) Video Streaming, which allows live, cloud-based, end-to-end video delivery to an external audience. Participants will attend from home, work, or any other location of choice and convenience. The technological infrastructure required, which includes a stable internet connection and a computer or laptop with audiovisual input, will be provided by the AKU Centre for Innovation in Medical Education (CIME).

CIME is the most advanced healthcare educational facility in the country. Technical analysts at the CIME will facilitate all hardware and software checks in preparation for the RW series, and will be at hand for any on-the-spot technical issues.

Study Population: The study population will comprise all surgical trainees currently enrolled in surgical residencies throughout Pakistan. According to the Research and Training Monitoring Cell of the College of Physicians and Surgeons Pakistan (CPSP), there are currently 2,857 trainees enrolled in surgical residency programmes across Pakistan.

Sample Size: Minimum sample size for pre- and post-test comparison was calculated using University of California San Francisco calculator.22 With alpha 0.05, beta 0.20, effect size 0.500, standard deviation (SD) of the post-over-pre change 2.25, and additional 10% to account for dropouts, the minimum required sample size was 175 individuals.

The SD was estimated using findings from a study which used a similar pre- and post-test methodology to evaluate the effectiveness of a RW.23 It was calculated as: SD of Change = p(SEM12 + SEM22). The Standard Error of Mean (SEM) 1 and 2 were calculated from SD on the pre-test (16.11%) and post-test (12.39%), respectively.

Participant recruitment: An online sign-up form using Google Forms will be circulated extensively through social media networks, including WhatsApp, Facebook, Twitter and Instagram. The sign-up form (Appendix A) will contain informed consent, demographics, work/training characteristics, research experience, and baseline proficiency.

 

Inclusion criteria

  • Adults aged >18 years of either gender.
  • Currently enrolled in any residency programme in Pakistan in any surgical subspecialty listed in the FCPS list of First Fellowships.24
  • Provides informed consent in the sign-up form.

 

Exclusion criteria

  • Participants not meeting the requirement for completion of the RW series (minimum 5/6 RWs attended with completion of both pre- and post-tests, and the submission of the feedback form at the end of the RW series).
  • Withdraws from workshops at any point during the RW series.

 

Sampling and enrollment

 

Convenience sampling will be used wherein all respondents who signed up using the sign-up Google Form and met the inclusion criteria will be sent an email for confirmation of enrollment in the course. This enrollment email will also contain a four-digit unique identifier number (UIN) for each participant, which will be used to de-identify participant responses for pre- and post-tests and the post-series feedback form.

In addition, the enrollment email will contain details regarding the scheduling of the RW series as well as outline the criteria for eligibility for the Certificate of Completion at the end of the RW series, meaning a minimum 5/6 RWs attended with completion of both pre- and post-tests, and the submission of the feedback form at the end of the RW series. Completion of only one of a pre- or post-test for a particular workshop will not be considered attendance for that workshop.

Although the minimum required sample size is 175 individuals, the confirmation email will be sent to all the surgical residents who respond to the sign-up Google Form and fulfil the selection criteria. This will allow the impact of these RWs to be maximised with minimum additional effort, and no additional costs or space concerns, which is a major benefit of adopting an online platform.

RW content: The content and structure of the RWs will be designed with input from faculty and surgical residents with rich research experience at the AKU, and will be centred on six core competencies of research at the level of the surgical resident (Figure-2).

Pre- and post-tests will be designed in collaboration with faculty and surgical residents at AKU, and will consist of questions exploring research-related knowledge. Faculty involved will be from the departments of Surgery and Community Health Sciences. To cater to multiple learning styles, the RWs will employ various teaching pedagogies, including presentations, interactive discussions and formative assessments through online gaming platforms such as Kahoot.

Facilitator training: A total of 3-6 facilitators will be recruited and trained to conduct the RWs. In order to maintain a peer-education model, the facilitators will be surgical residents with a strong track record of research involvement from the existing batches of surgical residents at AKU. Training of facilitators will take place in the form of small-group discussion (SDG) sessions with experienced research faculty from the departments of Surgery and Community Health Sciences. Apart from subject knowledge and skills pertaining to research, the facilitators will also be acquainted with teaching styles and methodologies.

RW piloting: Mock workshops will be conducted by the facilitators with surgical residents at AKU as participants in order to elucidate issues, such as level of interactivity, achievement of learning outcomes, hardware and software checks, appropriateness of pre- and post-tests, and time management. Feedback from the facilitators and participants regarding these mock RWs will be incorporated into the RWs, and re-piloted to the surgical residents at AKU.

Duration and schedule of RWs: The RW series will be held over six weeks, with one RW scheduled every week for a total of six RWs. Each RW will take place on Saturdays 5-7pm, with the link to the live-stream emailed to the participants at 4.30pm. Pre-tests will be open to attempt 24 hours before and up till the commencement of each respective RW, i.e., from 5pm on Friday to 5pm on Saturday. Post-tests similarly will be available for attempting starting from the conclusion of each RW till 24 hours later, i.e., 7pm on Saturday to 7pm on Sunday.

Recordings of the virtual RW will be available to participants on Google Drive so that those who miss the RW from 5-7pm on Saturday may watch the recording and then attempt the post-test. This is to cater to surgical residents who are unable to attend the RW on Saturdays due to training/work-related commitments which may be unavoidable in the field of surgery.

After the final RW, a feedback form will additionally be administered and available for 24 hours after conclusion of the final RW, i.e., 7pm on Saturday to 7pm on Sunday (Figure-3).

Data-collection: Data will be collected through the following means:

  • Sign-up form (Appendix A): Variables related to demographics, residency programme and work characteristics, research involvement, baseline research proficiency, and prior attendance of any RWs.
  • Pre- and Post-Tests: Each pre- and post-test will consist of 10-15 questions assessing knowledge related to the content of each respective RW.
  • Post-RW series Feedback Form (Appendix B): Feedback will pertain to overall organisation, content relevance, peer-teaching and online delivery.
  • Follow-up 6-month and 1-year Surveys (Appendix C): These will be emailed to participants at 6-months and 1-year after the completion of the RW series. Questions will pertain to changes in research involvement and research output seen at 6 months and 1 year following the conclusion of the RW series.

Google Forms will be used for all data-collection, including the sign-up form, pre- and post-tests, feedback form, and follow-up surveys.

Statistical Analysis: All analyses will be performed using SPSS 21. Categorical data will be represented by frequencies and percentages, and compared using chi-squared tests. Continuous data will be represented by means and standard deviations, and compared using independent t-tests for normally distributed data. Non-parametric tests will be used where the data deviates from normality. This will be assessed using histograms and the Shapiro-Wilk test. Pre- and post-test results will be compared as percentages using paired sample t-tests. Paired responses to dichotomous variables in the 6-month and 1-year follow-up surveys will be compared using the McNemar test (e.g. paired subject responses to "Did you have any peer-reviewed publications before you participated in the RW series? Yes/No" vs. "Do you have peer-reviewed publications currently, 6-months/1-year post-participating in the RW series? Yes/No"). For all analyses, p< 0.05 will be considered significant.

Ethical Considerations: An initial informed consent will be taken from every participant by means of a preliminary consent page (Appendix D) on the sign-up Google Form. This will explain the nature of the workshop, the extent of participants' involvement, and the confidential use of participants' de-identified data for research purposes and publishing in scientific journals / reviewed by ethical review committees. Moreover, participants' right to withdraw from the RW series at any time will be explained. Consent will be given by a checkbox at the end of the consent page, after which they will be able to proceed with the sign-up form.

The confidentiality of personal information and the identity of all the participants will be ensured. Each participant will receive a UIN on the initial enrollment email. All data collected, including pre- and post-tests, and feedback forms, will be collected using the UIN. For blinding purposes, only two members of the research team will assign and email UINs to participants in the enrollment emails. These two members will not be involved in data management and analysis, which will be managed by the other members of the research team so as to ensure complete anonymity of participant identity. Only the research team will have access to study data which will be stored in a password-protected database for 7 years after the study is completed, after which it will be deleted permanently.

This study has no associated risks or costs for participants. The only incentive for participants is a Certificate of Completion for the RW series. Eligibility criteria for the Certificate of Completion will also be explained explicitly to participants in the enrollment email. Performance on the pre- and post-tests and response to the feedback form will have no impact on the receipt of the Certificate of Completion.

The RW protocol is approved by the AKU ethics review committee.

 

Acknowledgement: We are grateful to the research and development (R&D) wing of the Society for Promoting Innovation in Education (SPIE) at Aga Khan University (AKU) for providing research mentorship to the authors, to the Centre for Innovation in Medical Education (CIME) for providing the technical infrastructure and support, and to the faculty of the departments of Surgery and Community Health Sciences for assisting in curriculum development for the research workshops (RWs).

Disclaimer: None.

Conflict of Interest: None.

Source of Funding: None.

 

References

 

1.      Zafar SN, McQueen KA. Surgery, public health, and Pakistan. World J Surg 2011;35:2625-34. doi: 10.1007/s00268-011-1304-3.

2.      Shamim MS, Enam SA, Kazim SF. Neurosurgical research in Pakistan: trends of publication and quality of evidence. Clin Neurol Neurosurg 2011;113:107-10. doi: 10.1016/j.clineuro.2010.09.013.

3.      Meo SA, Almasri AA, Usmani AM. Research productivity of Pakistan in medical sciences during the period 1996-2012. Eur Rev Med Pharmacol Sci 2013;17:2839-46.

4.      Gupta BM, Bala A. A scientometric analysis of Indian research output in medicine during 1999-2008. J Nat Sci Biol Med 2011;2:87-100. doi: 10.4103/0976-9668.82313.

5.      Graham SM, Brennan C, Laubscher M, Maqungo S, Lalloo DG, Perry DC, et al. Orthopaedic research in low-income countries: A bibliometric analysis of the current literature. SICOT J 2019;5:e41. doi: 10.1051/sicotj/2019038.

6.      Servadei F, Tropeano MP, Spaggiari R, Cannizzaro D, Al Fauzi A, Bajamal AH, et al. Footprint of Reports From Low- and Low- to Middle-Income Countries in the Neurosurgical Data: A Study From 2015 to 2017. World Neurosurg 2019;130:e822-30. doi: 10.1016/j.wneu.2019.06.230.

7.      Baladi ZH, Umedani LV. Pakistan Journal of Medical Sciences: A bibliometric assessment 2001-2010. Pak J Med Sci 2017;33:714-9. doi: 10.12669/pjms.333.13258.

8.      Iqbal MP. What ails medical research in Pakistan? Role of institutions. Pak J Med Sci 2015;31:1287-9. doi: 10.12669/pjms.316.9337.

9.      Gawad N, Allen M, Fowler A. Decay of Competence with Extended Research Absences During Residency Training: A Scoping Review. Cureus 2019;11:e5971. doi: 10.7759/cureus.5971.

10.    Rosengart TK, Mason MC, LeMaire SA, Brandt ML, Coselli JS, Curley SA, et al. The seven attributes of the academic surgeon: Critical aspects of the archetype and contributions to the surgical community. Am J Surg 2017;214:165-79. doi: 10.1016/j.amjsurg.2017.02.003.

11.    College of Physicians and Surgeons Pakistan (CPSP). Queries Related To Synopsis & Dissertations. [Online] 2020 [Cited 2020 September 27]. Available from URL: https://www.cpsp.edu.pk/synopsis-dissertation-faqs.php

12.    Merani S, Switzer N, Kayssi A, Blitz M, Ahmed N, Shapiro AM. Research productivity of residents and surgeons with formal research training. J Surg Educ 2014;71:865-70. doi: 10.1016/j.jsurg.2014.05.007.

13.    Taylor RH, Hollaar G. Review of a Canadian forum on international surgery: the Bethune Round Table. Can J Surg 2005;48:479-84.

14.    Allen L, Vogt K, Mele T, Ott M, Leslie K, Colquhoun P. Evaluating the impact of a resident research program in general surgery. Can Med Educ J 2017;8:e13-20.

15.    Torres D, Gugala Z, Lindsey RW. A dedicated research program increases the quantity and quality of orthopaedic resident publications. Clin Orthop Relat Res 2015;473:1515-21. doi: 10.1007/s11999-014-4080-1.

16.    Vinci RJ, Bauchner H, Finkelstein J, Newby PK, Muret-Wagstaff S, Lovejoy FH Jr. Research during pediatric residency training: outcome of a senior resident block rotation. Pediatrics 2009;124:1126-34. doi: 10.1542/peds.2008-3700.

17.    Papasavas P, Filippa D, Reilly P, Chandawarkar R, Kirton O. Effect of a mandatory research requirement on categorical resident academic productivity in a university-based general surgery residency. J Surg Educ 2013;70:715-9. doi: 10.1016/j.jsurg.2013.09.003.

18.    Kensinger CD, McMaster WG, Vella MA, Sexton KW, Snyder RA, Terhune KP. Residents as Educators: A Modern Model. J Surg Educ 2015;72:949-56. doi: 10.1016/j.jsurg.2015.04.004.

19.    Rathore FA, Mansoor SN. How to conduct a workshop on medical writing: Tips, advice and experience sharing. J Pak Med Assoc 2015;65:665-8.

20.    Dehghani MR, Amini M, Kojuri J, Nabeiei P. Evaluation of the efficacy of peer-learning method in nutrition students of Shiraz University of Medical Sciences. J Adv Med Educ Prof 2014;2:71-6.

21.    Ehrlich H, McKenney M, Elkbuli A. We Asked the Experts: Virtual Learning in Surgical Education During the COVID-19 Pandemic-Shaping the Future of Surgical Education and Training. World J Surg 2020;44:2053-5. doi: 10.1007/s00268-020-05574-3.

22.    Kohn MA, Senyak J. Sample Size Calculators for designing clinical research: Sample size for before-after study (Paired T-test). [Online] UCSF CTSI 2020 [Cited 2020 December 09]. Available from URL: https://sample-size.net/sample-size-study-paired-t-test/

23.    Shrivastava M, Shah N, Navaid S. Assessment of change in knowledge about research methods among delegates attending research methodology workshop. Perspect Clin Res 2018;9:83-90. doi: 10.4103/picr.PICR_41_17.

24.    College of Physicians and Surgeons Pakistan (CPSP). Training Programs: Fellowship of the College of Physicians and Surgeons - FCPS. [Online] 2020 [Cited 2020 September 27]. Available from URL: https://www.cpsp.edu.pk/fcps.php

 

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: