Objective: To assess whether problem-based learning enhances students’ perception of their active learning and critical thinking skills with time, and to validate the self-assessment scale regarding active learning and critical thinking in the local context.
Method: The exploratory study was conducted at the Bahria University Dental College, Karachi, from February to September 2020, and comprised dental undergraduates in their first and second years of the academic programme. Data was collected using the self-assessment scale on active learning and critical thinking questionnaire which was administered twice after problem-based learning tutorials. Data was analysed using SPSS 23.
Results: There was a significant difference in the mean questionnaire scores of modules 1 and 3 of the first-year batch and modules 4 and 6 of the second-year batch (p=0.001). Cronbach’s alpha value was 0.735 in the first-year cohort and 0.802 in the second-year cohort.
Conclusion: Dental students’ perception of active learning and critical thinking increased with time using problem-based learning as the tool. The self-assessment scale on active learning and critical thinking was also found validated in the local context.
Keywords: Active learning, Critical thinking, Dental undergraduates, Problem-based learning, PBL, Self-assessment.
Acquisition of knowledge and skills are the important domains for students to direct their learning process. Self-assessment of students’ knowledge and skills is one way to assess students’ overall learning in any educational intervention. Self-assessment helps learners to investigate their strengths and weaknesses in learning.1,2 It is a process to compare students’ performance with previous performances, others’ performances, or with specific standards developed for a learning strategy to achieve the learning outcomes.3 As such, through self-assessment, students can monitor their progress in achieving the goals of a learning process.
Problem-based learning (PBL) is one of the instructional designs having a student-centred approach. It is often employed for achieving the goals of dental education through pedagogic theories of constructivism to promote self-directed learning (SDL).4 It is known to enhance learners’ active learning (AL) and critical thinking (CT) abilities, along with sharpening other skills, like professionalism, communication and time management.5 These skills help the students in their professional endeavours during the training period and their practical work.6
Self-assessment is a vital part of the PBL process for effective monitoring of students’ learning, thus enabling them to monitor their knowledge and skills acquired during the process.7 Therefore, to maximise PBL benefits, students need to develop self-assessment competencies for continuously monitoring their progress.8,9 The competencies can be monitored by using self-assessment tools or exercises that enable the students to evaluate their progress and assess their weaknesses and strengths during the learning process.
The self-assessment scale on active learning and critical thinking (SSACT) is a validated tool used for monitoring the process of self-assessment. It is explicitly designed to assess students’ progress during PBL. Self-regulatory processes, like goal-setting by inquiry-based learning, rating of learning strategies, evaluation of learning through feedback and using self-assessment scales, have been suggested to minimise the negative impact on PBL.9,10PBL feasibility may also get jeopardised if the primary aims of enhancing students’ AL and CT skills are not achieved during this resource-intensive process.
Higher educational institutions in Pakistan encourage student-centred learning in which learners take ownership of their learning for professional and personal development.1 Dental curricula in several institutions were revised to incorporate such an approach. Th Bahria University Dental College (BUDC) in Karachi introduced PBL as a complete transformation of learning strategy in 2018.11 However, since its inception, the process has not been evaluated for achieving the goals. Literature suggests PBL can show “signs of erosions” that may negatively impact the learning of students.12-14 Therefore, the process needs to be monitored at BUMDC. The current study was planned to assess whether PBL enhanced students’ perception of their AL and CT skills with time, and to validate the SSACT for the Pakistani context.
Subjects and Methods
The exploratory study was conducted at the BUDC, Karachi, from February to September 2020. After approval from the institutional ethics review committee, the sample size was calculated sample using OpenEpi calculator.15 The sample was raised using non-probability convenience sample technique. Included participants were the first and second year undergrduates of bachelor of dental surgery (BDS) and attended both face to face sessions of PBL. Participants attended only one session of PBL and was repaeter were excluded. The students were approached after a general announcement was made in both cohorts of students. Students were assured that their decision to participate or not to participate would not affect their academic results. Those volunteered to participate were enrolled.
The permission to use the SSACT tool was obtained from the developer1 in December 2018.
SSACT is the most suitable tool to assess the PBL and covers the broader context of learning. It is most useful to enhance the value of assessment among students and provides continuous feedback.16 The SSACT is a self-administered questionnaire filled by the students after PBL tutorials to reflect on their performance. SSCAT was used for observing students’ perceptions of their CT and AL skills.17
The SSCAT was used twice. First, it was used with the first-year cohort after the third PBL session of modules (MOD)1 and3. Later, it was used with the second-year cohort after the third PBL session of MOD4 and6.
Student responses were recorded on a seven-point Likert scale, ranging from1 meaning “not very true of me” to7 meaning “very true of me”.1 The seven-point scale provides a better validity, reliability and discrimination power than a scale with lesser points.18
Further, the validity and reliability of SSCAT in the current setting was explored. Content validity of the questionnaire was measured by taking the opinions from five medical educators who had the experience of tutoring PBL. The experts were asked to match all14 questions of SSACT with AL and CT. Content validity is highly recommended to find the relevance of SSACT.19 A pilot study was conducted to measure the reliability of SSCAT before data-collection. This data was not used for the main study. The pilot study included15 undergraduate medical students. The result of the pilot study revealed >80% score at each time. The reliability of SSCAT was also measured for its use with the study participants.
Data was analysed using SPSS 23. Kolmogorov-Smirnov test was employed for checking data normality. Wilcoxon signed-rank test was used to compare the mean score of SSACT in different modules. Cronbach’s alpha was used to evaluate the validity of the SSACT questionnaire. Inferential statistical analyses were performed with the significance level set at p<0.05.
Of the 100 students enrolled, 86(86%) completed the study; 41(47.7%) from the first-year batch and 45(52.3%) from the second-year batch. Overall, there were 19(22%) males 67(78%) females with a mean age of 21±1.56 years.
SSACT scores were significantly increased in the third and sixth modules compared to the first and fourth modules (Table 1; Figure).
Cronbach’s alpha value was 0.735 in the first-year cohort and 0.802 in the second-year cohort (Table 2). AL ad CT were found in 14 items of the final SSACT scale. AL expressed in Item 5 as collaborative learning, and in items 1-4, 6 as SDL. The cognitive skills reflected in items 8, 9 and 13 were from the CT domain.
The study focused on the importance of self-assessment practices among dental undergraduates. The progress in self-assessment skills is the desired outcome of a dental undergraduate programme. Self-assessment/self-appraisal is widely accepted, and is practised as a formative or summative activity.20 It is usually performed after laboratory work, preclinical projects, clinical skills, laboratory practical examinations and procedures in dental education. The current study supported the view that PBL enhances students’ perception of AL and CT skills with time.1 The SSACT score increased in the third and sixth modules compared to the first and fourth modules, respectively. There was limited evidence available in literature regarding any systemic training of self-assessment among dental undergraduates. Mays and Branch21 conducted a systematic review in 2016 to explore the practice and self-assessment training in clinical and preclinical dental education in the United States, and reported a scarcity of self-assessment in undergraduate years of dental education, including faculty development. They suggested that institutions have to implement self-assessment training during undergraduate dental education. According to the grading criteria, the practical implications of self-assessment are to improve the students’ ability to self-appraise objectively and to assess their progress.
Regular self-assessment by students has shown enhanced learning. Al-Madi et al22 compared PBL with didactic learning. They focused on the effectiveness of PBL in a hybrid dentistry programme of the anatomy of head and neck course to enhance students’ knowledge and confidence. There was significant improvement observed in self-reported confidence and knowledge acquisition in PBL versus didactic learning in the short term.22 Others have also supported the view that regular self-assessment enhances students’ SDL for productive thinking, student-centred learning, reflective learning, process-oriented and problem-solving skills.23
The current study showed that as the dental undergraduates at BUMDC moved from the first to the third module, and from the fourth to the sixth module, their mean score of SSACT increased over time. The results provide evidence for the effectiveness of PBL in providing students with confidence in the learning process. The results are comparable to those of Al-Madi et al.22
The cognitive skills practised during PBL sessions are analysing, questioning, generating hypothesis, and organising ideas in item 8. The sense of responsibility of learning regarding participation, questioning and collaborative learning is assessed in item 9.16-24 Skill of rephrasing is assessed in item13, where students transfer the understood information to the group in their own words. Metacognitive skills are stimulated in PBL by focussing on the problem; the thinking process is initiated for problem-solving as an act of “think to think”, and correlates it to the cognitive activity of the learner.25
The SSACT was developed and validated in medical students of Indonesia in 2013-141 and the Asian population has a similar culture. As part of its second objective, the current study found Cronbach’s alpha to be 0.735 in the first-year cohort and 0.802 in the second-year cohort. Content validity and internal structure of SSACT was done in 2014.1 Content validity was expressed by every domain, such as constructing the definition, item development and expert reviews.19
In terms of limitations, the study design could have been further strengthened by having a control group not exposed to the strategy. However, PBL is used in the curriculum for all dental students at BUMDC to enhance their learning. A control group formation in this scenario would have been unethical, as it would have potentially limited the learning opportunities for students in the control group.
The unequal distribution of male and female dental students can be a confounding variable and may reflect selection bias. However, this selection is beyond the control of the researchers.
Also, the study was done at a single centre and had possible response bias as the primary investigator was a faculty member.
Despite the limitations, the current study is probably the first to investigate the self-assessment of AL and CT skills acquired through PBL pedagogy in dental students of Pakistan. Additionally, the study validated the SSACT tool for utilisation in the present context.
Other variables, such as learning environment and academic performance, can be further studied in the local context for enhancing the feasibility of the self-assessment tool.
Dental students’ perception of AL and CT increased with time using PBL. SSACT was found to be a validated self-assessment tool with the PBL strategy in the local setting.
Conflict of interest: None.
Source of Funding: None.
1. Khoiriyah U, Roberts C, Jorm C, Van der Vleuten CP. Enhancing students' learning in problem based learning: validation of a self-assessment scale for active learning and critical thinking. BMC Med Educ 2015;15:140. doi: 10.1186/s12909-015-0422-2.
2. Leach L. Optional self-assessment: some tensions and dilemmas. Assess Eval High Educ 2012;37:137-47. Doi: 10.1080/02602938.2010. 515013
3. Loyens SMM, Magda J, Rikers RMJP. Self-Directed Learning in Problem-Based Learning and its Relationships with Self-Regulated Learning. Educ Psychol Rev 2008;20:411–27. Doi: 10.1007/s10648-008-9082-7
4. Hazwanie H, Chong DWK, Er HM, Deb PK, Wong PS, Lee MS, et al. Student’s perceptions of live online virtual e-problem based learning (LOVE-PBL) using google hangouts. Education in Medicine Journal 2017;9:31–39. Doi: 10.21315/eimj2017.9.4.4
5. Joseph N, Rai S, Madi D, Bhat K, Kotian SM, Kantharaju S. Problem-Based Learning as an Effective Learning Tool in Community Medicine: Initiative in a Private Medical College of a Developing Country. Indian J Community Med 2016;41:133-40. doi: 10.4103/0970-0218.177535.
6. Okubo Y, Matsushita S, Takakuwa Y, Yoshioka T, Nitta K. Longitudinal PBL in Undergraduate Medical Education Develops Lifelong-Learning Habits and Clinical Competencies in Social Aspects. Tohoku J Exp Med 2016;238:65-74. doi: 10.1620/tjem.238.65.
7. Hanley K, Zabar S, Charap J, Nicholson J, Disney L, Kalet A, et al. Self-assessment and goal-setting is associated with an improvement in interviewing skills. Med Educ Online 2014;19:e24407. doi: 10.3402/meo.v19.24407.
8. Tousignant M, DesMarchais JE. Accuracy of student self-assessment ability compared to their own performance in a problem-based learning medical program: a correlation study. Adv Health Sci Educ Theory Pract 2002;7:19-27. doi: 10.1023/a:1014516206120.
9. Vuma S, Sa B. Self-assessment: how do third year medical students rate their performance during problem-based learning. Int J Res Med Sci 2017;5:3044-52. Doi: 10.18203/2320-6012.ijrms20172985
10. Vuma S, Sa B. Evaluation of the effectiveness of progressive disclosure questions as an assessment tool for knowledge and skills in a problem based learning setting among third year medical students at The University of The West Indies, Trinidad and Tobago. BMC Res Notes 2015;8:673. doi: 10.1186/s13104-015-1603-0.
11. Ali KFM, Rizvi KF. Comparing the learning approaches using biggs revised study process questionnaire (R-SPQ-2F) among dental undergraduates. J Pak Dent Assoc 2019;28:68-73. DOI: 10.25301/JPDA.282.68.
12. Moust JHC, Van Berkel HJM, Schmidt HG. Signs of erosion: Reflections on three decades of problem-based learning at Maastricht University. High Educ 2005;50:665–83. DOI 10.1007/ s10734-004-6371-z
13. Dolmans DH, Wolfhagen IH, van der Vleuten CP, Wijnen WH. Solving problems with group work in problem-based learning: hold on to the philosophy. Med Educ 2001;35:884-9. doi: 10.1046/j.1365-2923.2001.00915.x.
14. Hendry GD. Tutors’ perceptions of dysfunctional behaviour in problem-based learning tutorial groups. HERDSA News 2002;24:27–30.
15. van Blankenstein FM, Dolmans DHJM, van der Vleuten CPM, Schmidt HG. Which cognitive processes support learning during small-group discussion? The role of providing explanations and listening to others. Instr Sci 2011;39:189–204. Doi: 10.1007/s11251-009-9124-7
16. Sullivan KM, Dean A, Soe MM. OpenEpi: a web-based epidemiologic and statistical calculator for public health. Public Health Rep 2009;124:471-4. doi: 10.1177/003335490912400320.
17. Nelson LP, Crow ML. Do Active-Learning Strategies Improve Students’ Critical Thinking? High Educ Stud 2014;4:77–90. doi: 10.5539/ hes.v4n2p77
18. Zimmerman BJ. Becoming self-regulated learner: Overview. Theory Pract 2000;41:64–70. DOI: 10.1207/s15430421tip4102_2
19. Taherdoost H. Validity and reliability of the research instrument; how to test the validation of a questionnaire/survey in a research. Int J Acad Res Manag 2016;5:28-36.
20. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006;119:166.e7-16. doi: 10.1016/j.amjmed.2005.10.036.
21. Mays KA, Branch-Mays GL. A Systematic Review of the Use of Self-Assessment in Preclinical and Clinical Dental Education. J Dent Educ 2016;80:902-13.
22. Al-Madi EM, Celur SL, Nasim M. Effectiveness of PBL methodology in a hybrid dentistry program to enhance students' knowledge and confidence. (a pilot study). BMC Med Educ 2018;18:270. doi: 10.1186/s12909-018-1392-y.
23. Dharma IPS, Adiwijaya PA. The Effect Of Problem Based Learning And Self-Assessment On Students’ Writing Competency And Self-Regulated Learning. SHS Web Conf 2018;42:e00036. Doi: 10.1051/shsconf/20184200036
24. Van Boxtel C, Van der Linden J, Kanselaar G. Collaborative learning tasks and the elaboration of conceptual knowledge. Learn Instr 2000;10:311–30.
25. Bahri A, Corebima AD. Improving PBL in Empowering Meta cognitive Skill of Students. Indian J Sci Technol 2019;12:1-9. DOI: 10.17485/ijst/2019/v12i17/69226.