Objectives: To investigate the effects of design parameters on the user experience of virtual reality medical training.
Methods: The quantitative study was conducted at Punjab (Pakistan) from July 2018 to October 2018, and comprised final year students from eight medical colleges in Pakistan. Each respondent was given to experience laparoscopy operation in text, video and virtual reality-based learning methodologies. User experience and usefulness was assessed against a pre-validated scale and compared with the three learning methodologies.
Results: Of the 87, students, 50(57.5%) were male and 37(42.5%) were female. The overall mean age was 22.5±4 years. Result of virtual reality was better than others (p<0.05). Data was analysed using SPSS 20.
Conclusion: Virtual reality-based learning provided better user experience than traditional learning methodologies.
Keywords: User Experience, UX, Video-based learning, VBL, Virtual reality, VR. (JPMA 71: 1730; 2021)
Technology revolution has been rapid over the past few decades to encourage human computer interaction (HCI). The significant use of technology has been highlighted within the medical education sectors, helping the students to enhance their learning experience and improve their learning competence. Technologically improvised learning experience boosts the level of student motivation, provides them hands-on learning experience and increases the academic outcome. Learning experience and learning competence, on the other hand, are considered to be an important part of enriching the education system excellence.1 studies have identified the effects of using virtual reality (VR) simulation-based learning (SBL) to teach medical students and examine their learning experience and competence.2
Over the past few decades, VR has incredibly transformed the HCI prospect by contributing new prospective and unique ideas. VR, with its immersive control display, has created a simulation environment for the users that looks and feels real-world to some extent. This immersive technology, thus, engages students by capturing their full attention, and, in turn, producing better learning results.
The term "user experience" usually refers to design research. Design researchers working on Apple Systems in the 1990s were responsible for helping complement human interface by working among divisions and design processes.3 Previously, the HCI domain was more focussed on user interfaces; their usability, ease of use and ease of learning. This work is still going on, but contemporary technological advancements in the last few years have switched the focus more towards how the overall user experience (UX) can be improved.4
There is no accepted UX definition, but there is a promising consensus that user experience is determined by user's historical experience, user's internal condition, user's needs and goals, and user's external circumstances.4 International Organization for Standardization (ISO) further defines UX as the user's collective personal experience and perception of the use of a service, product or system.3
Due to increasing complexity in UX design, an adoption approach of traditional and contemporary user-centric design procedure has been established by many. The process requires collaboration between multi-disciplinary domains of software engineers, graphical design, sales people and marketers in order to create a seamless and pleasurable UX.5
VR is a platform used to create simulated computer-generated environments placing its user inside an immersive experience. Instead of viewing interfaces on a screen, users wear VR goggles which are able to track their head's movements, and controllers that allow them to interact with three-dimensional (3D) worlds. VR has recently gained a lot of popularity because of the improvement in performance on cellular phones.6 VR is primarily used for entertainment purposes, but its potential far exceeds that, such as education, training, physical and mental rehabilitation.7
Various improvements have been made to the VR technology to stimulate a better reality-based experience closely related to the real-life conditions. Different display sizes are offered and there are numerous models for VR goggles; each having its own unique features.8 Over the years, the human senses have been affected with the improvement in VR and sight, hear, and touch have progressed considerably since these senses adapt to VR. Despite these updates, there has been minor attention paid to the sense of smell and taste.9
Visual displays are a result of various configurations. Virtual reality can be used in multiple types of displays such as a large projection screen, or numerous projection screens interconnected with each other, monitors with the feature of tracking, and even head mounts which are commonly used in helmets.
VR also constitutes audio output which includes headphones, entertainment systems, and a series of speakers. Managing audio effectively is important because it helps in stimulating a real-time picture along with real-time sound, adding more to the concept of reality in a virtual world.10 In short, VR is one of the most useful and different human experiences. VR helps in making the most use out of a human brain where the entire information is processed in different ways. According to Harry Houdini, "What the eyes see and the ears hear, the mind believes." This technology is an actual representation of the line where seeing and hearing creates a reality in the human mind. Based on virtual reality, the normal brain reacts just like it is the actual reality.11 Various algorithms help in presenting the real world virtually in a way such that our senses and our mind combine together for a real experience.12
Driving a car in real life could be easy but learning to prepare for different unforeseen circumstances and scenarios is possible only when multiple different scenarios are put forward to a driver with the help of VR technology.13 Such scenarios may include blurred display, night drive, low-lighting drive, rainy drive, and fish eye effects. VR can help the drivers in many situations, such as drink-and-drive, and prepare them to handle the worst too.14 The conducted tests led to results that were surprising. The drivers were making more mistakes in the drunk scenario and had to improve.15
VR is still not a complete replacement for what is called real-life case. Drivers trained on a VR system tend to have more accidents compared to the ones facing real-life situations. VR-based environments can allow for learning but depending on a VR system entirely is not feasible.16 On the contrary, augmented reality (AR) can mix up real-life situations with virtual situations, leading to somewhat better experience towards computer simulation and immersive experience. In this way, AR can be considered a better learning facility since there are a lot of dynamic real objects enhancing a learner's mental effort.17
Various VR technologies are now being marketed on commercial basis which is growing a new market for VR technologies. VR systems can be divided into two categories. A room-scale VR is the one in which movement of the user is not restricted and real-time movements are possible considering the various effects a user faces while the head mount is being used. A stationary VR restricts movement by allowing the user to control the application with a controller rather than physical movement. A room-scale VR is more expensive than a stationary VR.18 The current study was planned to investigate the effects of design parameters on the user experience of VR-based medical training.
During research various hypothesis has been proposed. All of them tested through experimental measures to complete its reliability.
Ho1: User experience and usefulness of medical students will be relatively less than or equal to virtual reality learning in comparison with text based learning.
HA1: User experience and usefulness of medical students will be relatively higher through virtual reality learning in comparison with text based learning.
Ho2: User experience and usefulness of medical students will be relatively less than or equal to text based learning in comparison with video based learning.
HA2: User experience and usefulness of medical students will be relatively higher through text based learning in comparison with video based learning.
Ho3: User experience and usefulness of medical students will be relatively less than or equal to virtual reality learning in comparison with video based learning.
HA3: User experience and usefulness of medical students will be relatively higher through virtual reality learning in comparison with video based learning.
In order to compare experiences; virtual reality, video and text formats are selected by authors for this study. Content based on a laparascopy operation was selected for all three methodologies. 87 (50 male, 37 female) students were selected from 8 different private and public medical institutes of Pakistan with a mean age of 22.5 ± 4 years. Eleven participants were selected from 8 medical colleges using simple random sampling. One student dropped out during the experiment reducing the total to 87. This number was decided bearing in mind constraints on expenses and duration. Each student passed through each of the selected learning method one by one in order to evaluate the user experience. The VR experience was based upon the VR application by Medical Realities (MR). The app is available on both Android and iOS operating systems. The video application was similarly based on the same content of the VR application, accessible through the official YouTube page of Medical Realities (MR). An MBBS Doctor/ Medical Academic Lecturer was consulted to prepare the text-file based on the VR application and the video. Furthermore, the text file was proof-read by a senior medical practitioner.
Impression of the Virtual Reality
MR has multiple categories which can further drill down into related modules and provides its user with a real-time feel in Operation Theater positioned next to the surgical team. User can easily move in all possible directions for getting better understanding of the process. A 360 degree move provides awareness to how the surgical team can coordinate and react in different dynamic situations. Depending upon process, this can include laparoscopic, a 3D close-up feed and microscope feeds that is being operated on.
Experimental setup: In phase 1, first authors provide brief introduction about virtual reality, its learning and its application MR. Proper instruction about navigation of menu along with selection of appropriate module is given to each respondent. Total briefing of VR and instruction time was about 5 minutes.
After completion of briefing session each respondent were asked to experience MR through virtual reality glasses. After completion of their experience they were asked to complete a survey about the effects of user experience and usefulness for student learning through virtual technologies.
It is also important to explain that it was compulsory for every respondent to complete virtual reality experience at least 4 minutes to get a satisfactory understanding of the environment.
In phase 2, authors explain about video based learning and effect of user experience and usefulness at student's learning. Then a video of same content which was available in virtual reality was shown to the respondents. The duration of video was about 1 minute.
After viewing full video, respondents were asked to complete a survey regarding effect of user experience and usefulness at the student's learning.
In phase 3, authors give briefing about text based learning and effect of user experience and usefulness at student's learning. After giving these instructions, first author provided a printed text file to respondent for learning about anatomy experiment having same information which he already seen in virtual reality and video based learning. When they finish their text based learning then they were asked to complete a survey regarding effect of user experience and usefulness at student's learning through text based learning.
The above graph shows the comparison user experience and usefulness through virtual reality, text and video based learning. We found values of virtual reality at top of text and video based learning. Text based learning is in between video and virtual reality learning.
The research tool was adapted though a pre-validated survey. The tool was selected from the paper "Comparing Usability, User Experience and Learning Motivation Characteristics of Two Educational Computer Games".19
The research study has two main areas, user experience and usefulness for learning. Both categories have four questions and respondents used a Likert scale from 1 to 7 for feedback.
Following acronym are being used VRUX for virtual reality user experience, VRUL for virtual reality usefulness, VBUX for video based user experience, VBUL for video based usefulness, TBUX for text based user experience and TBUL text based usefulness.
Firstly we did comparison of means of virtual reality user experience and usefulness with text based user experience and usefulness in academic medical learning. Only at one point in usefulness, which was about the usefulness of interface value of text usefulness was higher than value of virtual reality usefulness.
Secondly we performed comparison of means of text and video based user experience and usefulness. Overall value of text based user experience and usefulness was much higher than video based user experience and usefulness. Only at one stage values of text and video based user experience and usefulness was the same as user interfaces.
Thirdly we did a comparison of user experience and usefulness of virtual reality and video based learning. According to this comparison, virtual reality user experience and usefulness is much better than video based learning for academic learning in medical education. At one point which is about usefulness of interface, the value of text based learning is equal to video based learning.
For hypothesis verification, paired sample t-test was performed through SPSS. All assumptions of this test were verified at sample data before conduction of t-test.
The standard deviation of mean difference (VRUX-TBUX) is 0.96 and value of mean difference is 1.95. Results reject H0 because p < 0.05. It means mean (VRUX) is significantly different from mean (TBUX). The standard deviation of mean difference (VRUL-TBUL) is 1.09 and value of mean difference is 1.69. Results reject H0 because p<0.05. It means mean (VRUL) is significantly different from mean (TBUL).
The standard deviation of mean difference (VBUX-TBUX) is 0.96 and value of mean difference is 1.95. Results reject H0 because p<0.05. It means mean (VBUX) is significantly different from mean (TBUX). The standard deviation of mean difference (VBUL-TBUL) is 1.09 and value of mean difference is 1.69. Results reject H0 because p<0.05. It means mean (VBUL) is significantly different from mean (TBUL).
The standard deviation of mean difference (VRUX-VBUX) is 0.80 and value of mean difference is 1.06. Results reject H0 because p<0.05. It means mean (VRUX) is significantly different from mean (VBUX). The standard deviation of mean difference (VRUL-VBUL) is 1.34 and value of mean difference is 0.91. Results reject H0 because p<0.05. It means mean (VRUL) is significantly different from mean (VBUL).
By observing the experimental results we have reflected that user experience and user learning of medical students while using VR is highest among other learning methodology. However, user experience and user expertise in video based learning in comparison to others is advantageous. Furthermore, while using text based learning, the user experience and user learning are better than video based learning but comparatively lower than VR based learning.
The results of the current study showed that user experience (UX) and perceived usefulness (UL) were highest for VR compared to the two other methodologies, while they were the lowest for VBL. This is something that is partially aligned with the available literature.20 It is perceived that the video based experience and its usefulness is better than text based experience/ usefulness.21 The current study concludes differently as it claims that video based learning methodologies' experience and their usefulness are not better than text based learning methodologies.
It is worth noting that there are few studies which have claimed that there is no difference between video-based learning and text-based learning. For instance, Gold 2016, Kramer 2017, Schneider et al. 2016 have argued that it is considered that video-based learning are more effective when it comes to present dynamic and complex events but there is no clear evidence that video-based learning is superior than traditional text-based learning. Moreover, for a better understanding, a combination of both video-based and text-based learning was found more effective as compare to only text-based learning.
However, it also infers that the VR-based learning methodologies have better usefulness and experience as compared to text based and video based learning methodologies which was very much expected. In our view, it was due to the immersive experience that the VR provides. It is very much evident that the modern technologies provides better user experience. Furthermore, we believe that the students are more comfortable with the conventional TBL resources, and, hence, leaned more towards TBL compared to VBL.
Limitations of the Study
The small sample size was a limitation of the study, but it was determined on the basis of financial and time constraints. The current study has collected data from 8 medical colleges, it is recommended that the large-scale studies comprising data from all over the country are needed to substantiate the findings of the current study. A comparative study of students from multiple sectors is also recommended to have a better view of the results. The current study has used a cross-sectional research design, a longitudinal research design is required for more generalized results.
The result of this study is very much aligned with the previous research available on the topic. The current study reveals that VR-based learning provided a better user experience than the conventional TBL and VBL methodologies. This is one of the major findings of the study. It calls for future researchers to look into the reasons why users have given preference to TBL methodologies over VBL methodologies.
Conflict of Interest: None.
Source of Funding: None.
1. Ahmed SI, Javed G, Mubeen B, Bareeqa SB, Rasheed H, Rehman A, et al. Robotics in neurosurgery: A literature review. J Pak Med Assoc 2018; 68:258-63.
2. Afridi A, Malik AN, Ali S, Amjad I. Effect of balance training in olderadults using Wii fit plus. J Pak Med Assoc 2018; 68:480-3.
3. MacDonald CM. User Experience Librarians: User Advocates, User Researchers, Usability Evaluators, or All of the Above? Proc Assoc Inf Sci Technol 2015;52:1-10.
4. Lallemand C, Gronier G, Koenig V. User experience: A concept without consensus? Exploring practitioners' perspectives throughan international survey. Comput Human Behav 2015; 43:35-48.
5. Wildner R, Kittinger-Rosanelli C, Bosenik T. How Good is Your User Experience? Measuring and Designing Interactions. GfK MarkIntell Rev 2015;7:52-7.
6. Men L, Bryan-Kinns N, Hassard AS, Ma Z. The impact of transitionson user experience in virtual reality. In: Men L, Bryan-KinnsN, Hassard AS, Ma Z, eds. Los Angeles, CA, USA: IEEE, 2017; pp 285-6.
7. Senno B, Barcha P. Customizing User Experience with AdaptiveVirtual Reality. In:Senno B, Barcha P, eds. Proceedings of the 23rdInternational Conference on Intelligent User InterfacesCompanion - IUI 18. New York, USA: ACM Press, 2018; pp 1-2.
8. Rebelo F, Noriega P, Duarte E, Soares M. Using Virtual Reality toAssess User Experience. Hum Factors. 2012; 54:964-82.
9. Jerald J. What Is Virtual Reality? In:Jerald J, eds.The VR Book:Human-Centered Design for Virtual Reality. New York, USA: Association for Computing Machinery and Morgan &Claypool, 2015; pp 635.
10. Bekelis K, Calnan D, Simmons N, MacKenzie TA, Kakoulides G. Effect of an Immersive Preoperative Virtual Reality Experience on Patient Reported Outcomes. Ann Surg. 2017; 265:1068-73.
11. Shin D. Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience? Comput Human Behav.2018;78:64-73.
12. Thomson ND, Aboutanos M, Kiehl KA, Neumann C, Galusha C, Fanti KA. Physiological reactivity in response to a fear? Induced virtual reality experience: Associations with psychopathic traits. Psychophysiology 2019; 56:e13276.
13. Sportillo D, Paljic A, Boukhris M, Fuchs P, Ojeda L, Roussarie V. An Immersive Virtual Reality System for Semi-autonomous Driving Simulation. In: Proceedings of the 9th International Conference on Computer and Automation Engineering - ICCAE 17. New York, USA: ACM Press, 2017; pp 6-10.
14. Knibbe J, Schjerlund J, Petraeus M, Hornbæk K. The Dream isCollapsing. In: Knibbe J, Schjerlund J, Petraeus M, eds. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. New York, USA: ACM Press, 2018; pp 1-13.
15. Taheri SM, Matsushita K, Sasaki M. Virtual Reality Driving Simulation for Measuring Driver Behavior and Characteristics. J Transp Technol 2017; 07:123-32.
16. Hasan O, Iqbal S. A framework for laparoscopic simulations. J Pak Med Assoc 2017; 67:1404-9.
17. Liou HH, Yang SJH, Chen SY, Tarng W. The Influences of the 2DImage-Based Augmented Reality and Virtual Reality on StudentLearning. J Educ Technol Soc 2017; 20:110-21.
18. Ma X, Cackett M, Park L, Chien E, Naaman M. Web-Based VRExperiments Powered by the Crowd. In: Ma X, Cackett M, Park L, eds. WWW '18: Proceedings of the 2018 World Wide Web Conference 2018. New York, USA: ACM Press, 2018; pp 33-43.
19. Álvarez-Xochihua O, Muñoz-Merino PJ, Muñoz-Organero M, Delgado Kloos C, González-Fraga JA. Comparing Usability, User Experience and Learning Motivation Characteristics of Two Educational Computer Games. In: Proceedings of the 19th International Conference on Enterprise Information Systems. [Online] 2017 [Cited 2021 April 25]. Available from URL: http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0006338901430150
20. Sattar MU, Palaniappan S, Lokman A, Hassan A, Shah N, Riaz Z. Effects of Virtual Reality training on medical students' learning motivation and competency. Pak J Med Sci 2019;35:852-7. doi: 10.12669/pjms.35.3.44.
21. Sattar MU, Palaniappan S, Lokman A, Shah N, Khalid U, Hasan R. Motivating Medical Students Using Virtual Reality Based Education. Int J Emerg Technol Learn 2020;15:160. doi: 10.3991/ijet.v15i02.11394