November 2021, Volume 71, Issue 11

Research Article

Utrecht work engagement scale: Construct validity and reliability in Chilean health science students

Rocío Glaría López  ( Department of Speach and Language, School of Medicine, University of Concepcion, Concepcion, Chile. )
Cristhian Pérez-Villalobos  ( Department of Medical Education, School of Medicine, University of Concepcion, Concepcion, Chile. )
Paulina Ortega-Bastidas  ( Department of Kinesiology, School of Medicine, University of Concepcion, Concepcion, Chile. )
Berta Schulz-Bañares  ( Department of Chemistry and Pharmacy, University of Concepcion, Concepcion, Chile. )
Ángela Pino-Zúñiga  ( Department of Speach and Language, School of Medicine, University of Concepcion, Concepcion, Chile. )

Abstract

Objective: The Utrecht Work Engagement Scale-Student (UWES-S) has been extensively applied in different countries to assess engagement in students, but there are discrepancies about its factor structure outcomes. To analyse the factor structure of Utrecht Work Engagement Scale-Student as construct validity evidence and reliability in medical students.

Methods: The non-experimental cross-sectional study was conducted at the public-sector University of Concepción in the Bio-Bio region in central Chile at the end of the first academic semester, between July and August 2017, and comprised students of health science undergraduate degree programmes in Medicine, Kinesiology, Pharmacy, and Speech and Language Therapy. Data was collected using a sociodemographic proforma and the Utrecht Work Engagement Scale-Student. The subjects were randomly divided into groups A and B. With group A, exploratory factor analysis was performed using the method of extraction of the main axis analysis. With group B, confirmatory factor analysis was performed using the maximum likelihood method. Data was analysed using STATA 11 SE.

Results: Of the 898 students, 538(59.91%) were females and 360(40.09%) were males. The overall mean age was 21.29±2.51 years (range: 17-34 years). Two factors, ‘involvement with studies’ and ‘enthusiasm for the career’, with eigenvalues 7.59 and 1.18 were identified with the potential to explain majority of the total variance of items. Comparative fit index, Tucker-Lewis index and the root mean square error of approximation showed the solution having the best fit.

Conclusion: Involvement with studies and enthusiasm for career were found to be critical factors, but did not show adequate adjustment even though confirmatory factor analysis found this to be the solution with the best fit.

Keywords: Health occupations students, Positive psychology, Psychometrics, Validation study. (JPMA 71: 2519; 2021)

DOI: https://doi.org/10.47391/JPMA.292

 

Introduction

 

In recent years, studies carried out in university students in the United Kingdom, Egypt and the United States have reported mental health problems, like fatigue, anxiety, depressive mood and panic crisis.1

Studies developed in Canadian and Chinese students from chiropractic and nursing careers have reported prevalence ranging from 19% to 28.7% for depression, 32.6% to 41.7% for anxiety, and 20.2% to 21.8% for stress.2,3 These studies analysed the behaviour of students in terms of deficit, although in recent years positivist psychology has proposed to focus attention on skills and attitudes of human beings.4 As such, it becomes relevant to investigate the student population for a positive phenomenon, like academic engagement.

Academic engagement, also known as academic commitment, has been defined as a psycho-affective state maintained over time and related to the studies. It is composed of three dimensions; vigour, dedication and absorption. Vigour is characterised by high levels of energy and mental resistance used while studying despite the adversities that arise. Dedication is characterised by high involvement with studies and career. Absorption is understood as the high levels of concentration while studying and performing academic tasks.5-7 It has been observed that academic engagement brings positive results in different academic aspects, such as improvements in academic performance, accumulation of social capital, a lower trend to give up studies, or better adaptation to university life.7

Similarly, it has been shown that students with higher levels of academic engagement remain in the career and graduate within established timelines.7,8

Besides, students who show engagement have energy, they strongly identify with their studies, are deeply involved with them8 and show improvement in academic performance.9,10 When motivation is added to engagement, a significantly positive correlation between these two factors emerge, indicating that higher motivation translates into higher level of academic engagement.9

Regarding the differences of engagement related to the career, research is still scarce, although it is believed that individuals related to social sciences and health careers would be the ones with higher levels of engagement.10 A study on Chinese students of Clinical Medicine, Basic Medicine, Forensic Medicine, Public Health, Nursing and Dentistry showed there were no significant differences in vigour, dedication or absorption among the different careers belonging to health sciences.11

Another research carried out on nursing students from 4 public universities in Spain showed that dedication was the engagement variable with the highest values.12

A study carried comprising students of Nursing, Marketing, Labour Relations and Human Resources showed that Nursing students had lower levels of stress in the burnout dimension, and higher in those of engagement and perceived stress.13

Currently, this phenomenon can be evaluated through the Utrecht Work Engagement Scale-Student (UWES-S) questionnaire, of which there are 2 versions; one with 17 ítems, and a shorter version with 9 ítems (UWES-S9). The psychometric properties of both versions have been studied in different countries with varying results. For instance, in Puerto Rico, Argentina, Finland, Spain, Italy, Germany, Norway and Sri Lanka, it was determined that the three-factor structure is the one that best fits the model10,14-16 which is in line with the original proposal.17,18 In other countries, like Brazil, China, Japan and South Korea, the presence of one factor has been found.14,19

The foregoing findings contrast with those of studies carried out in Chilean, Ecuadorian and Spanish students which proposed a bi-factorial model.14,20,21

The benefits that the students get through academic engagement include better academic performance, lesser dropout rate, greater involvement and greater motivation.7-10 Having a reliable instrument that allows the measurement of this critical factor will help educational institutions which may plan interventions to maintain or promote levels of engagement and thus ensure the success and satisfaction of students. The current study was planned to analyse the factor structure as evidence of construct validity and reliability of UWES-S instrument in medical students.

 

Subjects and Method

 

The non-experimental cross-sectional study was conducted at the public-sector University of Concepción in the Bio-Bio region in central Chile at the end of the first academic semester, between July and August 2017. After approval from the institutional ethics review committee, the sample was raised using non-probabilistic sampling technique from among adult students of health science undergraduate degree programmes in Medicine, Kinesiology, Pharmacy, and Speech and Language Therapy.  Students who were absent for more than three months in the preceding term were excluded.

The sample size represented 52.82 participants for each of the 17 items in the UWES-S questionnaire, and 26.41 for each factor analysis carried out, which was over the standard of 20 participants per item.21 The career studied was used as the segmentation criterion.

After taking informed consent, the students were handed a sociodemographic proforma to collect data about gender, age, year of study, type of high school of origin, children, having completed pre-university courses, subjects failed, and university background.

To evaluate academic engagement, the 17-item UWES-Student was used. The students were asked about the feelings they experienced while studying. The response options were presented in a Likert-type format, with the answer possibilities being: None (0), few times a year (1), once a month or less (2), few times a month (3), once a week (4), few times a week (5), or every day (6). Items were grouped in three dimensions or subscales; vigour (6 items), dedication (5 items), and absorption (6 items).17,18

The UWES-S scale and the sociodemographic proforma were part of a battery of instruments applied in the context of a larger study.

Before factor analysis, the relevance was evaluated by calculating the Kaiser-Meyer-Olkin (KMO) sample adequacy statistic. This value was equal to 0.93, and the Bartlett sphericity test was significant (p<0.001), providing evidence that factor analysis would be relevant. Subsequently, the number of scale factors was estimated, for which two criteria were considered complementary: the Kaiser-Guttman criterion or latent root that is usually used for this purpose, to which Horn's Parallel Analysis was added, which is a more appropriate tool for these purposes.21

To obtain evidence about the validity of the construct of UWES-S, complementary factor analysis (CFA) was performed for which the total sample was randomly divided into Groups A and B. Exploratory factor analysis (EFA) was performed in group A using the method of extraction of the Main Axis Analysis (MAA), which is the most relevant tool for the processing of composed scales, such as UWES-S. With group B, CFA was performed using the maximum likelihood method as well as comparative fit index (CFI), Tucker-Lewis index (TLI) and the Root Mean Square Error of Approximation (RMSEA).21 For CFI and TLI, values be >0.96 were considered, while for RMSEA, value must be <0.06 was considered in line with literature.21

Finally, to estimate the internal consistency of the factors identified, Cronbach alpha reliability of each item was calculated. In addition to evaluating the discriminatory capacity of the items, the correlation of each reagent with the total corrected score of its factor was calculated.

Data was analysed using STATA 11 SE.

 

Results

 

Of the 898 students, 538(59.91%) were females and 360(40.09%) were males. The overall mean age was 21.29±2.51 years (range: 17-34 years) (Table 1).

Two factors, ‘involvement with studies’ and ‘enthusiasm for the career’, with eigenvalues 7.59 and 1.18 were identified with the potential to explain majority of the total variance of items, and Horn’s Parallel Analysis, based on 500 random samples, confirmed the existence of the two factors.

All UWES-S items presented configuration coefficients >0.30 in at least one factor, which is the minimum to consider a statistically significant load. However, items 4, 8 and 9 presented significant loads in two factors simultaneously (cross-load) (Table 2).

Factor I, formed by items 15, 12, 17, 14, 16, 11, 3, 9, 6, 8, 1, 4 and 13 (ordered from the highest to the lowest load), was called ‘involvement with the studies’. The items presented a reliability of α=0.909, with correlations between items and a total corrected from r=0.42 (item 13) to r=0.76 (item 9).

Factor II, formed by items 5, 2, 7 and 10, was called ‘enthusiasm for the career’. The items presented reliability of α=0.889, with a correlation between items and the total corrected from r=0.69 (item 10) to r=0.82 (item 5).

Considering that the two factors presented adequate reliability, their score was calculated by adding the responses of the subjects to the items. From the resulting scores, the relation between the factors was evaluated using Spearman’s correlation coefficient through a unilateral contrast (rho=0.639; p<0.001).

Finally, fitness indices were employed using the maximum likelihood method in group B (Table 3).

 

Discussion

 

The exploratory factorial analysis, with half of the study sample, proposed the existence of a two-factor model of UWES-S’ involvement with studies and enthusiasm with the career. Likewise, CFA applied to the other half of the sample showed that it was one of the solutions with the best fit. However, the solution obtained in the current study not an adequate fit to the data, which is line with literature in which the worst fit was obtained while using only 15 of the reagents,20 followed by the uni-factorial solution15 with poor results.

The two-factor solution differs from that proposed originally17 but it is consistent with different studies conducted in Latin America; both in the 9-item and 17-item versions.20

A study20 reporting findings consistent with the factors called ‘predisposition to studies’ and ‘satisfaction with studies’, which are the labels used by another study.19 However, differences were observed concerning the distribution of the items between the two studies; one20 proposed that the ‘predisposition to studies; corresponded to the vigour and dedication dimensions of academic engagement, while ‘satisfaction with studies’ corresponded with the absorption factor. The present study, on the contrary, proposes that the first factor would be formed by all items belonging to vigour and absorption dimensions, whereas the second factor, called ‘enthusiasm with studies’, would be formed by items belonging exclusively to the dedication dimension. This is consistent with the findings of an earlier study17 which mentioned that there was a high correlation between the vigour and absorption dimensions to the extent that they could actually be grouped.14 However, this grouping can be influenced by the writing of the items since the factor ‘involvement with studies’ is more related to the commitment students have about their studies in general. Thus, the term used in the statements is “studies”, whereas in the statements of the ‘enthusiasm with the career’, the concept “career” is used. This aspect had already been noticed in literature.19

The current study decided to replace the labels of the factors, given the fact that the first factor would be referred to as the active commitment of the student in the academic tasks. This would imply more than a predisposition. For this reason, it has been decided to rename it as  ‘involvement with studies’, whereas the other, rather than referring to the mere fulfillment of expectations or satisfaction, alludes to the positive emotions generated by the studies and the activation students feel when studying. Therefore, it was called ‘enthusiasm with the career’.

The current study has limitations as it evaluated academic engagement through a self-reporting questionnaire that may be subject to social desirability. Also, only four health-related disciplines were considered for raising the sample. Further studies are recommended in this regard to cover other healthcare domains as well.

 

Conclusion

 

Involvement with studies and enthusiasm for career were found to be critical factors, but did not show adequate adjustment even though CFA found this to be the solution with the best fit.

 

Disclaimer: None.

Conflict of interest: None.

Source of funding: VRID Associative Project of the University of Concepción, Chile.

 

References

 

1.      Chau C, Vilela P. Determinants of mental health in students universitarios de Lima y Huánuco. Revista de Psicología. 2017; 35:387.

2.      Meckamalil C, Brodie L, Hogg-Johnson S, Carroll L, Jacobs C, Pierre Côté. The prevalence of anxiety, stress and depressive symptoms in undergraduate students at the Canadian Memorial Chiropractic College. J Am Coll Heatlh. 2020; 5:1-6.

3.      Zeng Y, Wang G, Xie C, Hu X, Reinhardt J. Prevalence and correlates of depression, anxiety and symptoms of stress in vocational college nursing students from Sichuan, China: a cross-sectional study. Psychol Health Med. 2019; 24:798-811.

4.      Carrasco Muñoz MA, Martínez Morales C. Nivel engagement and its implication in academic performance in health sciences university students from Unheval-Huánuco, 2015. bol.redipe [Online]. 2019 [Cited 2020 January 12]. Available from: URL: https://revista.redipe.org/index.php/1/article/view/692

5.      Hinrichs C, Ortiz L, Pérez C. Determinants of mental health in students of universitarios de Lima y Huánuco. Revista de Psicología. 2017;9:109-116.

6.      Glaría-López R, Carmona-San Martin L, Pérez-Villalobos C, Parra-Ponce P. Burnout and academic engagement in speech therapy. Inv. Ed Med. 2016; 5: 17-23.

7.      Urquijo I, Extremera N. Academic satisfaction at the university: Relationships between emotional intelligence and academic engagement. Electro J Res Educ Psychol. 2017; 15: 553-73.

8.      Martínez I, Youssef-Morgan C, Chambel M, Marques-Pinto A. Antecedents of academic performance of university students: academic engagement and psychological capital resources. Educ Psychol. 2019; 39:1047-67.

9.      Ariani D. Why am I burnout and engaged? the role of motivation and personality.  J Psychol Educ Res. 2017; 25:61-89.

10.    Wickramasinghe N, Dissanayake D,  Abeywardena G. Validity and reliability of the Utrecht Work Engagement Scale-Student Version in Sri Lanka.BMC Res Notes. 2018; 11:1-6.

11.    Liu HC, Yansane AI, Zhang YR, Fu HJ, Hong NR, Kalenderian E. Burnout and study engagement among medical students at Sun Yat-sen University, China: A cross-sectional study. Medicine (Baltimore). 2018; 97:e0326

12.    Liébana-Presa C, Fernández-Martínez M, Vásquez-Casares A, López-Alonso A, Rodríguez-Borrego M. Burnout and engagement among university nursing students. Enfermería Global. 2018; 50:142-52.

13.    Vallejo-Martín M, Aja Valle J, Plaza Angulo JJ. Estrés percibido en estudiantes universitarios: influencia del burnout y del engagement académico. Int J Educ Res Innov. 2017; 9:220-36.

14.    Portalanza-Chavarría C, Grueso-Hinestroza M, Duque-Oliva E. Propiedades of the Utrecht Work Engagement Scale (UWES-S9):exploratory analysis with students in Ecuador. Innovate.2017; 27:145-56.

15.    Sánches-Cardona I, Rodríguez-Montalbán R, Toro-Alfonso J, Moreno-Velázquez I. Psychometric Properties of Utrecht Work Engagement in university students from Puerto Rico. Mexican Magazine of Psychology. 2016; 33: 121-34.

16.    Casuso-Holgado M, Moreno-Morales N, Labajos-Manzanares M,  Montero-Bancalero F. Psychometric characteristics of the version Spanish of the UWES-S scale in Physiotherapy university students. Physiotherapy. 2017; 39: 4-9.

17.    Schaufeli W, Salanova M, González V, Bakker A. The measurement of engagement and burnout: A two sample confirmatory analytic approach. J Happiness Stud. 2002; 3: 71-92.

18.    Schaufeli W, Bakker A. UWES Utrecht Work Engagement Scale. [Online] [Cited 2020 Oct 29]. Available from: URL: https://www.wilmarschaufeli.nl/publications/Schaufeli/Test%20Manuals/Test_manual_UWES_English.pdf

19.    Pérez C, Parra, P. Psychometric properties of the scale Academic Engagement, UWES-S (abridged version), in psychology students. Rev Educ Cienc Salud. 2010; 7: 128-33.

20.    Cachón-Zagalaz J, Lara-Sánchez A, Zagalaz-Sánchez M, López- Psychometric properties of the UWES in education students. Suma Psicológica Magazine. 2018; 25: 113-21.

21.    Martínez R, Hernández M, Hernández M. Psicometría. [Online] 2014 [Cited 2020 Oct 29]. Available from: URL: https://issuu.com/ maldonadocarla/docs/psicometr__a_-_alianza_editorial.

 

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: