By Author
  By Title
  By Keywords

December 2018, Volume 68, Issue 12

ASSESSMENT RESEARCH

Psychometric testing of the Turkish version of the mentoring competency assessment scale for faculty

Hatice Camveren  ( Nursing Faculty, Nursing Management Department, Dokuz Eylul University )
Fahriye Vatan  ( Nursing Faculty, Nursing Management Department, Ege University, Turkey )

Abstract

Objectives: To evaluate the Turkish version of Mentoring Competency Assessment Scale in a faculty setting.
Methods: The study was conducted from November 2015 to November 2016 and comprised eight faculties affiliated to the Institute of Health Sciences of a university in Turkey. Data were collected usning a self- administered survey questionnaire from mentees and mentors working within the study universe. Data consisted of two parts; Mentoring Competency Assessment Scale with its mentor and mentee forms, and its Turkish translation which was back-translated for language validity of the scale. Confirmatory factor analysis was used to define the construct validity. Lisrel 8.7 was used for data analysis.
Results: Of the 326 subjects, 165(50.6%) were mentors and 161(49.4%), were mentees. The mean age of the mentors was 47.5±8.6 years, and the mean length of time employed as academicians was 21.6±9.0 years. The mean age of the mentees was 29±3.9 years, and mean length of time employed as academicians was 3.9±3.3 years.The content validity index of the scale was 0.82 for the mentee form and 0.84 for the mentor form. The Cronbach’s alpha reliability coefficient was 0.92 for the mentee’s form and 0.98 for the mentor’s form. Test-retest analysis determined a high-level positive significant correlation (p<0.05).
Conclusion: The validity and reliability of the Mentoring Competency Assessment Scale was supported by statistical analyses.
Keywords: Mentors, Health education, Confirmatory factor analysis, Validity, Mentoring competency. (JPMA 68: 1804; 2018)

Introduction

Mentorship is defined as the one-to-one relationship in which an experienced faculty guides a newly-employed faculty on career development and research subjects and supports this person. 1 Competence is defined as having the characteristics such as ability, power, skills and knowledge that people need to do a work.2Mentors\\\' competency is important for the continuation of leadership and career development of mentees and the retention of qualified faculty.The main purpose of mentorship practices is to provide individuals with an opportunity to improve their knowledge and skills in line with their personal development objectives.3Mentors have an important role in this development. Newlyemployed faculty are not familiar with the culture, policy and procedures, and curriculum of the university. Academic members’ adaptation will be more rapid and they will be more successful, thanks to the experienced mentors.4 In addition, mentors support the development of mentees’ leadership, teaching and training skills, as well as mentees’ involvement in the academic community and adaptation to their educator role.5,6 In the relevant literature, a few scales were found that assess the mentorship process. Berk et al.7 prepared a methodology report on the development of instruments in terms of mentorship activities. They developed two instruments. One of these instruments is a questionnaire where students define the relationship, and the other is the Mentorship Effectiveness Scale, which score the relationship. This scale assesses the mentors’ effectiveness only from the perspective of the mentees. Ozkalp etal. tested the mentorship scale developed by Noe . They defined the mentorship relationship ande valuated the mentors’ effectiveness from the perspective of mentees. 8,9 No valid and reliable instruments exist in Turkey that may include both the mentors’ self-assessment of their mentorship competency and the assessment of mentees. Therefore, the Mentorship Competency Assessment (MCA) scale with separate mentor form and mentee form, developed by Fleming et al,10is important because it provides direct feedback on mentors’ strengths, and is useful in the preparation of the content of the education programmes that mentors need for the formal mentorship programmes planned to be carried out in universities in the future because it determines the mentors competencies. Also, it generates different ideas on the mentors’ competency as a result of being administered to mentors and mentees, as all mentorship relationships do not have the same needs. Therefore, it allows the mentors and mentees to discuss the needs and different opinions that emerge in every relationship. Finally, it enables the mentors to indulge in selfassessment. The current study was planned to test the validity and reliability of the MCA scale with the goal of adapting it into Turkish.

Subjects and Methods

The reliability and validity assessment study was conducted from November 2015 to November 2016 and comprised eight faculties affiliated to the Institute of Health Sciences of a university in Turkey. The test-retest of the study was conducted in two nursing faculties and a faculty of health sciences. The study comprised mentors and mentees who met the inclusion criteria and volunteered to participate. Test-retest analyses were performed using the data of a part of the sample. The time between the first test and the final test was 3 weeks during the test-retest analysis. The sampling method used was similar to what the original scale used in the selection of its sample.10 The sampling technique used non-probability sampling from simple methods. Kaiser-Meyer Olkin (KMO) measure of sampling adequacy for mentees was 0.955 and Barlett’ssphericity test was significant (p=0.000). KMO measure of sampling adequacy for mentors was 0.867 and Barlett’ssphericity test was significant (p=0.000). This two results allowed us to do factor analysis of MCA scale. In order to perform factor analysis in scale studies, it is recommended that the sample size be at least 5 times the number of items and the sample size should be at least 100.11,12 Based on this knowledge, the sample size is approximately 6.5 times the number of items which can be said to be enough. The participants worked in the university between 8.30am and 4.30pm. Appointments were made to collect data when the participants were available during working hours. Survey forms were used for data-collection. The MCA scale consists of two sections, mentors’ selfassessment and mentees’ assessment of their mentors, to determine the mentors’ mentorship competency. It includes 6 subscales and 26 items in total. The subscales are: maintaining effective communication; aligning expectations; assessing understanding; addressing diversity; fostering independence; and promoting professional development.10 Each item is scored using a Likert-type (1-7) scale (1= not at all skilled, 4= moderately skilled, and 7= extremely skilled). The total score is calculated by summing the scores of each item. The minimum and maximum total scores of the scale are 26 and 182. The Cronbach’s alpha coefficient of the MCA scale was found to be 0.91 for the general mentor form and 0.95 for the mentee form. This value varied for the subscales between 0.62 and 0.91 in the mentor form and 0.59 and 0.90 in the mentee form.10 The scale was translated into Turkish simultaneously by 3 academic members in the Department of Education and Management in Nursing who were fluent both in English and Turkish, and by 2 separate academic linguists in the Department of English Language and Literature. These translations were then evaluated by the researchers, and the Turkish form of the scale was re-arranged. Next, the Turkish form of the scale was back-translated into English by another academic linguist.This back-translated form was compared with the original English form by another academic linguist, and the Turkish statements noncompliant with the original statements were reviewed by the researchers to obtain the final Turkish translation. Finally, the Turkish draft form of the scale was evaluated by 11 academic members with different specialties in the Department of Health Sciences, and the scale was finalised in line with their recommendations without changing the meaning of the items from the original scale. The specialists assessed each item by scoring between 1 and 4 using the Davis Technique. The scores were 1= Not appropriate, 2= The item should be made appropriate, 3= Appropriate but minor changes needed, 4= Very appropriate.The Content Validity Ratio (CVR) of each item was calculated during the evaluation of the specialists’ opinion. The Content Validity Index (CVI) was then determined by averaging the CVRs for the appropriateness level of the items. None of the items were excluded from the Turkish scale form.A pilot study was conducted through face-to-face interviews with a subset of mentors and mentees. Their recommendations were recorded, which provided data regarding the face validity of the scale. The scale was inalised with minor changes by the researchers in line ith these recommendations ithout making any changes in the meaning of the items. Data was analysed using SPSS version 22.0 (IBM Corp, BM SPSS Statistics for Windows, Armonk, NY) and Linear Structural Equations Model Language (LISREL 8.7) programmes. The introductory information of the participants was indicated using frequencies and percentages, mean +/- standard deviation, min-max values, and median. In the confirmatory factor analysis (CFA), chi-square goodness, root mean square error of approximation (RMSEA), comparative fit index (CFI), nonnormed fit index (NNFI), and normed fit index (NFI) were used to assess the validity of the scale. The reliability of the scale was analysed using the test-retest method and by determining the Cronbach’s alpha coefficient. Approval for the study was obtained from the Scientific Ethics Committee of the university, and written permission was obtained from of all the eight participating institutions.

Results

Of the 326 subjects, 165(50.6%)were mentors and 161(49.4%), were mentees. The mean age of the mentors was 47.5±8.6 years, and the mean length of time employed as academicians was 21.6±9.0 years.Among the mentors, 41(24.8%) were working in the Faculty of Medicine and 36 (23.6%) in the Faculty of Nursing; 118 (71.5%) were female, and 122(74%) were married. Besides, 74(44.8%) of them were professors. The mean age of the mentees was 29±3.9 years, and mean length of time employed as academicians was 3.9±3.3 years.Of them, 56(34.8%) were working in the Faculty of Nursing, and 42(26.1%) in the Faculty of Medicine; 115(71.4%) were female and 105(65.2%) were single. Besides, 125(78.3%) of them were research assistants. The CVR for the mentee form ranged between 0.64 and 1.00, and CVI was 0.83. The CVR for mentor’s form ranged between 0.66 and 1.00, and the CVI was 0.84.The first detrended fluctuation analysis(DFA) performed on the mentor form analysed the statistical significance of the tvalues of the items; the t values of all items were significant (p<0.05). Therefore, all items remained in the scale (Figure-1).



The fit indices were RMSEA=0.075, CFI=0.95, NNFI=0.94, and NFI=0.91 in the DFA performed for the mentor form. Regression values (factor loads) and t-values of all items found in the DFA analysis were noted (Table).



The first DFA performed on the mentee form analysed the statistical significance of the t-values of the items and indicated that the t-values of all items were significant (p<0.05). Therefore, all items remained in the scale (Figure-2).



The fit indices were RMSEA 0.086, CFI=0.98, NNFI=0.99, and NFI=0.97 in the DFA performed for the mentee form. The regression values (factor loads) of all items ranged between 0.65 and 0.95. The Cronbach’s alpha was 0.92 for the entire mentor form; 0.80 for the maintaining effective communication subscale; 0.79 for the aligning expectations subscale; 0.78 for the assessing understanding subscale; 0.83 for the fostering independence subscale; 0.42 for the addressing diversity subscale; and 0.82 for the promoting professional development subscale. The Cronbach’s alpha was 0.98 for the entire mentee form; 0.92 for the maintaining effective communication subscale; 0.94 for the aligning expectations subscale; 0.93 for the assessing understanding subscale; 0.97 for the fostering independence subscale; 0.71 for the addressing diversity subscale; and 0.93 for the promoting professional development subscale. The test-retest reliability coefficient of the MCA scale and its subscales were assessed using the intraclass correlation coefficient (ICC) analysis. In the mentee form, the ICC value was 0.97 for the entire form and ranged between 0.89 and 0.98 for the subscales. The values were positively and highly significant (p<0.05).In the mentor form, the ICC value was 0.89 for the entire form and ranged between 0.68 and 0.86 for the subscales. The values were positively and highly significant (p<0.05).

Discussion

The CVR related to the MCA mentee form was determined in the 0.63 1.00range.TheCVR of MCA mentor form was in the 0.63-1.00range. Because there were 11 experts, items that had greater than 0.59 CVR proved content validity.13 First, the psycholinguistic validity of the scale was tested. After that a pilot study using the Turkish prefinal version of the MCA was presented to 20 sample participants (10 mentors, 10 mentees). Each subject was interviewed about clarity and understandability of the Turkish form. CFA relies on previous research data or a theoretical background and presents a sound analytical frame for the assessment of different groups with cross-measurement models. 13 The values observed in the scale model confirm a good-fit model.14 For the MCA mentor and mentee forms, the hypothesised model with the six latent constructs and 26 items resulted in a good model fit to the data. Examining CFA results of original MCA mentor form, the important values are RMSEA=0.069, CFI=0.85, and the Tucker-Lewis Index (TLI)=0.83. Also, the CFA results of MCA mentee form are RMSEA=0.080, CFI=0.87,and TLI=0.85.14 The MCA has shown similar results for both mentor and mentee forms.10 Harrington15 stated that it was desirable to keep factor loadings under 0.30. Factor loadings are usually categorised as perfect (0.71 and above), very good (0.63), good (0.55), admissible (0.45), and poor (0.32). The first six items in the scale from 1 to 6 identified the 1st factor (maintaining effective communication); the following five items from 7 to 11 identified the 2nd factor (aligning expectations);the next five items from 12 to 16 identified the 3rd factor (assessing understanding); the items from 17 to 19 identified the 4th factor (fostering independence); the 20th and 21st items identified the 5th factor (addressing diversity); and the final five items from 22 to 26 identified the 6th factor (promoting professional development). Consequently, all 26 items could be included in the scale because regression values (factor loadings) of these items were above 0.40. Moreover, the scope-of-fit indices were analysed using CFA, which suggested a good model-data fit. Internal consistency must be determined before a test can be employed for research or examination purposes to ensure validity.16 Cronbach’s alpha reliability coefficients of the mentor and mentee forms were also found to be highly reliable and compatible with the coefficients in the original MCA. Reliability coefficients often take the form of ICC, which are measured to evaluate the durability and consistency of a scale: its proximity to +1 indicates high levels of reliability. The agreement between test-retest measurements is correlatively analysed, providing substantial data about the durability of a scale.17Testretest correlations of the MCA were conducted with ICC analysis, which indicated that the MCA produced reliable results. It can be said that the scale was found to be reliable after test-retest analyses for both the mentor and mentee forms. This study had some limitations, including its use of crosssectional data and the restriction to six mentor competency domains. Further, our sample focussed on individuals working in the domain of Health Science.

Conclusions

The Turkish version of MCA scale was found to be good enough for measuring the mentoring competency of faculty. The validity and reliability of the scale was supported by statistical analyses.


Declaration:
This manuscript has been accepted by the Kastamonu Education Journal and will be published by it in the Turkish language.
Disclaimer: The text is based on the Master thesis work of one of the authors. The research was presented in English verbal presentation at the IV International Eurasian Educational Research Congresson May 11-14, 2017.
Conflict of Interest: None.
Source of Funding: None.

References

1. Sorcinelli MD, Yun J. From mentor to mentoring networks: mentoring in the new academy. Chang Mag High Learn 2007; 39: 58-61.
2. Johnson WB. A framework for conceptualizing competence tomentor. Ethics Behav 2003; 13: 127-51.
3. Crisp G, Cruz I. Mentoring college students: a critical review of the literature between 1990 and 2007. Res High Educ 2009; 50: 525-45.
4. Gut DM, Beam PC, Henning JE, Cochran DC, Knight RT. Teachers’ perception of their mentoring role in three different clinical settings: student teaching, early field experiences, and entry year teaching. Mentor Tutoring Partnersh Learn 2014; 22:240-63.
5. Johnson WB. On Being a Mentor. In: A Guide for Higher Education Faculty. New York, London: Psychology Press; 2007; pp 1-17.
6. Nick JM, Delahoyde TM, Del Prato D, Mitchell C, Ortiz J, Ottley C, et al. Best practices in academic mentoring: a model for excellence. Nurs Res Pract 2012; 2012 937906.
7. Berk R a, Berg J, Mortimer R, Walton-Moss B, Yeo TP. Measuring the effectiveness of faculty mentoring relationships. Acad Med 2005; 80: 66-71.
8. Ozkalp E. Mentoring revisited from the perspective of mentors in learning process: The case study of Anadolu University in Turkey 1988; 1-8.
9. Noe RA. An investigation of the determinants of successful assigned mentoring relationships. Pers Psychol 1988; 41: 457-79.
10. Fleming M, House S, Hanson VS, Yu L, Garbutt J, McGee R, et al. The mentoring competency assessment: validation of a new instrument to evaluate skills of research mentors. Acad Med 2013; 88: 2-16.
11. Maccallum RC, Widaman KF, Preacher KJ, Hong S. Sample Size in Factor Analysis: The Role of Model Error. Multivariate Behav Res 2018; 36: 611-37.
12. Mundform DJ, Shaw DG, Ke TL. Minimum Sample Size Recommendations for Conducting Factor Analyses. Int J Test 2005; 5058: 37-41.
13. Kline RB. Principles and Practice of Structural Equation Modeling. 3rd ed. New York, London: The Guilford Press; 2011; pp 688.
14. Hooper D, Coughlan J, Mullen M. Structural Equation Modelling: Guidelines for  Determining Model Fit. J Business Res Methods 2008; 6: 53-60.
15. Harrington D. Confirmatory Factor Analysis,. New York: Oxford University Press; 2009.
16. Capık C. Gecerlik ve guvenirlik calışmalarında doğrulayıcı faktor analizinin kullanımı. Anadolu Hemşirelik ve Sağlık Bilim Derg 2014; 17: 196-205.
17. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ 2011; 2: 53-5. 

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: