Objective: To translate, adapt and validate the Adult Self-Report tool in Urdu language, and to establish internal consistency of its subscales.
Methods: The cross-sectional study was conducted from September 2017 to August 2018 at the National Institute of Psychology, Quaid-i-Azam University, Islamabad, Pakistan, and comprised adult stable psychiatric outpatients and non-clinical subjects from the community. After forward and backward translation of Adult Self-Report, the tool was tested on the subjects who responded on a three-point Likert scale from 'never' to 'very often'. The items were grouped under eight subscales. Data was analysed using SPSS 22.
Results: Of the 768 participants, 408(53%) were outpatients and 360(47%) were non-clinical subjects. The overall age range was 18-59 years. The tool was found to be effective for Pakistani sample, with root mean square error of approximation (0.03), comparative fit index (0.94) and Tucker-Lewis Index (0.94) values indicating good fit. Also, al items indicated good factor loadings (range: 0.25-0.94). Alpha values indicated that all subscales were internally consistent (range: 0.64-0.92).
Conclusion: Adult Self-Report was found to be a comprehensive tool showing a good model fit for Pakistani population.
Keywords: Adult Self-Report, Confirmatory factor analysis, Adult psychopathology, Achenbach system of empirically based taxonomies, Internalising behavioural problems. (JPMA 71: 1133; 2021)
Accurate assessment of psychopathology has remained a challenge for mental health practitioners. Most of the time, diagnosis is based on unstructured interviews. But now there is an emerging debate on advantages of using standardised tools for assessment as it can add to diagnostic efficacy.1,2 This has led to increased efforts to develop tools of international equivalence as well as translation and adaptation of existing measures.3 One such empirically-driven system is the Achenbach system of empirically-based taxonomies (ASEBA),1 which is a comprehensive system of assessment that assesses infants, adolescents and adults for different problematic behaviours. Different formats have been developed for different individuals.3 For the assessment of adult psychopathology, Adult Self Report (ASR) is a standardised instrument from this series that is used worldwide to assess behavioural, emotional, social, cognitive problems, and issues related to substance abuse.1,3
Much of the research in the field of clinical psychology in the West has been done on individuals belonging to higher socio-economic class, and the question of generalisability of these findings to other cultures carries significance. It has been proposed that "substantial evidence of the comparability" of translated or adapted instrument across original instrument is significant for conducting assessment across different societies.1,4,5 Literature has highlighted indicators, such as factor structure, correlations among scales and item difficulty, in order to consider in cross-national comparisons.6 Differences across cultures cannot be traced if instruments are used invariantly across cultures.2,6 Two issues of key importance while working with translated and adapted instruments are "spurious cultural differences" and "valid group differences". Spurious cultural differences are expected due to bias in instrument, whereas valid group differences are referred to as impact.1,7 Furthermore, method bias and item bias need to be given special consideration in cross-cultural studies. Some formats or items are more likely to be endorsed either positively or negatively in one culture compared to other. Item bias or differential item functioning includes relation between item scores and total scores varying across different cultures. Therefore it becomes important to test the factor structures of instruments proposed in one society across another as well.8-10 One of the methods to deal with these problems is testing factor structure. This can help in testing whether individual items are performing similarly or differently across cultures. Several methods have been proposed for comparing and correlating scale scores.2,10,11
The eight syndrome model of ASR broadly comprises two categories; internalising behavioural problems and externalising behavioural problems. Internalising behavioural problems further comprises three narrow band scales; anxious depressed, withdrawn behaviours, and somatic complaints. Externalising behavioural problem comprises three narrow band scales; aggressive behaviour, rule-breaking, and intrusive problems. It consists of two additional narrow band independent scales; thought problems and attention problems. Overall, these are summed together to get total problem score.1,2,10
The model has been tested and confirmed across 29 different societies.2 Advanced hierarchical linear modeling (HLM) analysis has indicated that differences across effect sizes are more because of individual differences rather than societal or cultural differences.12
The current study was planned to translate, adapt and validate ASR in Urdu language, and to establish internal consistency of its subscales.
Patients and Methods
The cross-sectional correlational study was conducted from September 2017 to August 2018 at the National Institute of Psychology, Quaid-i-Azam University, Islamabad, Pakistan. After approval from the institutional ethics review committee, sample size estimation was done on prior assumptions following the rule of thumb that sample >200 offers appropriate statistical power to test factor structure.13,14 Another guide for sample power was N-p ratio between the number of people and the number of measured variables. As a general guiding principle, N>p at least equal to 5 was followed.14 The sample was raised using non-probability convenience sampling technique from among those visiting the out-patient departments (OPDs) of public-sector hospitals of Islamabad and Rawalpindi after formal permission from hospital authorities. Non-clinical subjects were enrolled from the community.
Clinical samples included adult psychiatric patients marked as stable by their respective consulting psychiatrists. Patients with severe psychiatric illness having poor orientation of time and place were excluded. Those included in the non-clinical sample were adults not taking any psychiatric medicine for at least the preceding two years.
ASR was first translated into Urdu language keeping in mind the cultural considerations. Followed by a committee approach, the items of Urdu version were finalised. For language and content equivalence back-translation into English language was done. A three-member committee comprising experts than compared the back-translated English version and the original English version. The final ASR Urdu tool was thus finalised before it was distributed among the subjects along with a form seeking demographic information after taking informed consent. It took 30-40 minutes for each individual to complete the questionnaire.
The 99-item ASR was used to assess individuals' responses on questions related to the preceding six months. Items were rated on a three-point Likert scale, with 0 repressing 'never' and 2 representing 'very often'. The items are grouped under 8 subscales. Following the procedures, the three-point scale was converted to a two-point format, where 0 represented absence of a behaviours, and 1 represented presence of that behaviour.2
Data was analysed using SPSS 22. Mplus 7 was used to establish the factor structure of ASR. Eight factors were identified: anxious depressed, withdrawn, somatic, thought problems, attention problems, aggressive behaviour, rule breaking, and intrusive. All problems were derived as latent variables and their respective items were considered observed variables. For factor loadings, 0.25 value was considered acceptable.15
For execution of confirmatory factor analysis (CFA), the model was tested using items recommended by the original ASR.1,2 Following the proposed assumptions, all factors were modelled as first order correlated factors, with no hierarchical relation between the factors assumed. The root mean square error of approximation (RMSEA) was selected as the primary index of model fit.13 The value of RMSEA between 0.05 and 0.07 indicated good to moderate model fit. Comparative fit indices measure chi-square in comparison with the baseline model and assumes that all variables are uncorrelated.14 These indices involve Tucker-Lewis Coefficient (TLI) and Comparative Fit Indices (CFI). In the current study, these indicators were considered secondary to RMSEA. However, CFI and TLI values >0.95 are considered indicators of good fit,15 while some have considered it to be too stringent for complex factor models in applied research.16 As such, a less stringent criteria of 0.80 to 0.90 was considered in the current study to indicate acceptable model fit, and >0.90 to indicate good model fit.2,16
Of the 1060 subjects approached, 768(72.5%) participated; 408(53%) outpatients and 360(47%) non-clinical subjects. The overall age range was 18-59 years (Table-1).
Of the 600 psychiatric patients contacted, 408(68%) participated, while of the 460 non-clinical subjects approached, 360(78%) consented to participate.
RMSEA 0.03 indicated a good model fit, while CFI and TLI values were 0.94, indicating good fit. Moreover the chi-square/degrees of freedom (df) value was 2.13, also indicating good model fit (Table-2).
The items were grouped under eight subscales (Table-3).
The alpha reliability for anxious depressed was 0.92, for withdrawn 0.81, for somatic 0. 87, for thought problems 0.70, for attention problems 0.89, for aggressive behaviour 0.88, for ruling breaking 0.85 and intrusive behavioural problems 0.64 (Table-4).
ASR is an assessment tool to assess psychopathology. It is based upon empirically-driven approach and has been widely used across different societies for comprehensive assessment of psychopathology.1,2 Without empirical evidence, findings obtained on one construct cannot be generalised to other societies. Instruments and constructs need to be tested across multiple and diverse societies to warrant generalisability of findings.2 It has been established that it is important to test constructs across societies by employing procedures that are similar to the society from which the model was originally derived. One of the methods commonly employed to test generalisability is CFA.3
In the first phase of the study, ASR was translated in line with procedures suggested by studies.17,18
Findings indicated that default structure of Urdu version of ASR showed a remarkably good fit to the data. The eight syndrome model, as proposed by the authors,1,2 was used in the current study. RMSEA was taken as the primary index of model fit, while CFI and TLI were taken as secondary indices. RMSEA, CFI and TLI indicated a good model fit. Factor loading of items was done >0.25, which was taken as a criterion for acceptable factor loading.19
The eight syndrome model adopted in the current study has been tested across societies for different age groups.2,20,21 It can be thus inferred that the empirically-driven eight syndrome model exhibits generalisability across adolescent and adult psychopathology. It also confirms that the proposed model of psychopathology based upon eight syndromes can be generalised across societies.
It is a commonly held belief that there exists differences in manifestation of child, adolescent and adult psychopathology. As adults spend more time living in a particular society so the symptom manifestation is likely to be influenced by it. But previous researches are indicating that the eight syndrome model has been confirmed across all three age groups providing considerable evidence for generalisability of eight syndrome model.1,2,20,21 It can be thus inferred that a set of genetic factors overlapping with environmental factors converge across societies that is resulting in similar manifestation of psychopathology.
In order to assess internal consistency alpha reliabilities were computed. Reliabilities for eight narrow-band scales ranged from 0.64 to 0.92, showing acceptable to excellent internal consistency. Notably, low reliability was found for intrusive problems. Low reliabilities for narrow-band scales have been also observed in earlier studies.22 Diverse reasons can contribute to low internal consistency, like number of items in subscales and homogeneity in responses. For clinical scales, the common practice is to rely on pre-defined criteria of psychological constructs, which leads to scales that are often homogenous, and which ultimately affect the predictive validity of scales.23-25 In addition, critically reviewing the process of scale development of ASR indicates that items are derived from the mental health-related presenting complaints at the time intake of patients. This results in development of scales and subscales that have moderate internal consistencies, but replicable structure and good validity.22,24,26
The Urdu version of ASR appeared to be a reliable and valid tool for the assessment of adult psychopathology. The scale exhibited good internal consistency and can be used for establishing prevalence rate of mental health-related problems in Pakistan.
Disclaimer: The text is based on a PhD thesis.
Conflict of Interest: The individual who signed the ethics review form is a co-author.
Source of Funding: None.
1. Achenbach TM. Achenbach system of empirically based assessment (ASEBA). Child Adolesc Psychiatry Ment Health. 2019; 13:30.
2. Ivanova MY, Achenbach TM, Rescorla LA, Turner LV, Ahmeti-Pronaj A, Au A, et al. Syndromes of self-reported psychopathology for ages 18–59 in 29 societies. J Psychopathol Behav Assess. 2015; 37:171-83.
3. Achenbach TM, Rescorla LA, Maruish ME. The Achenbach system of empirically based assessment (ASEBA) for ages 1.5 to 18 years. The use of psychological testing for treatment planning and outcomes assessment. Child Psychiatry Hum Dev. 2016; 47:194-201.
4. Miller MJ, Sheu H. Conceptual and measurement issues in multicultural psychology research. Handbook of Counseling Psychol. 2008; 4:103-20.
5. Geisinger KF. Cross-cultural normative assessment: translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychol Assess. 1994; 6:304-12.
6. Poortinga YH. Equivalence of cross‐cultural data: An overview of basic issues. Int J Psychol. 1989; 24:737-56.
7. Van de Vijver FJ, Poortinga YH. Towards an integrated analysis of bias in cross-cultural assessment. Eur J Psychol Assess. 1997; 13:29-37.
8. Achenbach TM, Rescorla LA. Developmental issues in assessment, taxonomy, and diagnosis of psychopathology: Life span and multicultural perspectives. Dev Psychopathol. 2016; 29:1-48.
9. Collins PY, Patel V, Joestl SS, March D, Insel TR, Daar AS, et al. Grand challenges in global mental health. Nature. 2011; 475:27-30.
10. Achenbach TM. As others see us: Clinical and research implications of cross-informant correlations for psychopathology. Curr Direct Psychol Sci. 2006; 15:94-8.
11. Butcher JN, Han K. Methods of establishing cross-cultural equivalence. In: JN Butcher, Han K, eds. International adaptations of the MMPI-2: Research and clinical applications. Minnesota: University of Minnesota Press, 1996; pp-44-63.
12. Rescorla LA, Achenbach TM, Ivanova MY, Turner LV, Árnadóttir H, Au A, et al. Collateral reports and cross-informant agreement about adult psychopathology in 14 societies. J Psychopathol Behav Assess. 2016; 38:381-97.
13. Hoe SL. Issues and procedures in adopting structural equation modelling technique. J Appl Quant Methods. 2008; 3:76-83.
14. Kyriazos TA. Applied psychometrics: sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychol. 2018; 9:2207-030.
15. Field A. Discovering statistics using IBM SPSS statistics. Sage; 2013.
16. Yu CY. Evaluating cut-off criteria of model fit indices for latent variable models with binary and continuous outcomes. Los Angeles: University of California, 2002.
17. McDonald RP, Ho MH. Principles and practice in reporting structural equation analyses. Psychol Methods. 2002; 7:64-82.
18. Hu LT, Bentler PM. Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struc Equat Model. 1999; 6:1-55.
19. Marsh HW, Hau KT, Wen Z. In search of golden rules: Comment on hypothesis-testing approaches to setting cut-off values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Struc Equat model. 2004; 11:320-41.
20. Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross‐cultural health care research: a clear and user‐friendly guideline. J Eval Clin Pract. 2011; 17:268-74.
21. Mir I, Kamal A, Masood S. Translation and Validation of Dutch Workaholism Scale. Pak J Psychol Res. 2016; 31:331-46
22. Rescorla L, Achenbach T, Ivanova MY, Dumenci L, Almqvist F, Bilenberg N, et al. Behavioral and emotional problems reported by parents of children ages 6 to 16 in 31 societies. J Emot Behav Disord. 2007; 15:130-42.
23. Rescorla LA, Achenbach TM, Ivanova MY, Harder VS, Otten L, Bilenberg N, et al. International comparisons of behavioral and emotional problems in preschool children: parents' reports from 24 societies. J Clin Child Adolesc Psychol. 2011; 40:456-67.
24. Magai DN, Malik JA, Koot HM. Emotional and behavioural problems in children and adolescents in Central Kenya. Child Psychiatry Hum Dev. 2018; 49:659-671.
25. Smits N, van der Ark LA, Conijn JM. Measurement versus prediction in the construction of patient-reported outcome questionnaires: can we have our cake and eat it? Qual Life Res. 2017:1673-82.
26. Bongers IL, Koot HM, Van der Ende J, Verhulst FC. The normative development of child and adolescent problem behaviour. J Abnorm Psychol. 2003; 112:179-92.