Objectives: To assess the strength of agreement for each of the Pirani assessment categories between an orthopaedic surgeon and allied health worker on scoring clubfeet.
Methods: The descriptive cross-sectional study was conducted at the Indus Hospital, Karachi, from November 2012 to June 2013, comprised children below 12 months of age with untreated unilateral or bilateral clubfoot deformity an allied health worker, who was a plaster technician, was given 1-week training in the Pirani Scoring method. The feet were scored by the surgeon and the paramedic independently. SPSS 21 was used for statistical analysis.
Results: The study had 57 patients, and 92 feet were assessed. The difference between the means of score of each parameter, including the hind foot score and total score, was less than 0.09. The means of total score were found to be identical. Total Hind Foot Score was 0.48 and Total Score was 0.354. The statistical inter-observer reliability for all components was rated as substantial to moderate agreement except Equinus Rigidityand Total Score, which showed fair agreement.
Conclusions: Pirani scoring method for clubfoot assessment was found to be a reliable tool for use by plaster technicians for independent assessment of clubfoot. However, prior training and supervision in the early phase is necessary.
Keywords: Pirani scoring, Clubfoot, Inter-observer reliability. (JPMA 64: S-127 (Suppl. 2); 2014).
Congenital talipesequinovarus (CTEV) or clubfoot is one of the most common congenital abnormalities affecting the lower limb with a reported incidence of 1:1000 worldwide and 50% of cases are bilateral.1 Siblings of affected individuals possess up to a 30 times enhanced risk of clubfoot deformity. Likelihood of CTEV connected with monozygotic siblings is 32.5% whereas incidence among dizygotic twins is just 2.9%.2 Since no published data on the incidence of clubfoot is available from Pakistan, data from other South Asian countries can be used to approximate the disease burden. In Bangladesh, it is estimated that 3600-7300 children are born with clubfoot,3 whereas in India, an estimated 30,000 are born with clubfoot.4 Thus, extrapolating this information to the Pakistani population, we estimate that clubfoot incidence in Pakistan would be approximately 5403 children annually with approximately 7000 feet requiring correction. Another problem encountered is lack of trained orthopaedic surgeons. It is estimated that the number of practicing orthopaedic surgeons is less than 300, estimated at around 500-600 now, for a population of 190 million in the whole of Pakistan, almost all of whom are concentrated in urban areas. Considering these estimates, it is not practical to expect that the demand for treatment can be met by the limited number of surgeons. Since this problem exists in high numbers in under-privileged areas of the country, there is a need for a cost-effective treatment. This can be provided by Ponseti method because it requires relatively lower levels of skills and logistics than are required for surgical procedures.1 It is imperative that a uniform system for diagnosis and follow-up is applied to gauge the success of treatment. For this purpose a scoring system is required to provide us with a cost-effective, user-friendly and reliable method for the evaluation of this problem. Pirani's classification is simpler and more recent.5 Pirani scoring system has proved to be useful in Ponseti management of clubfeet.6 In Pakistan, the Ponseti Method project has been initiated at our institution, which is located in a low-income area of Karachi. So far, 250 children have undergone treatment in 1 year and 6 months, and the treatment has been carried out by orthopaedic consultants. However, with increasing demand and need of clubfoot treatment throughout the country, it is not possible to meet the demands or to expect the specialists to continue the treatment which can be successfully managed by non-specialist personnel.
As a first step in demonstrating the possibility of utilising trained allied health worker in the Ponseti casting technique, it is necessary to see if the Pirani score screening tool can be successfully used in assessing the severity of clubfoot by paramedical staff. Pirani score is favoured in studies because of its ease in use and the fact that it does not require motion and X-ray analysis,7 as in some other classifications e.g. the one suggested by Dimeglieo et al.8 The system suggested is much extensive, involving usage of four parameters. It is more suitable for further evaluation of each component of the clubfoot deformity.8 Pirani score is widely used in the prediction of tenotomy and number of castsrequired. If the screening scores between trained orthopaedic surgeons and the paramedic staff are comparable, it would lead to the next step in training non-medical staff the Ponseti method. The current study was undertaken with the primary objective of assessing the strength of agreement for each of the Pirani assessment categories as well as the Total Score between an orthopaedic surgeon and an allied health worker on scoring virgin clubfeet.
Material and Methods
The descriptive cross-sectional study was conducted at the Pehla Qadam Clubfoot Clinic of The Indus Hospital, Karachi, from November 2012 to June 2013, after approval from the institutional review board. The study included children with untreated unilateral or bilateral clubfoot whose parents consented to participate. Any child over 12 months of age or whose parents did not give consent was excluded. Also excluded were children attending the clinic for follow-up treatment. Once a new clubfoot patient fulfilling the inclusion criteria was identified, the study procedures were explained to the parent/guardian accompanying the child and informed consent was obtained. Consecutive sampling was done where every patient meeting the selection criteria was recruited until the sample size was considered satisfactory. Once enrolled, the child's right and left feet were assessed and scored by the 2 raters (the surgeon and the paramedic) at the same time but independently of one another. Two weeks prior to the first assessment session, the raters were given2-day training in the Pirani Scoring method by an experienced consultant. The raters were given demonstrations on anatomy and pathogenesis of clubfoot on dummy models. They were also asked to observe the scoring technique applied by the consultant in the outpatients department (OPD) on at least 10 feet each before being allowed to participate as a rater. The rater scored each clubfoot on a standard Pirani classification proforma which clearly described all the scores with pictures. The assessment was considered complete when all the raters had scored both feet.
Data was analysed using SPSS 21. The leg was considered the unit of analysis and each leg was considered an individual rating in statistical analysis. Descriptive statistics of the child and the raters was assessed. Shapiro Wilk's test was applied to check the normality of quantitative variable age which was not met, and so median [inter quartile range (IQR)] was computed for age. Frequencies and percentages were computed for qualitative variables like gender, and unilateral and bilateral presentations.
The assessment of agreement between the raters was done by using Fleiss kappa for each Pirani score parameter, Total Score (TS), which is the sum of curve lateral border (CLB), medial crease (MC), lateral head of talus (LHT), post crease (PC), equinusrigidity (RE) and empty heel (EH) scores. The same procedure was applied for Hind Foot Score (HFS) which is the sum of PC, ER and EH scores, and the Mid Foot Score (MFS), which is the sum of CLB, MC and LHT scores. Percentage of observed agreement between raters was calculated for each Pirani score parameter.
There were 57 patients in the study with a total of 92 clubfeet. The orthopaedic consultant's score was coded as rater A and the plaster technician's score as rater B. The age of patients ranged from 1 month to 12 months. The overall median age was 5 months IQR: 5-8). Of the 92 feet, 77(83.7%) related to boys and 15(16.3%) to girls. The overall male-to-female ratio was5.066:1.The difference between the mean scores of each parameter, including HFS and TS was less than 0.09. The highest discrepancy was found in RE scores. Means of TS were identical. Kvalues for the observations made were 0.713 for CLB, 0.617 for MC, 0.695 for LHT, 0.669 for PC, 0.37 for RE, 0.542 for EH, 0.48 for HFS and 0.354 for TS. The statistical inter-observer reliability for all components was rated as substantial to moderate agreement except RE and TS that were rated as fair agreement (Table-1).
The Kvalues calculated for the first 15 feet that were assessed in the month of November and December, 2012, were 0.347 for CLB, 0.524 for MC, 0.424 for LTH, 0.820 for PC, -0.123 for RE, 0.6 for EH, 0.123 for HFS and 0.252 for TS.
The Kvalues calculated for the remaining 76 feet assessed in 2013 were 0.753 for CLB, 0.641 for MC, 0.759 for LTH, 0.642 for PC, 0.508 for RE, 0.528 for EH, 0.501 for HFS and 0.373 for TS (Table-2).
The percentage of point-to-point agreement observed in the study ranged from 79% for EH to 70% for MC. Percentage of agreement for the calculated scores 58% for HFS and 41% for TS was also significantly higher than the percentage expected for agreement.
Initially the focus of treatment was towards finding a better surgical technique rather than looking for a better functional outcome. This focus shifted after long-term follow-ups became available.10 Ponseti reported a consistent success rate of 85-90% with serial casting period of 2-4months.11 He recommended post-correction foot abduction orthosis to be worn for a certain period of time to prevent recurrence. Currently the Ponseti approach is being recognised all over the world as the most efficient and affordable treatment method regarding the correction of clubfoot.12 Analysis of his technique and reproducibility of his work has prompted a reconsideration of all other methods of club foot treatment.13 It was found that the feet treated by Ponseti method were more supple, painless and had increased range of motion than the feet that were treated by surgical techniques. Patients who were treated surgically had stiffer feet and often became arthritic in adulthood. This interest in cast treatment further increased with better understanding of the pathology of the disease and consistent treatment.13
Our study successfully demonstrated that the Pirani score screening tool can be successfully used in assessing clubfoot severity by an allied health worker who was trained to use the tool. Our findings were similar to a study conducted in Sudan14 which utilised a trained physician and a trained nurse having three-year diploma and additional experience as a physiotherapist assistant. However, in our study, comparison was made between an orthopaedic surgeon and a lower level allied health worker, i.e. a plaster technician who had two-year operation theatre (OT) technician diploma and was experienced in applying casts.
The inter-observer reliability that our study showed was found to be fair to substantial whereas the Sudanese study showed moderate to substantial inter-observer reliability between the two types of raters.14
Using point-to-point inter-observer agreement for all components of deformity, the mean point of our study was found to be 82.15% which was very similar to the 83% reported in Sudan.14 Another study15 had a higher agreement of 89%. The only difference was that the raters were both physicians of comparable skills.
To see if training could make a difference, we divided our analysis into two parts — the first 15 observations when the experience of assessment would have been less, and the second part for the remaining period of assessment. We found that kappa interpretations were suggestive of an increase in agreement as experience increased. In the first part, the agreement was erratic from a whole range of agreement ranging from less-than-chance to substantial. However, with increased experience, four of the eight assessment points had substantial agreement; three out of eight had moderate; and only one had fair agreement. The result accentuates the fact that the skills of the second rater improved with experience. Pirani scoring is thus a reliable method for clubfoot assessment by lesser skilled assessors provided he is given adequate training and sufficient hands-on experience. Pirani scoring agreement improved with time reflecting increased experience. Other than training, professional supervision in the early phase is also vital.
We recommend the use of Pirani score by trained allied health workers for clubfoot assessment so that the burden on orthopaedic surgeons may be relieved, especially in areas where there is scarcity of orthopaedic surgeons. Secondly, we recommend the training of plaster technicians in Ponseti method for the application of casts so that they may be able to do independent treatment of clubfoot in areas where no physicians are available. Such models have been adopted in underdeveloped countries like Uganda and Malawi.16
Pirani scoring is a cost-effective, user-friendly and reliable method for gauging success with Ponseti method. It is easy to learn and use.
1. Canale ST, Beaty JH, Campbell WC.Campbell's operative orthopaedics. 12th ed. St. Louis, Mo.; London: Mosby; 2012. v. <1-4 (2050 p.) p.
2. Tachdjian MO. Pediatric orthopedics.Philad.1972.
3. Bangladesh . Global Clubfoot Initiative [Online] [Cited 2011 Dec 26]; Available from: URL:http://globalclubfoot.org/countries/bangladesh/.
4. India. Global Clubfoot Initiative [Online] [Cited 2011 Dec 26]; Available from: URL:http://globalclubfoot.org/countries/india/.
5. Pirani S, Hodges D, Sekeramayi F. A reliable & valid method of assessing the amount of deformity in the congenital clubfoot deformity. J Bone Joint Surg Br 2008;90:53.
6. Dyer PJ, Davis N. The role of the Pirani scoring system in the management of club foot by the Ponseti method. J Bone Joint Surg Br. 2006;88:1082-4.
7. Dimeglio A, Bensahel H, Souchet P, Mazeau P, Bonnet F. Classification of clubfoot. J Pediatr Orthop B. 1995;4:129-36.
8. vanMulken JM, Bulstra SK, Hoefnagels NH. Evaluation of the treatment of clubfeet with the Dimeglio score. J Pediatr Orthop. 2001;21:642-7.
9. Pirani S, Staheli L, Naddumba E. Ponseti Clubfoot Management: Teaching Manual For Health-Care Providers In Uganda [Online] 2008 [Cited 2011 Dec 25]; Available from: URL:http://www.global-help.org/publications/books/book_ponsetiuganda.html.
10. Cooper DM, Dietz FR. Treatment of idiopathic clubfoot.A thirty-year follow-up note. J Bone Joint Surg Am 1995;77:1477-89.
11. Ponseti IV. Treatment of congenital club foot. J Bone Joint Surg Am. 1992;74:448-54.
12. Staheli L. Clubfoot: Ponseti Management 3rd edition. Global HELP Organization, 2009.
13. Herzenberg JE, Radler C, Bor N. Ponseti versus traditional methods of casting for idiopathic clubfoot. J Pediatr Orthop 2002;22:517-21.
14. Shaheen S, Jaiballa H, Pirani S. Interobserver reliability in Pirani clubfoot severity scoring between a paediatricorthopaedic surgeon and a physiotherapy assistant. J Pediatr Orthop B 2012;21:366-8.
15. Flynn JM, Donohoe M, Mackenzie WG. An independent assessment of two clubfoot-classification systems. J Pediatr Orthop. 1998;18:323-7.
16. Pirani S, Naddumba E, Mathias R, Konde-Lule J, Penny JN, Beyeza T, et al. Towards effective Ponseti clubfoot care: the Uganda Sustainable Clubfoot Care Project. Clin Orthop Relat Res. 2009;467:1154-63.
This journal is a member of and subscribes to the principles of the Committee on Publication Ethics.