By Author
  By Title
  By Keywords

July 2012, Volume 62, Issue 7

Original Article

Impact of probability estimation on frequency of urine culture requests in ambulatory settings

Naheed Gul  ( Department of Medicine, Shifa College of Medicine, Islamabad, Pakistan. )
Mujtaba Quadri  ( Department of Medicine, Shifa College of Medicine, Islamabad, Pakistan. )

Abstract

Objective: To determine the perceptions of the medical community about urine culture in diagnosing urinary tract infections.
Methods: The cross-sectional survey based of consecutive sampling was conducted at Shifa International Hospital, Islamabad, on 200 doctors, including medical students of the Shifa College of Medicine, from April to October 2010. A questionnaire with three common clinical scenarios of low, intermediate and high pre-test probability for urinary tract infection was used to assess the behaviour of the respondents to make a decision for urine culture test. The differences between the reference estimates and the respondents\\\' estimates of pre- and post-test probability were assessed. The association of estimated probabilities with the number of tests ordered was also evaluated. The respondents were also asked about the cost effectiveness and safety of urine culture and sensitivity. Data was analysed using SPSS version 15.
Results: In low pre-test probability settings, the disease probability was over-estimated, suggesting the participants\\\' inability to rule out the disease. The post-test probabilities were, however, under-estimated by the doctors as compared to the students. In intermediate and high pre-test probability settings, both over- and under-estimation of probabilities were noticed. Doctors were more likely to consider ordering the test as the disease probability increased. Most of the respondents were of the opinion that urine culture was a cost-effective test and there was no associated potential harm.
Conclusions: The wide variation in the clinical use of urine culture necessitates the formulation of appropriate guidelines for the diagnostic use of urine culture, and application of Bayesian probabilistic thinking to real clinical situations.
Keywords: Urine culture and sensitivity, Urinary tract infection, Probability. (JPMA 62: 686; 2012).

Introduction

Urinary tract infection (UTI) is a common clinical problem both in primary and secondary care settings.1 In Pakistan, it is reported to be 17%2 by some studies and 30% according to one study that was conducted on women only.3 Almost half of all women will experience at least once UTI during their lifetime. Establishing a cost-effective strategy for the diagnosis of UTI is, therefore, important. Laboratory tests should be requested only if they are likely to alter further management.
A practical, cost-effective approach to diagnose UTI is limited by the lack of a gold standard. At the heart of this problem lies a debate over what actually constitutes a UTI. Traditionally, 105 cfu/ml in cultured urine was the threshold. More recently, low colony counts 102 to 104 may simply represent early UTI.4 Moreover, symptoms associated with low and high counts of colonies respond equally to antibiotic treatment.
Commonly urinalysis by dipstick and microscopy are used to diagnose UTI. Urine culture is more expensive, time-consuming and is generally not recommended unless compelling situations like recurrent UTI or complicating factors (Diabetes mellitus, immunosuppression, urologic structural/functional abnormality, nephrolithiasis, recent hospitalisation, catheterisation, and symptoms for more than 7 days) are present.5
For the last two decades, there has been an increasing trend in the number of investigations ordered for diagnostic purposes, which has increased the cost burden of management for the patients.6 By practising evidence-based medicine,7 we can alter the trend. Common diagnostic tests should be utilised only if indicated by pre-test probabilities, likelihood ratios and benefit-and-harm balance of diagnostic tests.
The diagnostic evaluation of UTI begins with an estimation of the pre-test probability based on the symptoms. It facilitates the diagnostic process, as it eliminates the need for further testing if the pre-test probability is very high for a target disorder.8 Available data indicates that dysuria, haematuria, nocturia, frequency and urgency increase the probability of UTI. Similarly, the presence of vaginal discharge decreases the likelihood of UTI.9
Moreover, the usefulness of pre-test probability as a derivative of prevalence in decision making has also been proven.10 This implies that if there is a high pre-test probability for a particular disorder, the physician should start treatment without further tests. In addition, it is important to consider the benefits and harms; the costs and inconvenience.11 This is especially important while considering urine culture and sensitivity for diagnostic purposes.
There are many studies confirming the utility of clinical prediction rules (pre-test probability and likelihood ratios) helping the diagnostic process from other parts of the world.12,13 Unfortunately no data is available from Pakistan whether our physicians are following these principles in making a diagnosis of UTI.
This study tried to assess whether urine cultures were appropriately utilised and whether the clinician made the best use of the diagnostic tools in clinical practice as part of a cost-effective strategy to reduce the number of unnecessary investigations.

Patients and Methods

The descriptive study was conducted at the Shifa International Hospital, Islamabad, which is a 550-bed, tertiary-level, community-based hospital. The study duration was seven months, from April to October, 2010, and it comprised 200 participants, including doctors of different clinical cadres and final year medical students of the Shifa College of Medicine, willing to participate. The curriculum of the students included introduction to the basic principles of epidemiology, design of clinical trials, introduction to Bayes\\\' theorem14 and the use of probabilistic approach in clinical diagnosis. Consecutive sampling was used and those who did not give consent for participation in the trial were excluded. Approval from the institutional review board was also obtained before data collection.
The participants were asked to fill a questionnaire. Three scenarios for UTI were provided (Table-1).

For each scenario, five questions were asked: 1. The pre-test probability for UTI in this case is: a) Low (20%), b) Intermediate (50%), or c) High (80%); 2. The post-test probability for detecting UTI in this case after urine culture and sensitivity is likely to be (mention your own figure e.g. 95%) ---- %; 3. Would you suggest urine culture and sensitivity to this patient? (Yes/No); 4. Is there any potential harm associated with this test? (Yes/No); 5. This test is cost-effective? (Yes/No).
The disease probability estimates of the respondents were estimated by two parameters, and evaluated the association of these probability estimates with the test ordering behaviour: 1) Respondents estimates were defined as the pre-test probability and post-test probability estimates; 2) Reference estimates of the pre-test probability were defined by a panel of experts for each scenario. Reference estimates of the post-test probability were calculated by substituting the reference estimates of pre-test probability into the Bayes formula. In literature, the adjusted sensitivity and specificity is 50% and 85% respectively, considering 105 colonies/ml as a positive culture.5 The positive likelihood ratio (LR+) of urine culture was, therefore, calculated as 3.33% using the formula (sensitivity/1-specificity). We used this likelihood ratio for estimating post-test probability using the Bayes\\\' nomogram.14
Statistical analysis was performed using statistical package SPSS version 15 where the respondents\\\' estimates of pre-test probability and its mean difference from the corresponding reference estimates were expressed in percentages. Measures of the post-test probability estimates were expressed as mean ± SD (Standard Deviation). Difference in estimates between groups of doctors was analyzed using Chi-square test. Graph pad software15 was used to evaluate statistical significance of difference in post test probability estimates using independent sample t test in which the post test probabilities of the respondents were compared to the actual/reference post test probability calculated in each scenario using Fagan\\\'s nomogram for Bayes,s theorem. Whether these probability estimates affected the number of urine C/S ordered was also evaluated.

Results

Of the 200 who comprised the initial study population, 145 (72.5%) responded whose baseline characteristics were noted (Table-2).


In scenario one of low pre-test probability, a significant number (n=55; 37.9%) of the respondents overestimated the pre-test probability from the reference estimates. The correct answer (20%) was marked by only 90 (62.1%); 38 (26.2%) reported it as 50%; 17 (11.7%) reported it as 80%, over-estimating it four times of the reference value. There was no statistically significant difference (p = 0.16) among the three groups: (Group 1 - students and house officers; Group 2 - medical officers and residents and Group 3 - consultants. The post-test probability estimates, the mean difference from the reference post-test probability and its significance were also worked out (Table-3).

Urine culture and sensitivity was suggested to the patient in the scenario by 32 (22.06 %).
In scenario no. 2 of intermediate pre-test probability, the correct answer (50%) was provided by 72 (49.7%); 28 (19.3%) under-reported it as 20% 45 (31%) reported it to be 80%, over-estimating it by > 50% of the reference estimates.. There was no statistically significant difference (p = 0.46) among the three groups. Post-test probability estimates etc. were also assessed. A total of 83(57.24%) participants suggested urine culture and sensitivity to the patient in the scenario.
In scenario 3 of high pre-test probability, 96 (66.2%) correctly answered as 80%. The pre-test probability was reported as 20% by 6 (4.1%), while 43(29.7%) reported it as 50%. There was no significant difference (p = 0.10) among the three groups. Post-test probability estimates calculated. A total of 122 (84.13%) participants suggested urine culture and sensitivity to the patient in the scenario. As such, more tests were considered in the case compared to scenarios 1 and 2.
In all the three scenarios, 142 (97.93%) thought that there was no potential harm associated with this test, while 3(0.02%) were of the opposite opinion. In scenario 1, 2 and 3, 42 (28.96%), 64 (44.13%) and 110 (75.86%) respectively were of the opinion that it was a cost-effective procedure.

Discussion

The most remarkable finding in our study was that over half of the respondents in low, and a quarter in intermediate pre-test probability settings failed to rule out UTI, mainly because of poor clinical prediction of disease probability. In high disease probability settings, most of the respondents estimated the pre-test probability that was close to the reference estimates. In all the three scenarios, most of the participants, except for the students, under-estimated the post-test probabilities that were different from the reference estimates. This shows that they cannot rule in the disease in intermediate and high probability settings. Whether this over-estimation of post-test probability is because of their over-reliance on knowledge or their inaptitude in applying the Bayes\\\' theorem to clinical situations, needs further evaluation.
In all the three scenarios, variation in the respondents\\\' estimated probabilities reflected the variation in the diagnostic thinking process in the same clinical setting and was probably responsible for the diversity in the test-ordering behaviour in clinical practice. This finding is further confirmed by the wide standard deviation in the estimated post-test probabilities in all the scenarios. The current study was performed using the more common perception among general medical practitioners of 105 colonies/ml as a positive urine culture for UTI diagnosis which showed great variability between the respondents\\\' and the reference estimates of probabilities. If the more stringent IDSA (Infectious Disease Society of North America) criteria15 of 103 colonies/ml was used, the likelihood ratio would have been much higher i.e. 8.0 and hence the magnitude of the effect or gap in the pre- and post-test estimates would have been much larger.
There was a positive association between the post-test probability and the decision to order the test, which means that the higher the post-test probability, the more likely the subject was to consider ordering the test. This increasing trend in the number of tests ordered is obvious in the results as one moves from scenario 1 of low pre-test probability to scenario 3 of high pre-test probability. This is an important concern in a resource-limited country like Pakistan where it can increase the economic burden in terms of health care costs.
One study16 assessed the effects of test results on the estimation of disease likelihood using hypothetical scenarios of breast lump. The participants consistently over-estimated the post-test probabilities associated with a positive test result. Somewhat similar results were found in a study done on students17 who used hypothetical patients with coronary artery disease. Our study was different in terms of the subjects. In addition to the students, participants of different cadres with clinical experience, were also included. Also, both over- and under-estimation of test results were noticed in our study.
Interestingly, there was no significant difference between the different levels of participants. Students and house officers who have very little experience performed almost similar to those with different levels of experience. It was inferred that students were familiar with the clinical decision tools and Bayesian probabilistic reasoning since the first year of their medical school and, hence, more adept at using these tools compared to the clinicians who had received the traditional medical education. This, however, needed further evaluation.
Most of the subjects were of the opinion that there was no potential harm associated with this test. Urine C/S is considered as an expensive procedure compared to urine microscopy and leukocyte esterase and nitrite tests.18 When asked about the cost-effectiveness, most of the respondents considered it as more and more cost-effective as the disease probability increased. This is contrary to the evidence-based approach which emphasises on less and less tests as the disease probability increases.
In terms of the study\\\'s limitations, we modelled the subjects\\\' diagnostic thinking process according to the Probabilistic/Bayesian approach19 which is not a natural thinking process, but is acquired only through education. Besides, as these were dummy cases, they may fail to account for individual judgment according to real-life situations.

Conclusion

There was a wide variation in the clinical use of urine C/S which necessitates the formulation of appropriate guidelines for the diagnostic use of the test in UTI. Moreover, considering the recent changes in the diagnostic criteria and variations in the bacterial resistance patterns, international guidelines for UTI may not be applicable in our setup and may need adaptations/modifications. More emphasis on clinical decision-making and the incorporation of Bayesian probabilistic thinking in clinical practice can help streamline the diagnosis of UTI.

References

1. Foxman B. Epidemiology of urinary tract infections: incidence, morbidity, and economic costs. Am J Med 2002; 113 (Suppl 1A): 5S-13S.
2. Bhurt AW, Bozdar NM, Fikree FF. Prevalence and risk factors of presumptive urinary tract infection in a rural community. JCPSP 2000; 10: 16-9.
3. Sheikh MA, Khan MS, Khatoon A, Arain GM. Incidence of urinary tract infection during pregnancy. East Mediterr Health J 2000; 6: 265-71.
4. Mehnert-Kay SA. Diagnosis and management of uncomplicated urinary tract infections. Am Fam Physician 2005; 72: 451-6.
5. Gradwohl C, Foude K, Harrison V, Munger K, Zoschnick L. University of Michigan Health System Guidelines for Clinical Care: Urinary tract infection. (Online) (Cited 2010 May 8). Available from URL: http://cme.med.umich.edu/pdf/guideline/UTI.pdf.
6. Williams S, Eisenberg J. A controlled trial to decrease the unnecessary use of diagnostic tests. J Gen Intern Med 1986; 1: 8-13.
7. Grover ML, Bracamonte JD, Kanodia AK, Bryan MJ, Donahue SP, Warner AM, et al. Assessing adherence to evidence-based guidelines for the diagnosis and management of uncomplicated urinary tract infection. Mayo Clin Proc 2007; 82: 181-5.
8. McGinn T, Jervis R, Wisnivesky J, Keitz S, Wyer PC. Tips for teachers of evidence-based medicine: clinical prediction rules (CPRs) and estimating pretest probability. J Gen Intern Med 2008; 23: 1261-8.
9. Giesen LG, Cousins G, Dimitrov BD, van de Laar FA, Fahey T. Predicting acute uncomplicated urinary tract infection in women: a systematic review of the diagnostic accuracy of symptoms and signs. BMC Fam Pract 2010; 11: 78.
10. Richardson WS, Wilson MC. The Process of Diagnosis. In: Guyatt G, Rennie D, Meade MO, Cook DJ, editors. Users\\\' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2 ed. United States of America: The McGraw-Hill Companies, Inc.; 2008; pp 399-406.
11. Papanikolaou PN, Ioannidis JP. Availability of large-scale evidence on specific harms from systematic reviews of randomized trials. Am J Med 2004; 117: 582-9.
12. Luciani D, Cavuto S, Antiga L, Miniati M, Monti S, Pistolesi M, et al. Bayes pulmonary embolism assisted diagnosis: a new expert system for clinical use. Emerg Med J 2007; 24: 157-64.
13. Green AD, Colon-Emeric CS, Bastian L, Drake MT, Lyles KW. Does this woman have osteoporosis? JAMA 2004; 292: 2890-900.
14. Fagan TJ. Letter: Nomogram for Bayes theorem. New Eng J Med 1975; 293: 257.
15. Mehnert-Kay SA. Diagnosis and management of uncomplicated urinary tract infections. Am Fam Physician 2005; 72: 451-6.
16. Lyman GH, Balducci L. Overestimation of test effects in clinical judgment. J Cancer Educ 1993; 8: 297-307.
17. Noguchi Y, Matsui K, Imura H, Kiyota M, Fukui T. Quantitative evaluation of the diagnostic thinking process in medical students. J Gen Intern Med 2002; 17: 839-44.
18. D\\\'Souza Z, D\\\'Souza D. Urinary tract infection during pregnancy--dipstick urinalysis vs. culture and sensitivity. J Obstet Gynaecol 2004; 24: 22-4.
19. Richardson WS, Wilson MC, McGinn TG. Differential Diagnosis. In: Guyatt G, Rennie D, Meade MO, Cook DJ, editors. Users\\\' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2 ed. United States of America: The McGraw-Hill Companies, Inc.; 2008; pp 407-18.

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: