## Putting Research Into Perspective: Thinking About Confidence LimitsBy Keith Charlton, DC, MPhil, MPainMed When thinking about prevalences or treatment success rates, does 3/10 equal a 30 percent prevalence or success rate? The answer, surprisingly, is no. The fraction 3/10 is a proportion and reflects harvesting of data from which one might be tempted to claim a 30 percent success rate. In fact, this is a proportion, the sizes of which may change with another clinician or in another circumstance. If we were able to check every human being on the planet and still came up with a 30 percent rate, it would represent a true proportion, but since we cannot, we must accept a sample that may vary from one sample-taking to another. Sampling "error" is what happens when we happen (randomly) to get 3/10 on Monday and 5/10 on Tuesday. Which is right and how confident can we be that any answer is representative of the true success rate?
We need a formula that will allow us to say we are 95 percent confident that the real proportion lies somewhere within A and B, the upper limit and lower limit of the values observed. The formula to use is: In the above formula, "p" is the observed proportion, "n" is the number of subjects, 1.96 is a coefficient that generates a 95 percent probability and "p*" is the range within which there is a 95 percent chance the true proportion actually lies. Let's look at our 3/10 rate using this formula: Thus:
In sampling 10 subjects and finding an index condition in three, the prevalence of that condition is not necessarily 30 percent. It may be as low as 2 percent or as high as 58 percent! How can we improve our utility? Look at "n." The bigger n becomes, the tighter the confidence limits. That is why we usually need reasonably large numbers to reassure ourselves our estimates are realistic.
Could a new treatment be placebo? Consider an investigator who audits a new (or old) treatment that has an apparent success rate of 8/10. Is this useful - or could it just be placebo? Let's compute the confidence intervals! If we do, we find that the confidence interval for 8/10, where "n" is 10, is 55 percent to 100 percent. On the face of it, this looks OK, but recall that we have already calculated the CI for 3/10 (the rate conventionally accepted as the rate of placebo response in pain conditions): 2 percent to 58 percent. Note the two ranges overlap, so 8/10 may just be placebo! Thanks again for thinkin' with me!
- Armitage P, Berry G.
*Statistical Methods in Medical Research, 3rd Edition*. Oxford: Blackwell, 1994: pp. 93-125. - Sackett DL, Haynes RB, Guyatt GH, Tugwell P.
*Clinical Epidemiology. A Basic Science for Clinical Medicine, 2nd Edition*. Boston: Little, Brown. 1991: pp. 175-6. - Bogduk N. Truth in musculoskeletal medicine: I. Confidence intervals.
*Aust Musculoskel Med*, 1997 Nov:13-16.
Page printed from: http://www.dynamicchiropractic.ca/mpacms/dc_ca/article.php?id=57530&no_paginate=true&p_friendly=true&no_b=true |