Putting Research Into Perspective: Thinking About Confidence Limits
By Keith Charlton, DC, MPhil, MPainMed
When thinking about prevalences or treatment success rates, does 3/10 equal a 30 percent prevalence or success rate? The answer, surprisingly, is no. The fraction 3/10 is a proportion and reflects harvesting of data from which one might be tempted to claim a 30 percent success rate. In fact, this is a proportion, the sizes of which may change with another clinician or in another circumstance.
If we were able to check every human being on the planet and still came up with a 30 percent rate, it would represent a true proportion, but since we cannot, we must accept a sample that may vary from one sample-taking to another. Sampling "error" is what happens when we happen (randomly) to get 3/10 on Monday and 5/10 on Tuesday. Which is right and how confident can we be that any answer is representative of the true success rate?
We need a formula that will allow us to say we are 95 percent confident that the real proportion lies somewhere within A and B, the upper limit and lower limit of the values observed.
The formula to use is:
In the above formula, "p" is the observed proportion, "n" is the number of subjects, 1.96 is a coefficient that generates a 95 percent probability and "p*" is the range within which there is a 95 percent chance the true proportion actually lies.
Let's look at our 3/10 rate using this formula:
In sampling 10 subjects and finding an index condition in three, the prevalence of that condition is not necessarily 30 percent. It may be as low as 2 percent or as high as 58 percent! How can we improve our utility? Look at "n." The bigger n becomes, the tighter the confidence limits. That is why we usually need reasonably large numbers to reassure ourselves our estimates are realistic.
Could a new treatment be placebo? Consider an investigator who audits a new (or old) treatment that has an apparent success rate of 8/10. Is this useful - or could it just be placebo? Let's compute the confidence intervals! If we do, we find that the confidence interval for 8/10, where "n" is 10, is 55 percent to 100 percent.
On the face of it, this looks OK, but recall that we have already calculated the CI for 3/10 (the rate conventionally accepted as the rate of placebo response in pain conditions): 2 percent to 58 percent. Note the two ranges overlap, so 8/10 may just be placebo!
Thanks again for thinkin' with me!
Page printed from: