Dynamic Chiropractic Canada – November 1, 2015, Vol. 08, Issue 11

# Putting Research Into Perspective: Thinking About Confidence Limits

By Keith Charlton, DC, MPhil, MPainMed

When thinking about prevalences or treatment success rates, does 3/10 equal a 30 percent prevalence or success rate? The answer, surprisingly, is no. The fraction 3/10 is a proportion and reflects harvesting of data from which one might be tempted to claim a 30 percent success rate. In fact, this is a proportion, the sizes of which may change with another clinician or in another circumstance.

If we were able to check every human being on the planet and still came up with a 30 percent rate, it would represent a true proportion, but since we cannot, we must accept a sample that may vary from one sample-taking to another. Sampling "error" is what happens when we happen (randomly) to get 3/10 on Monday and 5/10 on Tuesday. Which is right and how confident can we be that any answer is representative of the true success rate?

A Formula

We need a formula that will allow us to say we are 95 percent confident that the real proportion lies somewhere within A and B, the upper limit and lower limit of the values observed.

The formula to use is:

In the above formula, "p" is the observed proportion, "n" is the number of subjects, 1.96 is a coefficient that generates a 95 percent probability and "p*" is the range within which there is a 95 percent chance the true proportion actually lies.

Let's look at our 3/10 rate using this formula:

Thus:

Interpretation

In sampling 10 subjects and finding an index condition in three, the prevalence of that condition is not necessarily 30 percent. It may be as low as 2 percent or as high as 58 percent! How can we improve our utility? Look at "n."  The bigger n becomes, the tighter the confidence limits. That is why we usually need reasonably large numbers to reassure ourselves our estimates are realistic.

Another Use

Could a new treatment be placebo? Consider an investigator who audits a new (or old) treatment that has an apparent success rate of 8/10. Is this useful - or could it just be placebo? Let's compute the confidence intervals! If we do, we find that the confidence interval for 8/10, where "n" is 10, is 55 percent to 100 percent.

On the face of it, this looks OK, but recall that we have already calculated the CI for 3/10 (the rate conventionally accepted as the rate of placebo response in pain conditions): 2 percent to 58 percent. Note the two ranges overlap, so 8/10 may just be placebo!

Thanks again for thinkin' with me!

Resources

• Armitage P, Berry G. Statistical Methods in Medical Research, 3rd Edition. Oxford: Blackwell, 1994: pp. 93-125.
• Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology. A Basic Science for Clinical Medicine, 2nd Edition. Boston: Little, Brown. 1991: pp. 175-6.
• Bogduk N. Truth in musculoskeletal medicine: I. Confidence intervals. Aust Musculoskel Med, 1997 Nov:13-16.