Is as much as 90-percent of the published medical information flawed?
The 90-percent statistic comes from a column titled “Lies, Damned Lies, and Medical Science,” written by David H. Freedman, published by the Atlantic. The column focuses on the work done by meta-researcher Dr. John Ioannidis who is known as one of the world’s foremost authorities on the credibility of medical research. Dr. Ioannidis notes that medical research can be negatively influenced by the researchers wanting to fund tenured positions, keep jobs, and win attention.
The column in the Atlantic goes on to note that the problem of flawed research is not limited to the medical community. Though not specifically called to the carpet, certainly the world of clinical sport, sport medicine and sport performance cannot be immune from flawed research.
Some of the research claims disputed in the column leaves readers with a feeling of helplessness. If you can’t trust research “science,” then who do you believe?
It is easy to read the column and get the idea that all research is flawed. Read again and you’ll note that Dr. Ioannidis is not claiming all research is flawed.
To help you sort out research claims before you enter your VISA number to have the next miracle performance enhancer delivered to your doorstep, consider the following excerpt edited from Bicycling for Women:
Evaluating Research Claims
- Was the study published in a reputable scientific journal?
- Who were the researchers and did they have financial incentives to gain by the results of the experiment?
- What are the limitations of the study? Were the studies done on animals and not humans? Were the studies done on populations which are significantly different than those being questioned by the hypothesis? For example, if we want to determine the effects of 60-second interval training on masters level female cyclists, we don’t want the study group to be 20-year-old sedentary males. At best, the experiment may lend suggested relationships between the question and the results from the study group.
- Is the study "an amazing discovery?" Are there studies which directly contradict the amazing find? Or does the study, in some way, support research which is already known? This particular point will need to be reviewed, if there is a lack of past research on the subject.
- What was the size of the study group? A study on hundreds of people will have more weight than a study of ten people.
- What time period did the study cover? Many studies only cover a short time, a few weeks or months. How our bodies react to nutrients or a training stimulus are complex interactions of multiple factors. These reactions may take long periods of time.
If he has done nothing else, Dr. Ioannidis has brought the attention to the issues. Based on the buzz I’ve seen on the sport and science discussion boards, everyone wants to eliminate the problems with research reliability and that will help all of us.