Thanks, Tommy, for letting me address this issue -- I think it's important that everyone be able to distinguish what constitutes "proof" from what doesn't, especially because these days we all have access to so much information, and it is up to us to sort through it all and determine what is reliable from what is not. For doctors and scientists, "proof" involves large well-controlled, randomized, double-blind, prospective studies, preferably independently confirmed. Everything else falls into the category of "suggestive," as in the study you quoted (reading the first line of their abstract). Suggestive studies are a good starting ground for more rigorous investigation but don't prove anything by themselves.
Why have scientists, and the FDA, the USPSTF, and others put up all those requirements? Let's go through the requirements:
- Large: Well, it must be large enough so that the differences are statistically significant. We've all seen polls that state they are accurate to within say ±4% with 95% confidence. A large sample size is necessary to be able to read the results.
- Randomized:Statistics are only meaningful on random samples. Selection bias can confound the projectability of a study.
- Well-controlled: both the treatment group and the control group (which does not receive the treatment) must be matched on a set of relevant demographic characteristics, so that we are not trying to compare apples to oranges. The findings are only projectable to the kind of people it was tested on.
- Double-blind, controlled: Neither the patient nor the doctor can know if he is getting the treatment or the placebo or other control. Doctors who know they are giving the actual treatment may influence the outcome subconsciously. On the patient side, the placebo effect is very real -- people have been really cured by placebos. The treatment has to do significantly better than the placebo to be considered effective.
- Prospective studies: Both groups must be treated the same way from the start. There are a multitude of extraneous variables that may influence results looking backwards (retrospective).
- Replication: If only one investigator "proves" a treatment and all others don't, it leads us to question whether some unknown bias influenced the result that time. External validation is always desirable.
- Time: Especially for PC, it's also necessary that the study run long enough to detect an effect, which can take 20 years or more. The results are only valid for the length of time the observation was made.
Here's a partial list of types of studies that do not
constitute proof: retrospective studies, epidemiological studies, cohort studies, non-randomized studies, uncontrolled prospective studies, pilot studies, in vitro lab studies on tissue samples, in vivo lab studies on mice. They are all important steps along the way.
Because it's so hard to get real proof, the US National Institute of Health (NIH), the US Preventive Services Task Force (USPSTF), as well as other medical organizations have established "levels of evidence" to grade how reliable our data is. Rather than just decide if a treatment is proven or not, they judge its degree
of proof on a scale. Here's the US government's criteria (Sections 4.3 and on) if you want to learn more about it:
A good example of a "proven" Vitamin effect is the SELECT study on the effect of Vitamin E and Selenium. It proved that Selenium supplementation had no effect and that Vitamin E supplementation had a slight deleterious effect.
An example of a study that does not constitute proof is the pomegranate juice study. There was no control group and no one was blinded. Was the decrease in PSA doubling time due to the placebo effect? Did investigators unconsciously pick or treat the patients differently than they otherwise might have. For me, it was especially troubling because it would have been so easy to establish a group of matched controls to receive colored, flavored sugar water or some other fruit juice. It didn't surprise me that the FDA did not allow POM Wonderful to make any health claims based on it. The way they conducted the research made me suspicious myself.
It's extremely troublesome and expensive to conduct studies that will constitute proof. Yet reliable studies do crop up occasionally. There's an interesting one on red yeast rice now, for example. Every grad student wants to make his name by proving the next big overlooked thing. I remember my disappointment in watching lycopene go down -- I had a cabinet full of tomato sauce! I still hold high hopes for a few dietary interventions like soy isoflavones, sulforaphane, red yeast rice, and apigenin. I think that future molecular alterations may render EGCG and curcumin useful too. After seeing Vitamin E proven harmful, I would never recommend to anyone an unproven dietary supplement, but I do monitor them closely and with hope.