Scientists have found:
A low-fat, high-fiber diet halves the risk of death from all causes.
Antiarrhythmic drugs save the lives of heart attack patients.
Incoming e-mails and phone calls lower IQ by 10 points.
Wrong, wrong and wrong again.
In fact, David H. Freedman argues in his new book Wrong: Why Experts Keep Failing Us—and How to Know When Not to Trust Them, most scientific and medical studies that you see or hear mentioned in news reports turn out to be incorrect, misleading or irrelevant to most people. And, unfortunately, that's true even if they're published in a major peer-reviewed journal. Other commonly offered expert advice, on everything from investing to home repair, is equally suspect, Freedman says.
The forces pushing bad research to the forefront are plentiful, he argues. They start with a bias for publishing surprising new results, and are compounded by a range of researchers' sins, including mismeasurement, manipulation of data, conflicts of interest, ineptitude and outright fraud.
In an interview with the AARP Bulletin, Freedman discussed how consumers should use expert advice and how to recognize its pitfalls.
Q. Let's look at a specific finding. There was a report July 29 in the British Medical Journal that people taking calcium supplements were about 30 percent more likely to suffer a heart attack. If experts are usually wrong, how should people reading about this report react?
A. People shouldn't reject all expert advice that they read about in the papers. What we want is to learn how to separate the stuff that's more likely to be true from the stuff that's less likely to be true. Now it turns out that, for a variety of reasons, medical conclusions about taking supplements or eating certain types of food tend to be among the least reliable of all medical advice.
Q. How does that apply in this particular case?
A. It should set off alarm bells, but I definitely wouldn't recommend stopping calcium based on this, or basically making any medical decisions based on what you read in the paper without talking to your doctor. A supplement like calcium is going to have many different effects on the body. It's going to affect different people in different ways. It's going to affect the same people in different ways at different stages of their lives. There are so many complexities, the fact that someone tries to boil it down to a simple conclusion already means that the advice probably isn't going to be that useful. On the other hand, if there are several more studies that come out over the next few years that start to support this notion that calcium may cause more problems than it solves, then people might seriously consider stopping or limiting their intake of calcium.
Q. What are the clues that a study is likely to be valid?
A. First, it's a very large study that has followed thousands of people for five or 10 or even 20 years.
Also, it's a randomized study, which means one group got a treatment or did one thing and a different group didn't, and people were randomly assigned to the two groups.
And the most important thing is that the finding agrees with findings of many similar studies that have been done over a period of years.
Q. Let's say I hear about a study on some topic that's very important to me. Where would I look, beyond the mass media, for information about what it actually says and whether it's likely to be valid?
A. Obviously, the best place to go is your doctor, and if you don't have a doctor whom you feel comfortable talking with about this sort of thing, then you need a new doctor. Beyond that, go to medical journals. Even if you don't understand all the technical terms, it's still written in English, and it is understandable. It's often surprising for a lay person to look at a medical journal finding and compare it to a news report. Anybody can see that there's stuff that the mass media gets wrong, or leaves out, or distorts in some way.