Despite the dueling headlines generated by both reports, many steps remain in order to understand this medication (and many others being tested). For one thing, findings from the China study were not published in a journal or peer-reviewed. That process, in which experts assess the quality of research before allowing its release, is important, as is how studies are often further dissected after publication. “Doctors sit down around the table in ‘journal clubs’ and have vigorous discussions about whether the design of a study allows it to adequately answer the questions it claims to,” Doroshow explains. This often leads to additional research.
The incremental, often zigzag pace of science is playing out in many other areas of coronavirus news. For instance, estimated fatality rates for the disease change regularly. “This takes us back to grade-school fractions,” Doroshow says, because rates vary dramatically, depending on if you are calculating people who have died as a percentage of only those hospitalized with documented COVID-19 or counting everyone who is thought to have it (whether or not they're hospitalized). As more people in the community are tested, rates will keep changing, she says; in the meantime, experts make the best estimates they can.
Masks are another topic subject to refinement. Initially, experts focused on whether a mask could keep a person from contracting the disease and determined that common cloth masks do not. But then studies began to emerge about how masks keep droplets from an infected person from spreading as readily to others. Now the Centers for Disease Control and Prevention (CDC) recommends that people wear cloth face coverings to slow the virus's spread.
What to consider when you look at studies
When evaluating news about COVID-19 research, look closely at the number of people studied, recommends Leslie McClure, chair of epidemiology and biostatistics at Drexel University in Philadelphia. If the group is small (generally fewer than 100), the results may not be generalizable to others. Also ask yourself whether the benefits shown seem important, such as whether a treatment saves lives. And don't confuse correlation with causation. A correlation means that in a given study, two things seem to be linked; those results are a lot softer than outcomes that prove one thing definitively causes another.
It's meaningful to note, as well, where the study was published, which can range from a medical journal to a “preprint” site (which is not peer-reviewed) to a researcher's personal website. But right now, even well-regarded journals are publishing the kinds of preliminary studies about the coronavirus they wouldn't for other diseases, such as research with only a handful of subjects or those that didn't give a placebo to half the participants (making them the control group) in a study of a particular virus treatment, says Holly Fernandez Lynch, assistant professor of medical ethics at the University of Pennsylvania.
Ethicists are debating whether this is a good idea (Lynch thinks it isn't helpful in the long run to sacrifice quality for speed), but it means that even findings published in major journals are likely to be clarified or changed by later research, she says.
When this happens, it isn't a sign that science isn't working but that it is, Doroshow asserts. “The scientific process involves being humble enough to know that what we thought to be true can change over time. This doesn't mean we're doing a bad job. It means that by evaluating and reevaluating, we're doing a good job."