The dangers of undercooked science and a hungry public


The appetite for emerging medical news and discoveries has never been greater than in the past two years, as consumers have taken to online to research breakthroughs, treatments and cures related to COVID-19 . The problem is that those responsible for this information – primarily the scientists, but also the journalists who cover their work – have taken shortcuts by failing to distinguish unverified preliminary research from fully-reviewed findings published in reputable scientific journals.

Here are two quick examples:

Over the summer, poison control centers saw the number of reports of poisonings from people ingesting ivermectin, a drug to treat ringworm in farm animals, more than triple, mistakenly believing it. prevent or treat COVID-19. Although this drug is approved by the FDA to treat certain intestinal and skin conditions in humans, it has not been approved to treat COVID.

And last year, thousands of healthy people took a second-line treatment for rheumatoid arthritis called hydroxychloroquine, mistakenly thinking it would protect them from COVID. The result, again, was more death, not less.

In both examples, scientists first shared their “findings” with the public and without expert review on Internet platforms that were never intended to be seen by the public or covered by journalists as news. . These repositories, called preprint servers, are a kind of library of very preliminary and unverified scientific work. Their stated goal is the rapid dissemination of unedited work for the purpose of obtaining comments from other scientists prior to submission, peer review, and eventual publication in mainstream medical journals.

The problem is that these preprint servers are currently used for very different purposes. Some scientists use them to promote their work (often by making grandiose statements that will never pass peer review), and journalists have generally done a poor job of warning readers that the work is not just unpublished. , but it’s so crude it’s unlikely to ever be released. see the light of day in a responsible medical journal.

Health care consumers should be skeptical of such claims because they put their health and well-being at risk.

Let me explain.

In the past, medical researchers generally did not share their work publicly or promote their findings in the media until they were peer reviewed and published in professional journals. In fact, cutting that wedge would have made it difficult, if not impossible, for these researchers to publish their work, and scientific culture generally frowned upon it as well. Journalists, while not always happy with embargoes on media coverage of new treatments or discoveries, generally respected these embargoes because their own ethical standards prevented them from serving under-prepared science to hungry audiences.

Now cutting this corner is encouraged in medicine. Researchers publish unverified work on preprint servers, and journals cover their unexamined claims – which may be selfish, exaggerated, or incorrect – as if they were groundbreaking discoveries in major medical journals. They are not.

Why is this happening? Incentives. Scientists want to be the first to share a discovery; it may help them get the next grant. They may also think that they are improving their health by getting their ideas known more quickly. Newspapers and cable news channels want their journalists to publish articles.

But what’s good for the media can be bad for you. You have to do your own homework; if you are going to read medical news you should click on the links that take you to the sources: if they are preprint servers beware. What you read has generally not been rated by experts in the field. On the other hand, if the source is a journal like the New England Journal of Medicine or the Journal of the American Medical Association, your chances of the content being trustworthy are much higher.

Journalists shouldn’t force you to do this. They should avoid using preprints as sources in news articles.

There have been numerous examples of journalists covering preprints that have been published to augment a company’s stock, to promote a politically motivated conspiracy theory, or to undermine confidence in prudent science and public health measures. News articles themselves – disseminated even more widely through the use of social media by journalists – further amplify these unexamined scientific claims and, when proven to be inaccurate, further undermine public trust.

I recognize that medical journals don’t always get it right, and preprint servers sometimes have. It took 12 years for the Lancet to remove the article mistakenly linking vaccines to autism. But the fact that journals and preprints may contain errors does not make them equivalent, or even comparable. Pre-prints are available free of charge; major journals, like the one I edit, have to convince the world that our content is worth paying for. We go to great lengths to make sure this is true.

I can see why researchers looking for their next grants would want to get their work out quickly and with little oversight, and why they might not want to be forced to respond to peer experts who might insist that they edit exaggerated messages. , inject nuance, correct errors or demand disclosure of industry involvement in the work. It’s less clear to me why a journalist would cover this as news, or why we as readers should trust him.

It can be difficult for health-conscious readers to make sense of the medical findings we read or hear. Is red wine good or bad for you? What about dark chocolate? How bad is a little high cholesterol really? The answers seem to be changing all the time, and it can be frustrating to keep up with the pace. For better or for worse, this is how science works. We need to be open to the possibility that what we thought to be true is not, or that it is more nuanced.

If you think health has improved over the last 75 or 100 years, as I think, the main reason is that we used a process – peer review – that helped us sort out the generally helpful medical suggestions generally harmful suggestions. Bypassing this process with the rapid spread of poorly controlled science doesn’t help.

It is important to note that the main argument in support of the preprinted publication – that traditional journals cannot keep pace with rapidly evolving public health crises like COVID-19 – is not supported by the facts. . On January 21, 2020, the Seattle Times reported: “A man from Snohomish County has the first known case of the novel coronavirus in the United States. Three days later, a peer-reviewed scientific article on this topic appeared in the New England Journal of Medicine. The pace was just as fast 20 years ago, when the proven peer review approach helped us beat SARS and beat it quickly.

In my specialty, the editors of the four major journals have agreed not to publish articles that have been published on preprint servers because of the damage we associate with them. We were hoping that journals in more medical specialties would follow our lead, which we believe will help ensure patient safety. A few others go in this direction, or at least raise questions. I think journal editors owe their readers and the patients they serve at least that little protection.

But the most important fact is that the claims published on the preprint servers would not influence the behavior of the public at all and would not result in unnecessary damage or death if the newspaper journalists did not cover them as news. Generally speaking, they shouldn’t. When they do, they should clearly state that the “discovery” is preliminary and has not been peer reviewed. Equally important, they should start asking the tough questions about how we got here.


Comments are closed.