Flawed ivermectin preprint highlights challenges of COVID drug studies

Throughout the pandemic, the anti-parasite drug ivermectin has attracted much attention, particularly in Latin America, as a potential way to treat COVID-19. But scientists say that recent, shocking revelations of widespread flaws in the data of a preprint study reporting that the medication greatly reduces COVID-19 deaths dampens ivermectin’s promise — and highlights the challenges of investigating drug efficacy during a pandemic.

“I was shocked, as everyone in the scientific community probably were,” says Eduardo López-Medina, a paediatrician at the Centre for the Study of Paediatric Infections in Cali, Colombia, who was not involved with the study and who has investigated whether ivermectin can improve COVID-19 symptoms. “It was one of the first papers that led everyone to get into the idea ivermectin worked” in a clinical-trial setting, he adds.

The paper summarized the results of a clinical trial seeming to show that ivermectin can reduce COVID-19 death rates by more than 90%¹ — among the largest studies of the drug’s ability to treat COVID-19 to date. But on 14 July, after internet sleuths raised concerns about plagiarism and data manipulation, the preprint server Research Square withdrew the paper because of “ethical concerns”.

Ahmed Elgazzar at Benha University in Egypt, who is one of the authors on the paper, told Nature he was not given a chance to defend his work before it was removed.

Early in the pandemic, scientists showed that ivermectin could inhibit the coronavirus SARS-CoV-2 in cells in laboratory studies². But data on ivermectin’s efficacy against COVID-19 in people are still scarce, and study conclusions conflict greatly, making the withdrawal of a major trial particularly noteworthy.

Although the World Health Organization advises against taking ivermectin as a COVID-19 treatment outside clinical trials, the over-the-counter drug has become popular in some regions of the world. Some view it as a stopgap until vaccines become available in their areas, even though it has not yet been proven effective; scientists worry that it will also be seen as an alternative to vaccines, which are highly effective.

Ripple effects

The paper’s irregularities came to light when Jack Lawrence, a master’s student at the University of London, was reading it for a class assignment and noticed that some phrases were identical to those in other published work. When he contacted researchers who specialize in detecting fraud in scientific publications, the group found other causes for concern, including dozens of patient records that seemed to be duplicates, inconsistencies between the raw data and the information in the paper, patients whose records indicate they died before the study’s start date, and numbers that seemed to be too consistent to have occurred by chance.

In an editorial note, Research Square said that it has launched a formal investigation into the concerns raised by Lawrence and his colleagues. According to the Egyptian newspaper Al-Shorouk, Egypt’s minister of higher education and scientific research is also examining the allegations.

The paper was “withdrawn from the Research Square platform without informing or asking me”, Elgazzar wrote in an e-mail to Nature. He defended the paper, and said of the plagiarism allegations that “often phrases or sentences are commonly used and referenced” when researchers read one another’s papers.

Although dozens of ivermectin clinical trials have been launched over the past year³, the Elgazzar paper was notable for announcing one of the first positive results, as well as for its size — it included 400 people with symptoms of COVID-19 — and the magnitude of the drug’s effect. Few therapies can claim such an impressive reduction in death rates. “It was a significant difference, and that stood out,” says Andrew Hill, who studies repurposed drugs at the University of Liverpool, UK. “It should have raised red flags even then.”

Lawrence agrees. “I was absolutely shocked that no one had uncovered it,” he says.

Before its withdrawal, the paper was viewed more than 150,000 times, cited more than 30 times and included in a number of meta-analyses that collect trial findings into a single, statistically weighted result. In one recent meta-analysis in the American Journal of Therapeutics that found ivermectin greatly reduced COVID-19 deaths⁴, the Elgazzar paper accounted for 15.5% of the effect.

One of the authors of the meta-analysis, statistician Andrew Bryant at Newcastle University, UK, says that his team corresponded with Elgazzar before publishing the work to clarify some data. “We had no reason to doubt the integrity of [Professor] Elgazzar,” he said in an e-mail. He added that in a pandemic setting, no one can reanalyse all of the raw data from patient records when writing a review. Bryant went on to say that his group will revise the conclusion if investigations find the study to be unreliable. However, even if the study is removed, the meta-analysis would still show that ivermectin causes a major reduction in deaths from COVID-19, he says.

Reliable data needed

The paper’s withdrawal is not the first scandal to dog studies of ivermectin and COVID-19. Hill thinks many of the other ivermectin trial papers that he has scanned are likely to be flawed or statistically biased. Many rely on small sample sizes or were not randomized or well controlled, he says. And in 2020, an observational study of the drug was withdrawn after scientists raised concerns about it and a few other papers using data by the company Surgisphere that investigated a range of repurposed drugs against COVID-19. “We’ve seen a pattern of people releasing information that’s not reliable,” says Hill. “It’s hard enough to do work on COVID and treatment without people distorting databases.”

Carlos Chaccour, a global-health researcher at the Barcelona Institute for Global Health in Spain, says it has been difficult to conduct rigorous studies on ivermectin. That’s partly because funders and academics in wealthy countries haven’t supported them, and, he suspects, have often dismissed trials of ivermectin because most of them have been done in lower-income countries. Furthermore, says Rodrigo Zoni, a cardiologist at the Corrientes Cardiology Institute in Argentina, it is difficult to recruit participants because many people — particularly in Latin America — are already taking the widely available drug in an attempt to prevent COVID-19.

Adding to the difficulty are conspiracy theories holding that ivermectin has been proven to work and that drug companies are depriving the public of a cheap cure. Chaccour says he has been called ‘genocidal’ for doing research on the drug rather than just endorsing it.

Although the jury is still out on ivermectin, many say the retraction speaks to the difficulty of assessing research during a pandemic. “I personally have lost all faith in the results of [ivermectin] trials published to date,” says Gideon Meyerowitz-Katz, an epidemiologist at the University of Wollongong in Australia who helped Lawrence to analyse the Elgazzar paper. It’s not yet possible to assess whether ivermectin works against COVID-19 because the data currently available are not of sufficiently high quality, he says, adding that he is reading other ivermectin papers in his spare time, looking for signs of fraud or other problems.

Chaccour and others studying ivermectin say that proof of whether the drug is effective against COVID-19 rests on a handful of large, ongoing studies, including a trial in Brazil with more than 3,500 participants. By the end of 2021, says Zoni, around 33,000 people will have participated in some kind of ivermectin trial.

“I think it is our duty to exhaust all potential benefits,” says Chaccour, especially given that most countries still do not have widespread access to vaccines. “Ultimately if you do a trial and it fails, fine, but at least we tried.”