Are more Theranos -style scandals looming for investors in healthcare startups?
A team of researchers associated with the Meta Research Innovation Center at Stanford thinks so. They’ve published a paper warning investors in life sciences startups that a systemic lack of transparency exists in their portfolio companies — creating the possibility for more multi-billion dollar implosions and scandals like the one that toppled Theranos and its charismatic founder, Elizabeth Holmes.
Indeed, one of the study’s authors, Dr. John Ioannidis, the co-director of the Meta-Research Innovation Center at Stanford and director of the University’s PhD program in Epidemiology and Clinical Research, was among the first people to identify the risks associated with Theranos and its “stealth research”.
Now Dr. Ioannidis and his co-authors, Ioana A. Cristea and Eli M. Cahan have published a study surveying the publicly available research from the largest privately held companies in the healthcare space, and found them lacking.
Most of the highest valued startups in healthcare have not published any significant scientific literature, the study found. Nearly half of the publications from companies worth over $1 billion came from only two startups — 23andMe and Adaptive Biotechnologies, according to the paper.
“Many years ago I was the first person to say that Theranos had a problem,” says Ioannidis. “The problem that I had then was that Theranos did not have any peer-reviewed evidence to show.”
In an interview and in their paper, Ioannidis and Cahan warn that investors have overlooked systemic problems created by the lack of transparency among healthcare startups by
They write:
“It would be tempting to dismiss the Theranos case as just one rotten apple. However, we worry that the focus on fraud puts aside a more fundamental concern. Fraud is making waves in the news, but stealth research may have a more detrimental impact.”
According to the study’s findings, more than half of the healthcare startups that are worth more than $1 billion have published no highly cited papers at all. For companies that were acquired or are publicly traded that number is around 40%.
In all, healthcare startups that are currently valued at over $1 billion published 425 Pubmed papers. And of those papers only 34 (8%, including 2 reviews) were highly cited. For companies with valuations of over $1 billion who had been acquired or are publicly traded on stock exchanges, the researchers counted 413 papers, of which 47 (11%, including 9 reviews) were highly-cited.
Digging deeper into some of the companies which had high valuations but little or no published research revealed scores of operational and technological issues for the researchers.
For instance, StemCentrx, which was bought for $10.2 billion in 2016 by AbbVie, had published 16 papers — and only one highly-cited paper. Since the acquisition, the Food and Drug Administration had imposed a delay on the readout of the company’s phase II trial for its Rova T targeted antibody drug for cancer treatment. In December a Phase III trial for Rova T as a second-line treatment for patients with advanced small cell lung cancer was halted, because the treatment wasn’t working, according to a report in Targeted Oncology.
Acerta Pharma, another healthcare focused startup focused on cancer treatments was bought by AstraZeneca for $7.3 billion. That company published nine articles and had one highly-cited paper for a very early study of a potential treatment for relapsed chronic lymphocytic leukemia. Acerta received accelerated approval for a drug called acalabrutinib, which treats a rare form of lymphoma called mantle cell lymphoma. Two years ago, AstraZeneca had to retract data and admit that Acerta falsified preclinical data for its drug.
Then there’s Intarcia, the developer of a device for diabetes treatment that’s worth $5.5 billion. That company had its device rejected by the FDA and was forced to lay off staff and halt a couple of later stage trials. It had only published six papers — none of them very highly cited.
Ultimately, the researchers concluded that highly valued healthcare startups don’t contribute to published research and that the valuation of these companies by investors is divorced from any externally validated data.
For the researchers (and for investors) this should presents a problem.
“Many unicorns may be overvalued [21] and subject to unrealistic scientific expectations,” the study’s authors write. And they reject the argument that simply applying for — and receiving — patents is enough to prove that a technology in the healthcare space has been thoroughly vetted. “[Patents] do not offer the same level of documentation as peer-reviewed articles. For example, Theranos had over 100 patents [1], but these were unable to supplant the vacuum in their evidence,” the researchers wrote.
Even if companies want to protect their technology, there are still ways for them to be more transparent about the results or benefits of their technology. The authors acknowledge that publishing isn’t the primary mission of startups. They can, however publish a few high-value articles, secure their technology through patents and then work with researchers, universities or hospitals to validate the technology and have those organizations publish results of the tests, the authors argue.
As the authors conclude:
Start-ups are key purveyors of innovation and disruption. Consequently, holding them to a minimal standard of evaluation from the scientific community is crucial. Participation in peer review, with all its limitations, is the best way we have to uphold this standard. We are not arguing that start-ups should divert excessive resources to having peer-reviewed papers. However, when their products are destined to affect patient health, they should neither be solely doing marketing. Confidential data sharing with potential investors or regulators cannot replace more open scrutiny by the scientific community.
