Hot Startup Theranos Has Struggled With Its Blood-Test TechnologyI came across the news by way of THCB in a post by Health 2.0 co-founder Matthew Holt. My comment there:
Silicon Valley lab, led by Elizabeth Holmes, is valued at $9 billion but isn’t using its technology for all the tests it offers
By JOHN CARREYROU
On Theranos Inc.’s website, company founder Elizabeth Holmes holds up a tiny vial to show how the startup’s “breakthrough advancements have made it possible to quickly process the full range of laboratory tests from a few drops of blood.”
The company offers more than 240 tests, ranging from cholesterol to cancer. It claims its technology can work with just a finger prick. Investors have poured more than $400 million into Theranos, valuing it at $9 billion and her majority stake at more than half that. The 31-year-old Ms. Holmes’s bold talk and black turtlenecks draw comparisons to Apple Inc. cofounder Steve Jobs.
But Theranos has struggled behind the scenes to turn the excitement over its technology into reality. At the end of 2014, the lab instrument developed as the linchpin of its strategy handled just a small fraction of the tests then sold to consumers, according to four former employees.
One former senior employee says Theranos was routinely using the device, named Edison after the prolific inventor, for only 15 tests in December 2014. Some employees were leery about the machine’s accuracy, according to the former employees and emails reviewed by The Wall Street Journal.
In a complaint to regulators, one Theranos employee accused the company of failing to report test results that raised questions about the precision of the Edison system. Such a failure could be a violation of federal rules for laboratories, the former employee said.
Theranos also hasn’t disclosed publicly that it does the vast majority of its tests with traditional machines bought from companies like Siemens AG...
Yeah, wow, what an article. Ouch. This kind of stuff is red meat to me. I learned my lab precision and accuracy (they’re not the same) QC basics in a forensic-level radioanalytic lab in Oak Ridge in the mid 80’s (lot of the same screening and confirmatory technologies and methods as in regular clinical bioassay).
I did my grad work on the scam that is the illicit drug testing industry, e.g.,
www.bgladd.com/drugwar/chapter3.htm
Lately I’ve been warning of possible inadequacies in QA within the burgeoning sub-forensic clinical-level “genomics” commercial assay industry, given that there are no federal quality regs in that area. (In Vegas we had this paternity testing company billboard by the freeway near downtown featuring a huge picture of an infant with a Photoshopped Pinocchio nose. Tagline: “Who’s The Daddy?” LOL.)
This Theranos thing is a whole ‘nuther concern. Far more basic, but equally important. Cheap, “innovative/disruptive” lab testing, well, bears close scrutiny.
From The Washington Post:
A comprehensive guide to Theranos’ troubles and what it means for youCEO Elizabeth Holmes at TedMed last year.
By Carolyn Y. Johnson and Ariana Eunjung Cha October 16
A deeply reported story by the Wall Street Journal has pulled the curtain back on Theranos, the hot and intensely secretive Palo Alto, Calif., company that aims to revolutionize how we get our blood tested. The newspaper's investigation raises questions about the accuracy and also the novelty of what the company is doing. Here's a guide to the growing questions about this company, the industry, and why it matters.
What is so innovative about what Theranos claims it's doing?
When you get your blood drawn at the doctor's office, it's often sent off to an external laboratory to be tested. Theranos captured the attention of busy consumers, doctors and investors when it announced it had developed a technology that could provide results for nearly 200 tests with only a single, capsule-sized amount of blood drawn by a finger stick rather than a needle in the arm. Moreover, the company claimed that it could do the testing faster (in less than four hours) and cheaper than traditional labs.
But the company's appeal has been as much about its founder's efforts to empower consumers as about Theranos technology. Elizabeth Holmes, Theranos' chief executive, is a Stanford dropout who is believed to be America's youngest self-made female billionaire. She has touted the potential benefits of being able to order blood tests on your own without going through your doctor. In 2013, the company announced a major partnership with Walgreens to offer its tests at wellness centers located at the retail centers. In April, Arizona passed a law allowing any lab test to be ordered without a physician's request. Theranos' tests are available to consumers in 42 Walgreens in Arizona and one in California.
What's at stake for the blood testing industry?
Blood diagnostic tests, ranging from a herpes test to cholesterol checks, are a huge industry, valued at about $75 billion in the United States. The market is projected to grow to $90 billion by 2018, according to Divyaa Ravishankar, a senior industry analyst at Frost & Sullivan.
The biggest piece of the business -- about two-thirds -- come from blood samples taken outside hospitals, according to Michael Cherny, a managing director at Evercore ISI. About half of that testing is done by independent labs -- a $25 billion, fast-growing market, which is dominated by two big companies, Quest Diagnostics Inc. and Laboratory Corporation of America (LabCorp)...
UPDATE: From the L.A. Times,
...In a widely viewed video of a talk she delivered last year at TEDMED, Holmes describes on-demand blood tests as an issue of personal empowerment. "When people have access to information about their own bodies," she said, "they can change outcomes."So, beyond quasi-DTC "overdiagnosis" concerns, several big potential summary issues: [1] it looks like they've been doing most of their specimen runs on industry-standard assay equipment rather than their "proprietary" "Edison" machines they tout as the "innovative/disruptive" cost-cutting game-changer.
That sounds unexceptionable, even laudable, but healthcare is more complicated. As John P.A. Ioannidis of Stanford Medical School observed for an article in the Journal of the American Medical Assn., Holmes' talk ignored the drawbacks to expanded consumer-driven blood testing, such as "overdiagnosis, false-positive findings, or the potential for ... misplaced and perhaps overly zealous diagnostic and screening efforts."
As it happens, questions had been raised about Theranos' claims for months — but they had been swamped by a tide of fawning publicity. Ioannidis noticed that information about Theranos had appeared in the Wall Street Journal, Business Insider, Fortune and Forbes, "but not in the peer-reviewed biomedical literature."
In many articles, the company's choice to develop its technology secretly, as "stealth research," was treated as a virtue. But to Ioannidis, it presented a risk: "Stealth research creates total ambiguity about what evidence can be trusted in a mix of possibly brilliant ideas, aggressive corporate announcements, and mass media hype."...
[2] More importantly, serious questions pertaining to accuracy and precision have arisen.
You get a set of labs back, you need to be able to assume they're as accurate as can be. Your harried docs will simply take the numbers at face value and advise you accordingly. Bad labs get you bad advice.
[3] Most potentially damaging of all, whistleblower allegations have come forth asserting that Theranos gamed their accreditation proficiency specimens ("PE samples"). The WSJ article:
Whether labs buy their testing instruments or develop them internally, all are required to prove to the federal Centers for Medicare and Medicaid Services that they can produce accurate results. The process is known as proficiency testing and is administered by accredited organizations that send samples to labs several times a year.Theranos lawyers are now circling the wagons. I bet there are now "Crisis Management Consultants" on board as well. People are choosing their words ever-so-carefully, while also pointing fingers at the big incumbent lab players.
Labs must test those samples and report back the results, which aren’t disclosed to the public. If a lab’s results are close to the average of those in a peer group, the lab receives a passing grade.
In early 2014, Theranos split some of the proficiency-testing samples it got into two pieces, according to internal emails reviewed by the Journal. One was tested with Edison machines and the other with instruments from other companies.
The two types of equipment gave different results when testing for vitamin D, two thyroid hormones and prostate cancer. The gap suggested to some employees that the Edison results were off, according to the internal emails and people familiar with the findings.
Senior lab employees showed both sets of results to Sunny Balwani, Theranos’s president and chief operating officer. In an email, one employee said he had read “through the regulations more finely” and asked which results should be reported back to the test administrators and government.
Mr. Balwani replied the next day, copying in Ms. Holmes. “I am extremely irritated and frustrated by folks with no legal background taking legal positions and interpretations on these matters,” he wrote. “This must stop.”
He added that the “samples should have never run on Edisons to begin with.”
Former employees say Mr. Balwani ordered lab personnel to stop using Edison machines on any of the proficiency-testing samples and report only the results from instruments bought from other companies...
“Stories like this come along when you threaten to change things, seeded by entrenched interests that will do anything to prevent change” - TheranosI guess we'll see. While that is undoubtedly true (corporate espionage and the gamut of competitor skulduggery), particularly in inadequately regulated markets (think no further than the Gresham's Dynamic poster child, Uber), spare us the poignant Shocked Surprise and Dismay. Hel-lo? SWOT analysis/Risk Management 101, anybody?
More. From TechInsider:
Here's exactly what we know about how Theranos' 'revolutionary' technology worksRead the entire TechInsider piece.
Kevin Loria
On Oct. 15, a bombshell investigative report by the Wall Street Journal alleged that hot startup Theranos, founded by Elizabeth Holmes, was mostly relying on traditional technology to analyze blood test results.
Later, on CNBC's "Mad Money," Holmes said that as part of a voluntary transition to getting its tests approved by the Food and Drug Administration, the company stopped using its proprietary "nanotainers" to collect fingerpricks of blood — though some reports indicate that pressure from the FDA was what fueled that decision.
Many of these reports have observers wondering whether Theranos' supposedly revolutionary blood tests work at all.
Until either Theranos shows the data that Holmes has said disproves allegations from former employees that its tests don't work as well — we've requested that data multiple times — or until the FDA releases their decisions on the other approximately 130 tests the company has submitted for review, we won't know the answer to that question.
What we can do is look at existing science and expert opinion to try to make an educated guess about how these blood tests are supposed to work.
This past April, Tech Insider talked to experts in the fields of clinical pathology and laboratory medicine, microfluidics, and biomedical engineering to try to piece together an answer.
No one could figure it out conclusively, because as Dr. David Koch, president of the American Association for Clinical Chemistry and a professor at Emory University, said then, "It may be wonderful [or] it may bomb, but I really can't be more definitive because there's nothing to really look at, to read, to react to."...
More. From Wired...
Theranos’ Scandal Exposes the Problem With Tech’s Hype Cycle
THERANOS, A COMPANY that makes low-cost blood tests, has been celebrated as the inventor of the next major medical breakthrough, and its founder, Elizabeth Holmes, has been hailed as the next Steve Jobs. But in an expose published today, The Wall Street Journal portrays Theranos as a company that is not only grossly under-delivering on its promise, but is also working hard to hide its problems.
A months-long investigation by the Journal found that, for the bulk of its blood tests, Theranos has not actually been using its own technology, which the company has claimed are capable of running tests with only a few drops of blood. Instead, it’s been relying on traditional machines from companies like Siemens. What’s more, the article alleges that in proficiency tests, Theranos’s own Edison machines have produced radically different results than traditional machines have, and internal emails reveal the company may have tried to prevent those inconsistencies from seeing the light of day.
The Journal‘s claims, many of which Theranos has contested, are damning, portraying Theranos as an Oz-like company that looms large in public consciousness but turns out to have little going on behind the facade. At the same time, the story also exposes a deeper problem with the way Silicon Valley tries to spin hype into startup gold.
It’s hardly unusual for a startup to over-hype its potential in its early days. Mark Zuckerberg was crowing about connecting every person in the world long before that claim seemed plausible. Uber executives have been talking about taking over the taxi industry for years. Even Tinder sometimes bills itself as a global social network, not just a dating app. These grandiose promises are table stakes in the tech industry, a requirement for luring both investors and early users. And typically, they’re harmless. After all, who really gets hurt when, say, Ello fails to catch on?
But Theranos is a cautionary tale of what happens when that mentality creeps into sectors other than software, such as medicine. As one Theranos patient quoted by the Journal put it, “trial and error on people, that’s not OK.” It’s not that any of this is done maliciously. It’s just that the tech industry wants its heroes. It wants to find the next Steve Jobs. It wants to be responsible for discovering not just the next big chat app, but the next big idea that will change the world. Once the industry finds a protagonist who seems to fit that narrative, the story can take on a life of its own, separate from the more mundane reality...
The latter two paragraphs, precisely. I harbor no ill will toward this young woman (unlike, say, the arrogant cutthroat Enron 2.0 types at Uber). Maybe she's just gotten played by people who work for her. Silicon Valley sharks, after all, are not exclusive to the Pacific waters just to the west of Sand Hill Road. I've cited her previously a couple of times. See, e.g., my January post "Failure is the Siamese twin of innovation" - Jonathan Bush.
From CBS News:
"The sources relied on in the article today were never in a position to understand Theranos' technology and know nothing about the processes currently employed by the company ... We are disappointed that, in an effort to make its story more dramatic, this reporter relied only on the views of four 'anonymous' disgruntled former employees, competitors and their allies, instead of reaching out to many of the scientific, health care and business leaders who have actually seen, tested, used and examined our breakthrough technologies."Ahhh... the tried and true "disgruntled former employees."
Fair enough. Let's trot these other people out, CVs and Theranos ties in tow for all to see and evaluate at an extended news conference replete with lengthy Q&A.
UPDATE
From TechTimes:
...Due to the lack of scientific transparency and suspicion over the voluntarily submitted data to win approval of their methods, the FDA made an unannounced inspection of the Theranos lab. The FDA later announced that the Nanotainers used to contain the finger-pricked blood sample is "an unapproved medical device."This will certainly be interesting to follow. Wafts of VW? Questions of relative "innovativeness" and concomitant marketing puffery aside, if they in fact gamed their PE samples and she knew it, well, she and her company have a very, very big problem.
Moreover, Theranos uses standard lab equipment purchased from external vendors to do their tests, and not their own Edison machines. Under pressure from government regulators, the tech startup has ceased operations using the proprietary technology to collect tiny blood samples, with the exception of the herpes simplex 1 test, which was FDA-approved in July.
NEWS UPDATE
Surprise FDA inspection at one of Silicon Valley’s hottest startups Theranos
Theranos is stopping its revolutionary blood work technique after being blindsided by the inspection.The big IPO Payday is starting to look a little iffy. Perhaps.
Federal officials showed up unannounced to the facilities of super-hot startup laboratory Theranos, slapping its revolutionary pinprick blood vials as “unapproved medical devices.”
Now, Theranos is halting work on its brand new blood test technique, which involves using tiny vials to collect droplets of blood through a pinprick rather than large vials of blood from a traditional needle in vein procedure, according to a Wall Street Journal report...
From The New Yorker:
The Secrets of a Billionaire’s Blood-Testing StartupThe hits just a-keep on comin'...
BY ERIC LACH
As far as corporate origin stories go, the blood-testing company Theranos has a pretty good one. Its founder, Elizabeth Holmes, quit Stanford in 2004, as a sophomore, to dedicate herself to the company. Over the next decade, Theranos attracted hundreds of millions of dollars in investments. Former senior government officials, including Henry Kissinger and George P. Shultz, joined its board of directors. A partnership with Walgreens envisaged Theranos “wellness centers” in thousands of stores across the country. As the company’s valuation soared, Holmes took to wearing Steve Jobsian black turtlenecks, and was proclaimed “the youngest self-made female billionaire in the world” by Forbes. All of this rested on the idea that the company could revolutionize the seventy-five-billion-dollar diagnostic-lab business, by developing new kinds of tests that were quicker, cheaper, and required much less blood than their predecessors.
On Thursday, the Wall Street Journal presented a different story. Citing four anonymous former company employees, among other sources, the Journal’s John Carreyrou reported that Theranos has “struggled behind the scenes to turn the excitement over its technology into reality.” Rather than a tale of technological and commercial triumph, Carreyrou told one of wonky equipment, skeptical employees, legal threats, and evolving standards of “marketing accuracy.” According to the Journal, as of late 2014, Theranos was running less than ten per cent of its tests on its own machines. The rest of the tests were being run on old-fashioned machines, purchased from old-fashioned manufacturers like Siemens AG. In a bizarrely clumsy move for a multibillion-dollar company, Theranos deleted several claims from its Web site while Carreyrou was reporting his article, including a sentence that read, “Many of our tests require only a few drops of blood.”...
___
More to come...
Great article, especially the title, modern PT Barnum, so fitting.
ReplyDelete