@Theranos: The Empire Strikes Back, sort of.

From the Theranos website:

"In addition to its work as the first lab voluntarily submitting all its Laboratory Developed Tests to FDA, Theranos undergoes continuous proficiency testing on blinded samples from the College of American Pathologists (CAP), American Proficiency Institute (API), and New York State Department of Health. To date in 2015, Theranos Proficiency Testing met or surpassed performance goals 99.5% of the time for CAP, API, and 100% of the time for New York State across 414 assays. Theranos is the first lab to begin publishing this data along with regular updates to it."

They also display a long set of X-Y QC scatterplots for selected analyte parameters, from which I selected six FYI.

"Representative Clinical Correlations - here's how our tests compare to reference methods."

Pretty much uniformly hugging the "reference" diagonal, with high 90's "R-squared" indicators. "R-Squared" is also known as the "coefficient of determination," the square of the Pearson-R Correlation Coefficient, a statistical measure of the extent of Y-axis variability "explained" by X-axis variability (all of which assume normal/bell curve data distributions). An R-square of 1.0 means you hit the reference bulls-eye every time. "Accuracy," recall, is hitting the "bulls-eye." "Precision" refers to how often you can get close results (even if they're inaccurate) in repeated trials.

"We realize our mission only when our tests are performed to the highest standards of quality. The performance of our tests is routinely demonstrated through multiple accredited proficiency testing programs. And despite not being required to do so, we are the first lab that has been and will continue to submit all our Laboratory Developed Tests to the more rigorous standards of the FDA."

The foregoing still beg a lot of questions. e.g., from Wired:

OK, Theranos: Here’s [sic] the Data the World Needs to See
Nick Stockton

I reached out to Theranos, asking them if they’ll be releasing their primary data. Their response: “All of the information we have released at this time is available online; if we release more, we will be sure to let you know.” Since I’m not holding my breath anymore, let’s talk about data.

Data, data, data. If you’ve been following Theranos long enough, you’ll recognize this refrain: a call for transparency so the medical community can verify that Theranos’ technology does what the company claims. When Theranos deigns to respond, it usually says something along the lines of “We have data! We’ve shown it to the FDA! We post it on our website!”

All technically true, but the FDA data shown in that paper are aggregate results, not primary. What does [sic] primary data look like? What does it do? How would you know it when you saw it?...

I have my own. For one thing, each of the graphically depicted Theranos data plot points represents a requisite accompanying "QA package," i.e., the full documentation (paper and electronic) comprising things like end-to-end chain of custody signoffs, as well as records pertaining to instrument calibrations, chemical reagents, standards solutions, and supplies consumed during the specimen runs. Independent auditors would be requesting these at the outset.

Back during my 80's lab days, a scandal erupted when one of our competitor EPA-certified environmental labs was found to have been fabricating results for more than a year, just pouring the samples down the drain and into the incinerators, writing up phony results reports, and simply sending invoices to the clients. They eventually got caught when their reagent/supplies inventory records were examined by auditors and found to be impossible for the volume of samples supposedly run. These dopes thought they were gonna save some money. Instead, they got fined and disbarred.

Another interesting article, from San Francisco Business Times: "Theranos' week offers transparency lesson for 'disruptors' 

...The main question, of course, is whether Theranos' methods of divining test results actually are as cutting edge as the company has claimed. We will discover that in the coming months, but the company has acknowledged that it is running only one test with its finger-prick method while using traditional testing methods for the 200-plus other tests it runs.

Granted, these tests are offered at a cheaper price point than that set by LabCorp or Quest. But that's not the story anymore. The story is about Theranos' finely crafted, even manipulated, message and its secrecy, even to the point of not publishing its test results in a peer-reviewed scientific journal.

That secrecy has come back to stick Theranos like a bad needle, whether or not its tests are indeed as accurate as traditional lab tests...

___

More to come...