Novel Coronavirus pandemic screening assays: Laboratory QC issues.

APRIL 21ST UPDATE: Criminally negligent, lethal lab QA failure at the CDC.

____ 

APRIL 1ST UPDATE: I typically post 2-3 times a week, and Lord knows there are plenty of topics of interest and importance to move on to, but this Covid-19 pandemic issue is blotting out the sun these days. Per this "testing" topic below, there is now worrisome "crisis response theater" political pressure to waive FDA certification / regulatory standards to "get more test kits out there quickly." Bad idea. Accuracy matters.

I will be looking into your and your competitors' lab QA performance (e.g., @LABCORP). Do you publish your #coronavirus RT-PCR screening assays false positive/negative rates? https://t.co/H7zcyAWRuh

— Bobby Gladd (@BobbyGvegas) March 16, 2020

Yeah, right. I'm sure I'll get responses. Of course.

I first alluded to my technical concerns here. Accruing Hopkins real-time pandemic data here.

CLINICAL ASSAYS

You pee in a cup. You have a blood draw. You submit to having your nose or throat swabbed. You spit up some sputum into s vial, provide a stool sample. You undergo a biopsy, provide a hair sample, or blow into the breathalyzer. These specimens are then "tested" for their possible qualitative or quantitative "results" of pressing diagnostic (and subsequent treatment options) import.

If you assume the findings to be uniformly "accurate," you would be wrong. Not exactly news, yet repeatedly glossed over amid the throes of the crisis du jour.

The upshots of (always present) lab error risks have varying consequences. Some of them potentially quite adverse.

Note: should you accurately test "negative" for this virus, the result only holds for the date and time the specimen was collected. You could become infected with the pathogen at any time thereafter and subsequently be a "positive."

SOME DISCUSSION KEYWORDS OF INTEREST

Prevalence, Incidence, R0 ("R-naught"), Etiology, Sensitivity, Specificity, PPV, NPV, Bayesian Priors, Accuracy, Precision, False Positives, False Negatives, Cross-Reactivity, Coefficient of Variation (CV), Spikes, Blanks, Replicates, Matrices, Blinds, Reference Standards, Reagents, Chain of Custody, Cross-Contamination, Screening Test, Confirmation Test, Aliquot, Qualitative, Quantitative, Hockey Stick Curve, de minimus, LLD, RT-PCR (Real Time Polymerase Chain Reaction), CLIA...

That's enough for openers. Again, see my prior remarks.

THE RT-PCR ASSAY (now used to process coronavirus swab "test kits")

Pay attention to the workflow and technology used. This is not a litmus paper test. It's a DNA "amplification" assay (specifically RNA). It requires time, adequately trained lab workers, and QA-validated equipment and supplies.

UPDATE: from Scientific American on COVID19 testing.

Per The Daily Beast:

On New Year’s Eve, Chinese officials notified the World Health Organization that a new type of viral pneumonia was circulating in the city of Wuhan. Less than two weeks later, virologists published the entire genetic sequence of a new type of coronavirus from the same family of viruses that caused the SARS and MERS outbreaks.

That blueprint, or genome, provided an effective means of identifying the infectious agent, now officially called SARS-2-CoV, or the 2019 novel coronavirus. Within two weeks of the release of that critical information, another team, led by researchers in Berlin, published a diagnostic method. The test offered one means by which labs could collect throat or nasal swabs and screen for new cases of COVID-19, as the disease caused by the virus is called, based on isolating and amplifying a genetic signature specific to it.

The race was on as the WHO adopted the German-developed tests and distributed it to dozens of countries. But China, the United States, and several other countries developed their own ways to screen for a modern plague that has since infected over 200,000 people worldwide, including 6,500 people in all 50 states, causing at least 107 U.S. deaths...

So, right off the bat you're gonna need new "genetic signature" "reference standards" comprised of known quantities of the novel pathogen. Without these "controls" you have no way to determine the relative precision and accuracy of patient specimen outcomes (and the concomitant false positive and false negative error rates).

Again:

No clinical test is immune from error (particularly qualitative yes/no screening measures--especially newly rush-developed and deployed assays). And, error rates vary from worker to worker and lab to lab. Vigilant, consistently documented lab QC is neither easy nor cheap (pdf). Without it, you're flying blind.

Some other considerations: the quality (incl expiration dates) of the "reagent" chemicals used in specimen processing; the training and experience levels of the line lab staff; calibration of the analytical instruments, the specimen workload (sample overload raises a number of exigent risks, including workflow corner-cutting and fatigue errors, and elevated likelihood of cross-contamination).

I repeat:

WHAT DOES THE CDC SAY ABOUT THIS NEW COVID-19 TESTING?

Not much. Certainly not anything quantitatively specific pertaining to QC.

CDC has developed a new laboratory test kit for use in testing patient specimens for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes COVID-19. The test kit is called the “Centers for Disease Control and Prevention (CDC) 2019-Novel Coronavirus (2019-nCoV) Real-Time Reverse Transcriptase (RT)-PCR Diagnostic Panel.” It is intended for use with the Applied Biosystems 7500 Fast DX Real-Time PCR Instrument with SDS 1.4 software. This test is intended for use with upper and lower respiratory specimens collected from persons who meet CDC criteria for COVID-19 testing. CDC’s test kit is intended for use by laboratories designated by CDC as qualified, and in the United States, certified under the Clinical Laboratory Improvement Amendments (CLIA) to perform high complexity tests.

On Monday, February 3, 2020, CDC submitted an Emergency Use Authorization (EUA) package to the U.S. Food and Drug Administration (FDA) in order to expedite FDA permitted use of the CDC diagnostic panel in the United States. The EUA process enables FDA to consider and authorize the use of unapproved, but potentially life-saving medical or diagnostic products during a public health emergency. The U.S. Secretary of Health and Human Services declared the SARS-CoV-2 virus a U.S. public health emergency on Friday, January 31, 2020. FDA issued the EUA on February 4, 2020. IRR began distribution of the test kits to states, but shortly thereafter performance issues were identified related to a problem in the manufacturing of one of the reagents which led to laboratories not being able to verify the test performance. CDC is remanufacturing the reagents with more robust quality control measures. New tests will be distributed once this issue has been addressed. CDC continues to perform initial and confirmatory testing.

I'd be wanting particulars addressing "robust quality control measures." Not that I'm likely to get much in response.

Below, different surveillance topic, same concerns.

If you remain asymptomatic (and don't meet CDC pre-screening criteria such as traced-contact or high prevalence area recent travel), you don't want to be screened for COVID-19. Think about the upshot of being a false positive. You could well be forcibly quarantined away from home on your own dime.

But, to be fair, I am by no means oblivious to plausible pushback, e.g., what if there is a significant sub-stratum of those who never evince clinical symptoms yet are nonetheless truly infected and (even if transiently) contagious: i.e., "vectors," asymptomatic "carriers?" How will we reliably estimate that prevalence?

One last observation here for now. There are thousands of other types of routine clinical lab tests in process every day, many of them of pressing importance, and all in need of uniform QC for effective dx accuracy and precision.

UPDATE: MORE TERMS

"R-naught"

Degrees of contagion exponentiality. For a different context see my 2006 post "Each one reach two."

Erratum: Psychiatric disease vectors.

UPDATE

This Is How We Can Beat the Coronavirus
Mitigation can buy us time, but only suppression can get us to where we need to be.

While many watched the coronavirus spread across the globe with disinterest for months, in the last week, most of us have finally realized it will disrupt our way of life. A recent analysis from Imperial College is now making some Americans, including many experts, panic. The report projects that 2.2 million people could die in the United States. But the analysis also provides reason for hope—suggesting a path forward to avoid the worst outcomes…

Very interesting article by two nationally esteemed clinician / scholars. Read all of it. They recommend mass screenings, for mandatory "suppression." My QA/QC concerns remain. I'm open to rebuttal.

MARCH 21 UPDATE

Serious long-read from The Atlantic.

The in-depth case for mass screening.

Some relevant words curiously not to be found in the article: Sensitivity, Specificity, Prevalence, QC, Quality Control, PPV, NPV, Error, Accuracy, Precision.

100% (or, "good enough, albeit unknown") testing reliability assumed, in light of the exigency.

ISSUES OF FRAUD

Coronavirus (COVID-19) Update: FDA Alerts Consumers About Unauthorized Fraudulent COVID-19 Test Kits

Yeah, that direct-to-consumer BS stuff is so predictable.

Much less well-known to the public is what we call Dry Labbing.

During my lab QC days in Oak Ridge in the 1980s, one of our commercial competitor facilities finally got busted after a year of "dry labbing"--discarding incoming EPA specimens, simply faking and reporting "results," and sending the invoices.  

More recently, anyone recall Theranos?

UPDATE

An excellent New Yorker article:

Why Widespread Coronavirus Testing Isn’t Coming Anytime Soon
By Robert P. Baird
March 24, 2020

…The clinical lab director expressed concern that granting regulatory authority to the states means that “we are now in the Wild West of laboratory regulation. It’s really a let-the-buyer-beware world. Essentially, apart from the F.D.A.’s E.U.A. process, there is very limited regulation of the quality, accuracy, and specificity of diagnostic tests for covid-19, and I think that’s a dangerous situation.” Bartkus, in Minnesota, said, “I will tell you, there is pressure to get these tests out: from the public, from the laboratories, from the politicians. It is a challenge to do that in a scientific and equitable way when you have no expertise in authorizing other labs to do testing.”

Read all of it. Goes beyond the scope of my concerns here, but, yeah.

MORE GREAT JOURNALISM

Everyone should read this comprehensive piece by Ed Yong:

How the Pandemic Will End
The U.S. may end up with the worst COVID-19 outbreak in the industrialized world. This is how it’s going to play out.

…the second pressing need: a massive rollout of COVID-19 tests. Those tests have been slow to arrive because of five separate shortages: of masks to protect people administering the tests; of nasopharyngeal swabs for collecting viral samples; of extraction kits for pulling the virus’s genetic material out of the samples; of chemical reagents that are part of those kits; and of trained people who can give the tests. Many of these shortages are, again, due to strained supply chains. The U.S. relies on three manufacturers for extraction reagents, providing redundancy in case any of them fails—but all of them failed in the face of unprecedented global demand…

Just a little snip bearing on my part of the "testing" topic.

UPDATE

Epidemic population screening test issues are just the tip of the iceberg. Click the title. Read all of it.

...The word “epidemiology” is derived from “epi” and “demos”—“above the people.” It is the science of aggregation, the science of the many. Yet it works most effectively when it moves in step with medicine, the science of the one. On the morning I visited the Shitala shrine in Kolkata, the goddess of bygone population-decimating epidemics was also serving as the personal goddess of a mother who had brought a child with a weeklong fever. To win the Kampf against covid-19, it’s essential to trace the course of the virus as it moves through populations. But it’s equally essential to measure its course within a single patient. The one becomes the many. Count both; both count.

 Siddhartha ROCKS!

AND, THERE'S MORE

From Bats to Human Lungs, the Evolution of a Coronavirus

For thousands of years, a parasite with no name lived happily among horseshoe bats in southern China. The bats had evolved to the point that they did not notice; they went about their nightly flights unbothered. One day, the parasite—an ancestor of the coronavirus, sars-CoV-2—had an opportunity to expand its realm. Perhaps it was a pangolin, the scaly anteater, an endangered species that is a victim of incessant wildlife trafficking and sold, often secretly, in live-animal markets throughout Southeast Asia and China. Or not. The genetic pathway remains unclear. But to survive in a new species, whatever it was, the virus had to mutate dramatically. It might even have taken a segment of a different coronavirus strain that already inhabited its new host, and morphed into a hybrid—a better, stronger version of itself, a pathogenic Everyman capable of thriving in diverse species. More recently, the coronavirus found a new species: ours. Perhaps a weary traveller rubbed his eyes, or scratched his nose, or was anxiously, unconsciously, biting his fingernails. One tiny, invisible blob of virus. One human face. And here we are, battling a global pandemic.

The world’s confirmed cases (those with a positive lab test for covid-19, the disease caused by sars-CoV-2) doubled in seven days, from nearly two hundred and thirteen thousand, on March 19th, to four hundred and sixty-seven thousand, on March 26th. Nearly twenty-one thousand people have died. The United States now has more confirmed cases than any country on earth, with more than eighty thousand on March 26th. These numbers are a fraction of the real, unknown total in this country and around the world, and the numbers will keep going up. Scientists behind a new study, published earlier this month in the journal Science, have found that for every confirmed case there are likely five to ten more people in the community with an undetected infection. This will likely remain the case. “The testing is not near adequate,” one of the study’s authors, Jeffrey Shaman, an environmental-health sciences professor at Columbia University, said…

ANOTHER, WAPO:

Three months into the pandemic, here is what we know about the coronavirus

CNN BREAKING NEWS

FDA authorizes 15-minute coronavirus test

Can we trust that they have performed and submitted adequate QA?

APRIL 1ST UPDATE

The Atlantic. Excellent. Read all of it carefully.

Among other adverse possibilities, an incentive for "Dry Labbing."
_____________

More to come...