The KHIT Blog: #COVID19 assay update: "Test performance evaluation of SARS-CoV-2 serological assay"

Monday, April 27, 2020

#COVID19 assay update: "Test performance evaluation of SARS-CoV-2 serological assay"

"This is a preliminary report of work that has not been certified by peer review. This should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information."

Ran across a link to this recent R&D paper (39 pg PDF) over at Naked Capitalism. Interesting, in light of my prior rants on lab QA issues.

ABSTRACT
Background
Serological tests are crucial tools for assessments of SARS-CoV-2 exposure, infection and potential immunity. Their appropriate use and interpretation require accurate assay performance data.

Method
We conducted an evaluation of 10 lateral flow assays (LFAs) and two ELISAs to detect anti SARS-CoV-2 antibodies. The specimen set comprised 130 plasma or serum samples from 80 symptomatic SARSCoV-2 RT-PCR-positive individuals; 108 pre-COVID-19 negative controls; and 52 recent samples from individuals who underwent respiratory viral testing but were not diagnosed with Coronavirus Disease 2019 (COVID-19). Samples were blinded and LFA results were interpreted by two independent readers, using a standardized intensity scoring system.

Results
Among specimens from SARS-CoV-2 RT-PCR-positive individuals, the percent seropositive increased with time interval, peaking at 81.8-100.0% in samples taken >20 days after symptom onset. Test specificity ranged from 84.3-100.0% in pre-COVID-19 specimens. Specificity was higher when weak LFA bands were considered negative, but this decreased sensitivity. IgM detection was more variable than IgG, and detection was highest when IgM and IgG results were combined. Agreement between ELISAs and LFAs ranged from 75.8-94.8%. No consistent cross-reactivity was observed.

Conclusion
Our evaluation showed heterogeneous assay performance. Reader training is key to reliable LFA performance, and can be tailored for survey goals. Informed use of serology will require evaluations covering the full spectrum of SARS-CoV-2 infections, from asymptomatic and mild infection to severe disease, and later convalescence. Well-designed studies to elucidate the mechanisms and serological correlates of protective immunity will be crucial to guide rational clinical and public health policies.

"Well-designed studies to elucidate the mechanisms and serological correlates of protective immunity will be crucial to guide rational clinical and public health policies."

Important, vitally necessary work. This particular effort focuses on qualitative ("positive/negative") screens via which to detect the presence (or absence) of Novel Coronavirus antibodies--possible markers of immunity for those having been exposed to the pathogen (even if subsequently remaining asymptomatic).

Suffice it to observe again than bioassay accuracy does not come quickly or on the cheap (gotta love their poignantly euphemistic "heterogeneous assay performance" admission). Moreover, after a test method is validated and approved at the R&D level, ongoing vigilant QA remain necessary at the production bench level. Neither is that cheap. Concurrent best-practice minimization of false positives and false negatives (the "sensitivity" vs "specificity" thing) is no cakewalk.

We remain seriously behind the curve on effective screening at scale for the current pandemic crisis.

STATnews article:

Many states are far short of Covid-19 testing levels needed for safe reopening, new analysis shows

More than half of U.S. states will have to significantly step up their Covid-19 testing to even consider starting to relax stay-at-home orders after May 1, according to a new analysis by Harvard researchers and STAT.

The analysis shows that as the U.S. tries to move beyond its months-long coronavirus testing debacle — faulty tests, shortages of tests, and guidelines that excluded many people who should have been tested to mitigate the outbreak — it is at risk of fumbling the next challenge: testing enough people to determine which cities and states can safely reopen and stay open. Doing so will require the ability to catch reappearances of the coronavirus before it again spreads uncontrollably…

UPDATE FROM SCIENTIFIC AMERICAN

Scientists have long warned that the rate of emergence of new infectious diseases is accelerating—especially in developing countries where high densities of people and animals increasingly mingle and move about...

POTUS TODAY

President Trump: "The testing itself is going really well. No complaints." pic.twitter.com/dVSUWNOCiB
— The Hill (@thehill) April 27, 2020

APR 28TH UPDATE

YASCHA MOUNK, associate professor at Johns Hopkins University

America is still behind on testing for COVID-19. Although Trump promised almost two months ago that anyone who wanted a test could get one, the U.S. has still conducted only about 5.4 million. The country needs to increase its testing rate at least threefold to reopen safely.

America is also behind on test and trace. Some countries, such as South Korea, now have robust systems in place to inform people that they have been exposed to the coronavirus, and need to self-isolate. But implementing such a system requires two things the United States sorely lacks: widespread trust in the government and a coordinated response from the White House.

In the absence of a federal strategy, some states, such as New York and Massachusetts, are trying to develop their own test-and-trace systems. But without help from Washington, they will likely lack both the resources to build a comprehensive system and the ability to persuade a large majority of their residents to sign up for an app that tracks their movements. Even if, against the odds, they should succeed in both these tasks, they face another obvious obstacle: Viruses don’t respect state lines.

If he were truly interested in limiting the damage to America’s economy, and opening up the country, Trump would be laser-focused on remedying these problems. Instead, the president has doubled down on culture wars and quack cures.

Early last week, Trump fanned the flames of the irresponsible protests against stay-at-home orders that are now being staged in cities across the country. A few days later, he vowed to “suspend immigration” to the United States. Then he suggested that scientists look into the possibility of injecting patients with bleach.

For all his blustering demands to get the country back to normal, the president is failing to take the steps that are required to reopen the economy without a horrific death toll. And for all the ingenuity shown by individual governors, the absence of a coordinated federal strategy may prove impossible to overcome…

Yeah. Interesting Atlantic article.

A SAD DAY HERE

Two years ago today, my younger daughter Danielle succumbed to her severe pancreatic cancer, 20 years after we lost her elder sister Sissy to an unrelated cancer.

Sigh...
_____________

More to come...

No comments:

Brave New Health

Commonwell Health Alliance

Another important read (pdf)

I love this kind of stuff. It sustains and humbles me. "As politicians, advertisers, salesmen, and propagandists for various political, economic, moral, religious, psychic, environmental, dietary, and artistic doctrinaire positions know only too well, fallible human minds are easily tricked, by clever verbiage... Common language—or at least, the English language—has an almost universal tendency to disguise epistemological statements by putting them into a grammatical form which suggests to the unwary an ontological statement. A major source of error in current probability theory arises from an unthinking failure to perceive this."

Quotes

"An economist is a person who sees something that works in practice and tries to figure out whether it will work in theory."

- J.D. Kleinke, medical economist
___

"The only person who enjoys change is a baby with a wet diaper."

"Every misspent dollar in our health care system is part of somebody's paycheck.

- Brent James, M.D., M.Stat

“We could do healthcare, at markedly higher quality, for everyone in this country, without rationing or denying anybody the care that they need, without having the government dictate how doctors practice or whether hospitals could expand, at half the cost we do it now.”

- Health Care Futurist Joe Flower

Most of the sciences, unlike parts of medical science, are not concerned with the impossible. There is not complementary and alternative physics, or chemistry, or biochemistry, or engineering. These disciplines compare their ideas against reality, and, if the ideas are found wanting, abandoned."

- Mark A. Crislip, MD

"Q: How much alcohol is too much?
A: More than your doctor drinks."

- a physician I once heard speak during a CME presentation

“Just because science doesn’t know everything, doesn’t mean you get to fill in the gaps with whatever fairy tale most appeals to you.”

- Dara O’Briain

'[I]t is one small step from using the computer for "helping" doctors to monitoring them, judging them, dictating to them what to do, and withdrawing payment for computer non-compliance. The use of computer data is a multi-edged sword. It can be used for the "good," facilitating diagnosis and treatment and making it more accurate and up-to-date, and for “evil,” invading privacy, inviting security breechs, and making decisions based on the opinions of remote authorities rather than those present at the patient-doctor encounter.'

- Richard Reece, MD

“[T]here ARE statistics which are non-political. Just because The Washington Post/Fox News reports the temperature is 75 degrees doesn’t mean it’s really snowing and sunscreen is a liberal/conservative plot. Even if you earn a living being ideological.”

- Michael L. Millenson

"It is a generally a fairly convincing argument that people shouldn’t have to be subsidized to undertake a change which is in their best interest.

The reconciliation seems to be that EHR is not supposed to make a doctor’s practice more efficient and higher quality. It is supposed to make the system of care more efficient and higher quality, which is not the same thing. Those of you who took calc recall that maximizing the total of variables is not achieved by maximizing any one variable and this is a perfect example of that.

Those of you have served in combat certainly noticed that too — if everyone works as a team the unit takes fewer casualties. If you try to save your own hide, you might, but at the expense of more casualties overall."

- Al Lewis

"There are two ideas to keep in mind about Bayesian reasoning and how we tend to mess it up. The first is that base rates matter, even in the presence of evidence about the case at hand. This is often not intuitively obvious. The second is that intuitive impressions of the diagnosticity of evidence are often exaggerated."

- Daniel Kahneman, "Thinking, Fast and Slow"

"Physicians apply advanced scientific knowledge, but they must do so without the favorable conditions that experimental scientists create for themselves. Multitasking is forced on physicians, often in chaotic environments and under severe time and resource constraints."

- Lawrence and Lincoln Weed, "Medicine in Denial"

"It’s time to stop the whining about Obama care and acknowledge we already have universal health care. We just pay for it in the stupidest way possible that ensures problems are that much more disastrous and complicated when they’re finally treated."

- Mark Hoofnagle, MD, PhD

"Every act of conscious learning requires the willingness to suffer an injury to one's self-esteem. That is why young children, before they are aware of their own self-importance, learn so easily."

- Thomas Szasz, MD
___

"Of course, one reason that process metrics* are so popular is that processes are much easier to define and measure than outcomes."

- The Skeptical Scalpel
___

"There is an “illusion of validity” for any random data point, a seductive sense that is colored by what we hope will be true. Mountains of pharmaceutical claims are often made from mere molehills of data."

- Danielle Ofri, MD
___

"Joy empowers people. It is a source of energy that enables people to hope and plan and change their lives for the better. Spend some time around someone who is relentlessly negative and how do you feel–drained, right? More and more research shows that joy is not something that just happens to you, like a bolt of lightening out of the blue. Joy is, instead, a habit to cultivate. Negative thinking and despair are the crabgrass of our souls–weeds that take root and spread, sometimes to all areas of life. Joy, in contrast, is a soul’s rose–hardy when cared for, able to put down roots over time and withstand disease and extremes. Like a rose, however, your joy can become blighted from neglect or harsh conditions. We all need to tend to our joy–to prune away the badness, and to know that, even though it may look like a prickly bare root, if you invest time in a joyous outlook, gorgeous things will bloom, even in the harshest conditions."

- Dr. Jan Gurley
___

"'Solutions' exist only in mathematics."

- Karen Martin
___

"The issue of how to regulate clinical software is, in the long run, indistinguishable from the issue of how to regulate medicine. The only difference is that medicine is practiced in the open, without secrecy, subject to peer review, and under a merit-based state license."

- Adrian Gropper, MD
___

"Economist, rope, tree: some assembly required."

- Source unknown

DISCLAIMER:

I write this blog wholly on my own time and my own dime. The views proffered are expressly my own as a concerned and active citizen/taxpayer (in addition to being the result of my substantive experience in the various IT fields), and in no way reflect any policy views of my former employer, notwithstanding that some of the thinking has indeed obviously been spurred by the implications of the work with which I have been doing for them.

FAIR USE POLICY
I cite a ton of news and web sources spanning the breadth of relevant technical and policy domains, sometimes at substantial length. I believe I remain well within the bounds of "Fair Use," as [1] I am not doing any of this for profit, [2] I always provide attribution and links -- which, [3] far from negatively impacting any copyright holders' commercial interests, might actually increase traffic to and interest in their offerings.

Nonetheless, should I post anything of yours regarding which you have any objection, just let me know and I will remove it forthwith.

The KHIT Blog

Search the KHIT Blog

Monday, April 27, 2020

#COVID19 assay update: "Test performance evaluation of SARS-CoV-2 serological assay"

No comments:

Post a Comment