Search the KHIT Blog

Monday, December 13, 2010

My Excellent Adventure into the Mecca of Quality

American Society for Quality (ASQ) Healthcare Division Chairman Dr. Joe Fortuna (an enthusiastic REC supporter, it should be noted) invited me (no, emphatically, repeatedly implored me) to come and participate in the 2010 December weekend Strategic Planning Retreat of ASQ's Healthcare Division (HCD) Leadership.

You just can't say "no" to that (I was honored and flattered). So, above, here I am on final approach over Lake Michigan into snow-covered MKE. Below, downtown Milwaukee, the ASQ headquarters building.

Above, "River Walk," right outside HQ and the adjacent hotel. Yeah, it was cold. And got progressively colder across the weekend. By the time I flew out last night, the wind chill was 5 degrees F, with 40+ MPH wind gusts. My plane fishtailed down the runway and then seemed to just jump right up off the ground after only several hundred yards.

"...uh, yes, thanks, I would like a drink. What Scotch do you have? Dewars? That'll be fine. Rocks. Make it a double..."

Below, a wall of ASQ history.

Above, in the foyer just outside the main conference room, a Deming Red Bead display.

Mecca for us QC/QA/QI types. I, in particular, am hardcore Deming. In addition to internalizing his acclaimed work "Out of the Crisis," I even own and have read "Some Theory of Sampling" (not that I fully grasp all of it).

My wife and I have been loyal ASQ members for more than two decades. We live and breathe this stuff.
(BTW, you can click on any of the photos to see the original size shot).

We worked hard the whole time. A packed agenda (all focused on how the ASQ HCD can add tangible value and consequently grow the membership for the Division and ASQ more broadly). On Saturday we went straight through from 8 a.m. to 6 p.m., with a "working lunch" amid the proceedings. Dr. Joe was an active and forceful (and genial) taskmaster.

Everyone I met was utterly impressive, too. Lots of talent in that assemblage. Pretty cool weekend for this lowly REC grunt.


We had a presentation regarding ASQ's strategic plan, part of which involves the organization's "re-branding" effort. We are to henceforth regard ourselves as "ASQ: The Global Voice of Quality." No more reference to the "American Society for Quality" long the descriptive phrase allusive of the ASQ acronym. It's now an explicity international effort.

ASQ CEO Paul Borawski, who presented to us on the current and upcoming ASQ strategic direction, blogs about ASQ's new path on his "View from the Q" blog.

Here's Paul speaking to the future of "quality" in 2009:


We also had a nice presentation on the "Socially Responsible Organization" of which ASQ is an integral part.
"The SRO* is a new international movement to share insights and expertise on, and examples of, the evolving area of social responsibility, and to help define what it means to be a socially responsible organization.

"The SRO* aims to explain the business case for social responsibility and the role of quality to achieve results..."

"...ASQ was awarded the administration of the U.S. Technical Advisory Group (TAG) on Social Responsibility by the American National Standards Institute. The U.S. TAG is the U.S. member of an international working group chartered with developing an international standard on social responsibility. ASQ’s involvement is a direct extension of its existing leadership role in standards development."

"The social responsibility guidelines will be released in mid- to late 2010 as ISO 26000. The ISO 26000 guidelines will cover seven core areas:

  • organizational governance
  • human rights
  • labor practices
  • fair operating practices
  • environment
  • consumer issues
  • contribution to the community and society..."
I signed up to participate. We'll see what ensues.


This was a very interesting demo presentation of a web app built by ASQ wherein clinical providers can sign up, enter data, and receive continually updating dashboard status feedback regarding their readiness for both ARRA / HITECH Meaningful Use and Baldrige criteria. This was to have been presented to ONC brass in DC the week following our conference, and, as soon as I learn of the upshot, I will report on it.

Sunday, November 21, 2010

First, Do No "Hold Harmless"

I read a very interesting post the other day, "The Fine Print," written by one of my favorite heath care blogging colleagues, Margalit Gur-Arie, regarding the
"alleged contracting practices of EHR vendors and their notorious “hold harmless” clauses, which indemnify the EHR vendor from all liability due to software defects, including liability for personal injury and death of patients. What this means in plain English is that if a software “bug” or incompetency caused an adverse event, and if you (or your hospital) are faced with a malpractice suit, the EHR vendor cannot be named a co-defendant in that suit and you cannot turn around and bring suit against the vendor for failure to deliver a properly functioning product."

"The AMIA paper also asserts the existence of contractual terms preventing users and purchasers from publicly reporting, or even mentioning, software defects, including ones that may endanger patient safety..."

Wow. Assertions of blanket indemnity, coupled with a "gag order"? Is this something regarding which our REC provider clients be made aware during EHR vendor selection and contract negotiation? (Beyond things such as practice "data ownership" and -- relatedly -- EHR "source code escrow"?) How substantive is the liability concern?

The grist for this post was the recent publication of an American Medical Informatics Association (AMIA) Board Position Paper entitled "Challenges in ethics, safety, best practices, and oversight regarding HIT vendors, their customers, and patients: a report of an AMIA special task force." (PDF)
"...Some vendors incorporate contract language whereby purchasers of HIT systems, such as hospitals and clinics, must indemnify vendors for malpractice or personal injury claims, even if those events are not caused or fostered by the purchasers. Some vendors require contract clauses that force HIT system purchasers to adopt vendor-defined policies that prevent the disclosure of errors, bugs, design flaws, and other HIT-software-related hazards..."

One commenter made this observation on Margalit's post:
"I'm still waiting to see an installation of any EMR be fully tested by either the vendor or the organization who purchased it."


All of which set me to reflecting on my first professional technical paper, written in 1988 during my tenure as an environmental radiation lab programmer and quality control analyst.

While those days were a time prior to indoor plumbing in IT terms, some of the points still resonate. As I began the paper:
"... [vendor] makes no representations or warranties with respect to the content hereof, and specifically disclaims any implied warranties or merchantability or fitness for any particular purpose for both this manual and the product it describes. Furthermore, [vendor] reserves the right to revise this publication and make changes in the content hereof without obligation to notify any person of any such revision or changes..."

"In less than a decade, microcomputers and software applications have become ubiquitous, indispensable tools in business, industry, and the sciences. The end-user faces a bewildering array of options with respect to makes, models, peripherals, and software compilers, libraries, firmware, and applications packages. since the foregoing disclaimer may be found in the product documentation of virtually every commercial microcomputer hardware and software vendor, the end-user must blend the array of options, possible algorithmic deficiences, and system incompatibilities into a comprehensive product. The user assumes -- usually unwittingly -- the cumulative responsibility for assuring a quality output..."

Twenty three years later, the core liability concerns remain. And, still, the "user assumes -- usually unwittingly -- the cumulative responsibility for assuring a quality output."

The lab wherein I worked did a signification volume of "forensic-level" analysis, i.e., much of our output was destined for use as evidence in radiation contamination and dose-exposure liability litigation
. Consequently, we turned over every rock, pebble, and grain of sand in search of conditions inimical to legally-defensible data "quality." Our Technical Director in particular, Dr. James Dillard (my mentor on this and other projects), had amonished us to never take computer-generated results at face value. He was fond of saying "you get what you INspect, not what you EXpect." As I note on my website preface citing and linking my paper:
You enter some numerical data into a computer and get some results back out. Do you simply assume they are "accurate" and report them to the client? Not in our lab. For example, even assuming your data and formula/function entries into a spreadsheet are correct, does it thereby necessarily invariably follow that the calculated results will be so?

And so it came to be express policy (reflected in our "IT/ORL Software Quality Assurance SOP" I had a hand in writing) that we thoroughly test every software application -- in-house developed and commercial (off-the-shelf and 3rd-party custom-developed alike) -- and every computer wherein they would be installed for use in generating client-reportable results.

Complex as all of that was, those were the bucolic ancien days of relative computing simplicity. Today's hyper-connected, mobile world of 24/7 exponentially increasing digital apps and platforms -- from the client-server to the "Cloud" -- presents a potential host of new challenges.

Given the considerable complexities comprising Health Information Technology (very little of which go to actual mathematical computing, it should be noted), these challenges are already within the crosshairs of the Medical Liability people. Recall, again, my prior citation of the Brouillard article.

Brouillard concludes:
"Although EHRs have now achieved mainstream, clinical adoption, EHR-related liability trends have not developed fully. At this early point, we can discern some potential liability areas. In an early EHR implementation stage, source of truth issues and expansion of liability issues may arise. In using EHR systems, the evolving standards of care for clinical documentation and work-arounds pose risks. Security as mandated by data breach laws or retention and storage issues involving e-discovery liability and data integrity have also emerged as important areas."

Consequently, it is rather unsurprising that Counsel for EHR vendors would insist on boilerplate blanket "Hold Harmless" beg-offs. I rather doubt such stipulations would survive the first serious court test, given a case (assuming a jury trial) wherein a patient was harmed as a result of a documentable software flaw that prevented a provider from being made aware of an exigent patient circumstance (or induced a provider to take injurious action she would otherwise have not absent the software flaw). Moreover, I am with AMIA on this point:
"f. “Hold harmless” clauses in contracts between Electronic Health Application vendors and purchasers or clinical users, if and when they absolve the vendors of responsibility for errors or defects in their software, are unethical." [pg 3]


To be fair, one principal reason for attempts at fine print "hold harmless" inoculation goes to the civil litigation reality of "Joint and Several Liability" -
Joint and several liability is a form of liability that is used in civil cases where two or more people are found liable for damages. The winning plaintiff in such a case may collect the entire judgment from any one of the parties, or from any and all of the parties in various amounts until the judgment is paid in full. In other words, if any of the defendants do not have enough money or assets to pay an equal share of the award, the other defendants must make up the difference.

e.g., if you are found to be perhaps only, say, one percent "liable" (exacerbated by the fact that civil liability is determined by subjective "more-likely-than-not" "preponderance" criteria), but your relatively deep pockets finds you with 100% of the attachable assets via which to satisfy a judgment, well...

So, what're really perhaps at play here are the relative deep pockets of EHR vendors vis a vis the materially shallower ones of individual potential defendant physicians.


"Medical errors and adverse events may result from individual mistakes in using EHRs (e.g., incorrectly entering information into the electronic record) or system-wide EHR failures or “bugs” that create problems in care processes (e.g., “crashes” that prevent access to crucial information)." [pg 2061]

' the use of EHRs grows, failure to adopt an EHR system may constitute a deviation from the standard of care. The standard of care is usually defined by reference to what is customary among physicians in the same specialty in similar settings. Once a critical mass of providers adopts EHRs, others may need to follow..." [pg 2065]

Medical Malpractice Liability in the Age of Electronic Health Records Sandeep S. Mangalmurti, M.D., J.D., Lindsey Murtagh, J.D., M.P.H., and Michelle M. Mello, J.D., Ph.D. n engl j med 363;21 November 18, 2010

So, given that the ever-wider-spread deployment of HIT is seemingly inevitable (and, a prospect which I obviously support), what are the some of the truly salient liability risk concerns?
  • EHR "Usability" issues (PDF) that might contribute to inadequately "idiot-trapped" data input errors (including mistakes and omissions);
  • Code logic flaws that could lead to exigent "alerts" missed (or, conversely, irritating recurrent "false positive" alerts that precipitate user cynicism and apathy);
  • Relatedly, "clinical decision support" logic flaws (or simple inadequacies);
  • More general code flaws resulting in "crash" prone systems, leaving clinicians potentially in the lurch during time-sensitive points of care;
  • OS and other incompatibilities (including adverse interactions with other resident apps);
  • Math errors.
As I've previously noted, EHRs typically do very little outright math of any appreciable sophistication, so I put that last on the list (though that may indeed a bit change over time -- apps going beyond, e.g., simply doing BMI arithmetic, lab value averages, and growth chart plotting, etc). The typical EHR (at least of the ambulatory variety) is really just usually a Java or C++ (or otherwise ".net") coded GUI front end app sitting atop and hooking into a relatively complex RDBMS (typically a multi-table SQL relational database these days), one whose principal purpose is to record and then re-display (either onscreen or in print) the myriad requisite subsets of administrative and clinical data as efficiently as possible (ideally).

Nonetheless, the systems are indeed extremely transactionally complex, and, absent thorough and consistent QA (including industry consensus stds? FDA oversight?), they could be vulnerable to a host of "gremlins," the upshot of which could range from the merely exasperating-to-the-workflow to the punitive "joint-and-several liability" class-action judgment in the wake of patient harms.
I have to note yet again that ONC-ATCB EHR "certification" for Meaningful Use has to do exclusively with an application's ability to reliably record and regurgitate the MU measures, in a HIPAA-compliant manner. Nothing more. Nothing pertaining to application "quality" more broadly (and in the more truly "meaningful" sense of efficient, "usable" functionality).
I will be watching the developing law here with interest. I would also exhort all vendors and users alike to think long and hard proactively about the considerable breadth of EHR "software quality" issues.


"Physicians and other healthcare providers are increasingly relying on EHR systems for the practice of medicine. As the number of EHR system providers increases and as these systems integrate with other systems to import and exchange data, it is important to track and understand issues of concern as they develop. This will, in-turn, allow for improvement in EHRs, in patient safety, and may result in liability reduction..."




I was sifting through papers in my office today, and ran back across this law journal article I'd read a while back (PDF) and had given the thorough yellow highlighter treatment.

I should have cited this by now. Just slipped my attention. It antedates both the Brouillard and NEJM pieces. It's the first place wherein I ran across a call for FDA (or some federal entity) regulation of EHRs.
...the novel and significant risks generated by EHR systems cannot be ignored. Products with poor information display and navigation can impede rather than facilitate providers’ work. The growing capabilities of EHR systems require increasingly complex software, which heightens the danger of software failures that may harm patients...

...Thus far, the legal literature has not assessed the need for careful regulatory oversight of EHR systems akin to that required, in principle, by the Food and Drug Administration (“FDA”) for life-critical medical devices. This Article begins to fill that gap. It analyzes EHR systems from both legal and technical perspectives and examines how law can serve as a tool to promote HIT. Extensive regulations already exist to govern the privacy and security of electronic health information. Privacy and security, however, are only two of the concerns that merit regulatory attention. Perhaps even more important are the safety and efficacy of these life-critical systems. [pp. 106-107]

A good read. And, in light of the more recent news of EHR vendor "Hold Harmless" attempts, this observation near the end of the monograph is interesting:
...the threat of product liability or medical malpractice litigation could deter misconduct by both EHR system vendors and health care providers. Plaintiffs may sue providers if they suspect that they suffered poor outcomes because providers failed to implement or properly use EHR systems, for example, by neglecting to utilize decision-support features that may have averted a medical mistake. Likewise, plaintiffs might name EHR system vendors as defendants if they believe the harm is rooted at least partly in a design flaw, and health care providers might bring in vendors as third party defendants if they believe the vendors to be partially at fault. Audit logs and capture/replay would be helpful to all parties in investigating and proving their claims concerning system failures and provider negligence or lack thereof. [pg 161]

Do no "Hold Harmless."

Thursday, November 11, 2010

November 2010 updates


The Ingenix 2 gig USB flash drive is apparently machine-washable. Tumble dry, medium heat. LOL. I can't believe I did that. Found it in my cargo jeans pocket after the dryer stopped. Didn't lose a single file. Wow. (It was a freebie I got a while back at some HIT marketing event; has an Ingenix "CareTracker" demo on it. This is not to compromise my REC "vendor neutrality," BTW.)


No one at ONC really wants to talk about it for the public record these days (nor even much privately). One number that wafted my way recently was ~17,000 sign-ups nationally to date. "PPCP" means "Priority Primary Care Providers," who can get federally subsidized REC soup-to-nuts consulting/facilitation services during the first two years of the HITECH program. Eight months in, then, this means that REC "enrollment" (which is optional, recall) is running only about 50% of target-to-date in the aggregate, in light of the national goal to bring 100,000 primary care providers "meaningfully" on board within two years.

REC provider recruitment barriers persist, obviously - to wit,

  • EHR cost and concomitant ROI dubiety (given productivity loss and "everyone-benefits-except-me" concerns);
  • Skepticism that the Meaningful Use reimbursement money will actually be forthcoming (particularly in the wake of the 2010 mid-term election outcomes -- with Republicans now loudly vowing to de-fund everything associated with "ObamaCare");
  • Specific physician anxiety and anger regarding the yet-again pending draconian Medicare reimbursement cuts;

For example: "Unless Congress acts, physicians are just weeks away from taking a 23% cut in Medicare reimbursements mandated by the sustainable growth rate (SGR) formula. The cut is scheduled to go into effect on Dec. 1 and will be followed on Jan. 1 with an additional cut, bringing the total to 25%."
  • Medicaid provider participation remains an acute concern. Some states (including my own) are considering dropping out of the Medicaid program, in light of rapidly increasing patient enrollment (owing to our record high and seemingly intractable unemployment rate) concomitant with reductions in provider reimbursements during a difficult time of state budget deficits. Even should this not happen, their participation in the HITECH Meaningful Use program is voluntary, and not fully federally funded (states have to submit a "plan" to the feds, and come up with an unfunded 10% of "reasonable administrative expenses");
  • More broadly, antipathy toward anything related to the motives of "the government," one being the allegation that this is all about the feds wanting to be able to easily gather all of our medical information in order to dictate [1] what treatments we can obtain (and how doctors must perform them) and [2] what we can eat (Michelle Obama's menacing BMI Violator Celery Stick Police);
  • Vendor, VAR, and commercial consultant assertions that "you don't need REC help, we'll get you to MU" (owing to that many of us have to charge subscription, hourly, or back-end fees, given that the HITECH subsidy is not 100% and is incrementally "milestone" based);
  • General resistance-to-change inertia. As my mentor Dr. Brent James was fond of saying, "the only person who enjoys change is a baby with a wet diaper."
etc. It's all getting a lot of news coverage these days, pro and con.

"...Noting that the private sector has not been able to develop and adopt methods for significantly improving health outcomes in the U.S., [Paul Tang, MD] called recent federal legislation "transformative" by providing the goals, timetables, and financial incentives necessary to achieve desired change. "All the necessary forces are aligned," he said.

The Affordable Care Act (ACA), and the American Recovery and Reinvestment Act (ARRA) of 2009 share many of the same requirements for achieving and documenting cost and quality of care criteria, said Tang, and both will depend on the "meaningful use" of certified EHRs targeted by ARRA for widespread use by 2014. Meaningful use requirements address key health goals, including improved quality, safety and efficiency; engaging patients and their families; improving care coordination; improving population and public health; and ensuring privacy and security protections.

Tang was a keynote speaker at the MedeAnalytics Clinical Leadership Summit held Nov. 11-12 at the Fairmont Hotel in San Francisco."
Notwithstanding the truth of the foregoing assertions, I detect increasing wafts of circle-the-wagons angst, given that The Tea Party Cometh soon to the halls of Congress. While some feel that HITECH and the RECs will be safe from the 2011 legislative truncheon, I am not so sure.

HIT (and REC assistance) continues to be a tough sell, at least from my perspective in Nevada (but, in fairness, my state is the fiscal basket case of the nation; highest unemployment rate, highest mortgage foreclosure rate; 80% of non-foreclosed residential mortgages "under water," largest state budget deficit in the nation, proportionally, etc. A swell place to live these days).

I had a bit of irascible Photoshop commentary fun with this entire "recruitment" notion early on.

Whatever. They ostensibly re-hired me for the Meaningful Use Adoption Support "technical assistance"/QI effort, upon which I really want to focus my energy in lieu of all this "sales rep" stuff.


The minimal essence I fought hard internally to establish. It was not universally loved within my REC team, having been deemed in some incumbent quarters too "old school / industrial." Right. Click the image to enlarge.

Rather simple, conceptually (props to my QA Guru wife for giving me her SOP template to use). My argument (which prevailed in the end) was that, should we manage nothing else beyond getting our client providers to Stage One Meaningful Use compliance, we would have met both our ONC and client-contractual obligations. Simply a matter of exigent priority, i.e.,

  1. Document and visualize the workflow components for each Meaningful Use criterion, as dictated by each EHR platform;
  2. Write it all up in a "Standard Operating Procedure" (SOP*) for client distribution;
  3. Include in the SOPs the sets of "screen shots" (where relevant) that can navigate any authorized users to the target MU "money fields" (where the data have to be consistently recorded).
[* We tend to soft-pedal to a degree the "SOP" thing, using the "softer" phrase "Standard Work" that is part of the "Lean Methodology" lexicon. "SOPs" conjure up MEGO inducing images of dull ring binder "Policies and Procedures Manuals" that typically sit on shelves unopened once written.]
As I have stated before, I would favor laying this sort of thing off on the EHR vendors (given that they know their products -- specifically the MU documentation navigation paths within their respective systems, and that such are -- implicitly, anyway -- part of ONC Certification), leaving RECs the time to concentrate on actual "process improvement" via which to make the entire effort a bottom line winner for the providers.

In fact, were I setting ONC EHR Cert policy, I would have made such explicit instructions part of certification requirements. Perhaps once "permanent certification Registrar" requirements are set forth, such will be the case. I will certainly lobby for it. But, I won't be holding my breath.


You might recall my earlier blog post reflections regarding how Meaningful Use compliance could easily consume so much additional FTE labor cost as to negate the incentive reimbursements. "Use" case in point, from a study I found posted on our HITRC (again, click the image to enlarge).

I'm just too sorry: THIRTY FIVE seconds to document a CPOE lab order (in the VistA EHR)? OK, annualize that, extrapolating from a couple of dozen patients per day. It's not viable. As I've pointed out elsewhere on this blog, just adding an aggregate additional couple of minutes labor per chart documenting all the Meaningful Use criteria will serve to nullify the reimbursement payments. Hence my personal imperative of trying to inculcate "Lean" principles within my client clinics as I work with them in order to find ways to decrement the workflow FTE burden overall in support of effective HIT adoption.


Nice resource. Click the image to visit their site.


Click the image to enlarge.


Returning for a moment to the excellent book "On The Mend."

"Lean transformation is all about Dr. Deming’s Plan Do Study Act (PDSA), otherwise known as the scientific method. There is no simple formula to copy and no quick path to success. Instead you must perform your own experiments— tailored to the mission and circumstances of your organization. And then you must honestly study the results and act on your findings, including sharing them with the healthcare community.

Yeah, "PDSA." All the rage in progressive process QI circles for quite some time now (and, color me a believer, as I stated early on).

I would revise to acronym, however, to "SPDSA," i.e., "STUDY, Plan-Do-Study-Act." At the risk of belaboring a point that may be implicit in the minds of most people involved in this work (i.e, that initial quantitative baseline assessment "study" is reflexively included in the "Plan" phase if it's truly "scientific"), I never lose sight of the source, the late Dr. Deming himself:
"Statistical control. A stable process, one with no indication of a special cause of variation, is said to be, following Shewhart, in statistical control, or stable. It is a random process. Its behavior in the near future is predictable. Of course, some unforeseen jolt may come along and knock the process out of statistical control. A system that is in statistical control has a definable identity and a definable capability (see the section "Capability of the process," infra).

In the state of statistical control, all special causes so far detected have been removed. The remaining variation must be left to chance -- that is, to common causes -- unless a new special cause turns up and is removed. This does not mean do nothing in the state of statistical control; it means do not take action on the remaining ups and downs, as to do so would create additional variation and more trouble (see the section on overadjustment, below). The next step is to improve the process, with never-ending effort (Point 5 of the 14 points). Improvement of the process can be pushed effectively, once statistical is achieved and maintained (so stated by Joseph M. Juran many years ago)."

["Out of the Crisis," Ch 11, page 321]

In Deming's view, attempting to "improve" processes that are empirically unstable constitutes what he called "tampering," interventions that typically only serve to make matters worse.

A quick example from my own environmental lab "SPC" experience more than 20 years ago (before indoor plumbing, in IT terms). One of my projects (PDF) involved the development of a computerized system via which to track the accuracy, precision, and stability of the lab's numerous multi-channel radiation detector instruments.

QC personnel ran and recorded daily "sealed source" and "background" checks for every detector, along with QC specimens known as "matrix spikes," "D.I. water spikes," "matrix blanks," "matrix dupes," etc (we were also subject to "blind" QC samples submitted by clients and regulators, posing as production samples). My software calculated the requisite statistics every day (e.g., mean response, std deviation/"coefficient of variation," upper and lower "warning" and "control limits," and linear regression trend slope), updating them with each new data point (through the first 20 entries, that is). Statistical "outliers" would then be readily apparent, and any out-of-calibration "drift" would become apparent via the fitted "slope." In the foregoing example, the detector looks to be "stable," evincing a random daily variablity of about 1% of the average.

Below, same chart, annotated to call attention to a couple of early potential "mini-trends." Across the initial 20 or data entries, it appeared that the detector might be trending sharply down, and, shortly thereafter, after the "dive" had abated, there was a downward "run" of six consecutive entries (the red oval).

Given that our lab hewed to "forensic standards," owing to the fact that much of our work was destined for use as evidence in contamination/dose exposure litigation, we tended to be hyper-vigilant when it came to QC. Nonetheless, textbook "applied process control statistics" can't do your thinking for you. Should detector LB5100/TA/002 have been taken offline for intervention after the first 20 accrued data points? Or, in the wake of a subsequent negative "run" of six daily results? Obviously, our lab's Technical Director didn't think so, and the overall plot tended to back his sanguinity.** No SPC "outliers" and about a 1% or less daily average variation across nearly three months; what's not to love? (LOL, such was not always the case.)

** In this case, you bring to bear contextual expert judgment. Like an aircraft pilot cross-referencing her instrument panel under IFR conditions for accurate and safe flight absent the ability to see outside the cockpit, our lab manager would take into account not only the daily sealed-source detector disintegration "counts," but also the correlative results of routine bench-chem derived lab QC samples (the breadth of spikes and dupes inserted into the production stream). Morever, expert judgment also informs that, given the stochastic nature of radionuclide decay eminations (all of them half-life "decay-corrected" in our system btw), were you to re-run sealed sources multiple times per day (not practical), you would still see random intra-day variation.

Statistical estimates are
tools. They don't properly replace domain knowledge and experience. (Moreover, just because Excel now enables you to effortlessly fit an "nth-degree" polynomial to a scatter of a handful of data points
-- replete with a nicely displayed equation and impressive R-Squared stat -- doesn't mean you should.)
Those were fun times. And, I have tried to internalize all of the "lessons learned," foremost among them that, should you fail to discover (or you otherwise ignore) knowable "special causes" impacting a process and rectify them, your efforts at "improvement" way well be counterproductive.

But, as I now begin to actually work with my small outpatient primary care clinics doing "workflow analysis and re-design," though, I know I'm not even going to get close to such a level of rigorous statistical process analysis. Most of our workflow assessments are necessarily bound to be mostly "qualitative" (and MU compliance driven, at least across the Stage One period).

I guess we'll just take our "wins" where we can get them. Still, I remain drawn to true PDSA "lean" process improvement principles and procedures, e.g., as set forth by leading organizations such as the Thedacare Center for Healthcare Value:

"The A3 method assures that the PDSA cycle is followed and the changes are monitored. The process steps can be documented in a variety of formats, but it typically includes the following elements, on a single piece of paper. A3 refers to the standardized paper size of 11” x 17”.
  1. Title – Names the problem, issue, or topic
  2. Owner/Date – Identifies who owns the problem or issue and the date of the latest revision
  3. Background – Why is this important? What background information is important? What have we seen in gemba?
  4. Current Conditions – Show the current state using pictures, graphs, data, etc. What is the problem?
  5. Goals/Targets – What results do you expect? What are the key measures? (quality, cost, morale, delivery, access, etc.)
  6. Analysis – What is the root cause(s) of the problem? If you work to eliminate this root cause, will you make progress toward solving the problem?
  7. Countermeasures – What proposed actions do you intend to take to reach the target condition? How will you show how your countermeasure will address the root causes of the problem? What is the new standard process?
  8. Implementation – What needs to be done? Who will do it? By when? What are the performance indicators to show progress? How will people be trained in the new process?
  9. Follow Up – What issues can be anticipated? How will you capture and share learning? How will you continuously improve or begin the cycle again (PDSA)?"


One place I wherein routinely hang out.

These people are hardcore. (I have to admit to some episodic "scientism" wariness when reading some of their content). Among other topics, there's an ongoing debate at SBM regarding the epistemological and utilitarian differences (if any) between "EBM" (Evidence-Based Medicine) and "SBM" (Science-Based Medicine). As we ponder expanded "CER" (Comparative Effectiveness Research) as envisioned to be facilitated by ever more widespread HIT adoption, I would expect to see some heated debate at SBM on the subject (CER is already much unloved in other curmudgeonly clinical circles).

BTW: ONC now has a blog.

Worth visiting and perusing.


Very nice topical podcasts. Two thumbs up.

Wednesday, October 13, 2010

Mulligan: 7.5 months into my REC adventure

I was re-hired on March 2nd by HealthInsight for the REC initiative. Been a most intense, interesting, and at times sleep-deprivingly frustrating experience. I have a number of developments and concerns on which to report. I started this post on September 3rd, but my heavy work and travel schedule and a bout of illness have necessitated a Mulligan. So, here we go again.

As with most of my blogging, this will necessarily be incremental as time permits, as this is a personal, assiduously off-the-clock independent project
(albeit rather unloved by one of my Sups nonetheless) in support of the REC effort.

Among other things, I will reflect on my experience at the recent Salt Lake City REC Regional ONC Conference, the subsequent inaugural (sparsely attended) "Digital Medical Office of the Future Conference" in Las Vegas, the interesting October 12th Nevada Health Care Forum 2010, as well policy developments in the wake of the release of the 864 page Stage One Meaningful Use Final Rule and the subsequent announcements of both the release of NIST test specifications for ARRA Certification of EHR systems and the first two Certification Registrars -- CCHIT and The Drummond Group (I believe there is now a 3rd Registrar in place as well).


You ought read this timely, excellent piece ever so closely, yellow highlighter and red pen at the ready
(click the title image above for a PDF copy). A couple of takeaways: While everyone is focally anguishing over the intensified HIPAA privacy and security implications of HIT going forward, this legal writer (a medical liability attorney) points out that
  • routine use of HIT (inclusive of both EHR and HIE) will likely become an integral part of accepted legal "standards of care," e.g., "it seems logical that a liability inquiry will turn to how information technology has been used by clinicians and whether that use itself comports with the standard of care for maintaining and using medical records."
  • relatedly, issues of data accuracy, security (specifically with respect to inadvertent or intentional data corruption), and retention for "E-Discovery" are likely to find their way into court (including issues relating to the net probative value of the soon-to-be-mandatory always-on "audit logs" in certified EHR systems).
Calls continue for FDA regulation of EHRs as de facto "medical devices," and there are now legislative stirrings for mandating EHR use a condition of medical licensure.
"...from the health care law and medical liability perspective, EHRs mark a fundamental shift in how we use and collect electronic medical evidence, as well as what we must think about when anticipating health care litigation."

A lot to consider here. Download it and read it carefully.

BTW, I'd like to give an unsolicited shout-out to fellow (literally neighboring) HIT blogger John Lynn. I check his blogs every day (among the numerous others within which I routinely lurk), and comment there regularly in the spirit of engaging constructive dialogue. Click the images below for direct links (they're also in my "Links" column on the right).

Also: The NIST EHR Certification test SOPs:

There are 46 SOPs, addressing the full range of Meaningful Use criteria across outpatient and inpatient settings, e.g.,
§170.302 (g) Smoking status. Enable a user to electronically record, modify, and retrieve the smoking status of a patient. Smoking status types must include: current every day smoker; current some day smoker; former smoker; never smoker; smoker, current status unknown; unknown if ever smoked.
The standard here is silent on exactly how the options are to be presented to the EHR user. Mutually exclusive "radio buttons"? A drop-down menu choice list? (It is also silent on the underlying data structure -- which I would assume to be mostly SQL RDBMS tables. The entries could be stored literally as text strings [all lower case, as per the std?], or they could be recorded in binary 0/1 integer fields mapped to the "human-readable structured data" responses a user sees on the screen or in a printed report?)

One could also interpret this standard to mean that some pre- ONC certification EHRs already comply, with no upgrade revision necessary for this particularly criterion. See, for example, a screen shot I clipped of the current release of e-MDs:

One could, I suppose, manually (or via a macro) enter "current every day smoker" or one of the MU-requisite forced-choice alternatives into this catch-all "Tobacco/Alcohol/Supplements" free-text box.

Cleanly and efficiently tabulating/reporting the data back out, however, would likely be a mess. Let us count the ways user could (and would) misspell these.

One more example:
§170.302 (h) Incorporate laboratory test results.

1) Receive results. Electronically receive clinical laboratory test results in a structured format and display such results in human readable format.

2) Display test report information. Electronically display all the information for a test report specified at 42 CFR 493.1291(c)(1) through (7).

3) Incorporate results. Electronically attribute, associate, or link a laboratory test result to a laboratory order or patient record.
A question arose recently regarding lab findings that typically come back in as simply "positive" or "negative" (or "POS"/"NEG") or, equivalently, "present" or "not found/not present" etc text. Again, the standard is silent, in that the specified "test data sets" all simply provide numeric lab values for the variety of assay parameters the EHR must be able to replicate for certification.

The phrase "human-readable structured data," it should be apparent, goes beyond numeric values, and the standard does not address this.
[BTW: It should be noted that not all of the MU criteria involve capturing numerator data in the EHRs for subsequent reporting. e.g., §170.302(a) Drug-drug, drug-allergy interaction checks, §170.302(o) Access Control, §170.302(p) Emergency Access, §170.302(q) Automatic Log-off, §170.302(r) Audit Log, etc. Just to observe that, for the criteria that do involve numerator/denominator data, effective analysis and reporting assume uniform data input, as no one in clinical settings is likely going to have time or inclination to to any post-hoc "data cleaning."]



The final of five 2010 ONC regional meetings. I was struck by the large number of very young (to me) ONC staffers, an ebullient, brash cadre of ambitious MBAs and MPHs mostly, full of evangelistic energy and smarts (but most of them relatively short on direct operational HIT experience). I wanted to card some of these people during the evening open bar reception.

It was very well attended, and very busy; a packed agenda, the most contentious of which was the "Meaningful Use Deep Dive" session. A harbinger of the frequently inscrutable, vexing interpretations of the Meaningful Use criteria that continue to this day. At our REC we continue to hold bi-monthly Meaningful Use Study Group teleconference sessions to help us clarify issues as they emerge during interaction with our providers.

One fun activity during the Plenary Session: "Meaningful Use Jeopardy," LOL.

Your tax dollars at work. They did a good job with this, I have to say. The questions spanned the gamut, and were often very tough. My HealthInsight REC tied for first place.

One of the principal ONC operatives present was Thomas Tsang, M.D., M.P.H., Medical Director, Meaningful Use. I approached him during an intermission to ask why the RECs had not been made mandatory as part of the legislation (regarding which he'd been involved in drafting), lamenting that we were burning up a lot of FTE "marketing" our federally subsidized services, in lieu of focusing on actual technical assistance activities -- that it opened the administration up to the criticism of just wildly throwing money at the problem at cross-purposes.

He ever so politely blew me off, responding that "you are federal grantees, you are not part of the federal government."

Right. Well, my very same not-for-profit federal contractor company is also the NV-UT Medicare QIO, and our QA/beneficiary protection "statutory side," while equally "not [directly] part of the federal government" is nonetheless mandatory for providers who serve Medicare beneficiaries. You provide medical services to Medicare benes, you will submit to QIO statutory review purview, period.

I dropped the line of inquiry. He obviously didn't want to hear it, and it was moot by now in any event.


The MassPro QIO people led this session. It was very good, though much of the material struck me as "warmed-over DOQ-IT" (the now discontinued CMS HIT initiative within I which I worked during 2005-2007). My concern here is that both the ends and the scale of the Meaningful Use program require a significant re-thinking of the "adoption support" workflow M.O.

Notwithstanding, one has to give MassPro credit for sharing their considerable experience and expertise.


When I began this blog series, I noted that the Health IT Research Center, "whose purpose is to gather relevant information on effective practices from a wide variety of sources across the country and help the Regional Extension Centers (RECs) collaborate with one another and with relevant stakeholders to identify and share best practices in EHR adoption, effective use, and provider support" was not slated to be in place until FY 2012. Happily, though, I can report that this online resource is in fact now up and running, with REC staff nationwide contributing and sharing tools and ideas.

Not without bursts of silliness here and there, though. Some REC folks have had the earnest idea of doing "SWOT analysis" on the Meaningful Use criteria themselves. This is what ensues, I suppose, when MBAs go unsupervised. SWOT analysis is a tactical tool for assessing the competitive characteristics of organizations.

How about mere "pros and cons," folks? To me, this is somewhat methodologically akin to calculating Likert Scale data out to four decimal places (and then maybe even doing regression analyses on them, just because Excel makes it so easy).

Another bit of puzzlement. Someone enthusiastically posted to the HITRC regarding the nascent "ONC branding logo." I quickly grabbed the image and merged it with our corporate logo for a quick bit of Photoshop "co-branding" imagery (at the request of my Sup)...

Only to be subsequently informed that the ONC logo was not yet available for REC use, as it was still "undergoing the trademarking process."

A taxpayer-funded graphic. "Trademarked"?

Legalistic "ownership" and "use in commerce" hair-splitting aside, why not just slap the common law "TM" on it and put it to use? Awaiting granting of a "registered" ® symbol seems pointless, given the typical long timeline (not to mention the expense) of that process. A good number of RECs may well have folded their tents by
"® Day."

Whatever. Your tax dollars at work.


Well, can you say "underwhelming"?

They gave us 40 comps. Telling, right there, I suppose. Obviously, they hoped we'd spread them around to our REC client and prospect providers. A tough sell here in southern Nevada, asking marginally viable docs to give up a couple of days of billable visits to come and hear yet more rah-rah about the virtues of HIT.

When I arrived and looked in the attendee packet, there was a total of 88 people on the attendee list -- seven of which were us (from HealthInsight). The total actual body count was more like 60 or so. As I ambled about the expansive lower-level conference center of Green Vally Ranch, the quietude evoked scenes from "The Shining."

It was during this conference that I got my first taste of the overt vested-commercial-interest REC-bashing that has ensued. A presenter, who was also a presentation track chair (a private EHR consultant whom I will not name), launched into his session with an inappropriate slide deriding the "Socialist" federal HIT program, whereafter he went on to mischaracterize the RECs as all "choosing the EHRS" for their REC provider clients and all "driving doctors toward SaaS model platforms" (web-based EHRs). His presentation also inappropriately gave ample play to his own EHR evaluation website and newsletter and books.

I had to bite my tongue.


It picked up steam here, in an online article entitled "Five Reasons We Think RECs are RECkless," written by one Houston Neal, a person with no apparent experience in HIT (I emailed him to ask. He deflected with a vague non-answer, while at the same time having the amazing chutzpah to ask me if I'd have his derogatory article linked on our HealthInsight website).

His summary allegation:
In our opinion, there are five fundamental flaws with RECs:
  1. Doctor’s [sic] aren’t moving as fast as the money is flowing
  2. The market already delivers on what RECs promise
  3. “Preferred vendor lists” limit choice and free markets
  4. RECs won’t get doctors to “meaningful use” fast enough
  5. The REC model leads to under-staffed, ephemeral entities
Anyone who has troubled themselves to fully read my REC blog to date has to admit -- if honest -- that I am no Pollyana REC cheerleader, but, this this stuff is loaded down with dilettante and ulterior motive Straw Men. I cannot speak for every REC, but I bristle at the broad-brush assertion that we are all under- and otherwise inadequately staffed, that we don't deliver any value not already available via the commercial consultant market (that is patently absurd), and that we are all "limiting choice" via "preferred vendor lists" (HealthInsight is assiduously vendor neutral). Moreover, that the docs may not be moving as fast as the money and the MU deadlines is certainly not of our doing. We have play the hand we were dealt.

Finally, with respect to the "RECs can't compete with the private market for HIT talent" assertion, let me just observe that when I went through the HealthInsight re-hiring process, I was one of several hundred candidates.

Unhappily, this article got rather wide and mostly uncritical play (e.g., at The Health Care Blog and on one of my friend John Lynn's blogs). Like we need more sand in the gears (and, again, not to assert that we are above criticism).


Policy. Gets my juices flowing (my grad degree is in "Ethics and Policy Studies"). While I was not re-hired by HealthInsight for my policy chops (I'm a technical assistance project coordinator), it's nonetheless in my blood, and I was gratified to attend this event, which was all about a breadth of policy issues as they bear on the fiscally ailing state of Nevada. Our CEO Marc Bennett served on a discussion panel, and, to me, he was the most astute person on the podium that day (and I am not sucking up, either, we have very little contact, notwithstanding that I have known him since 1993).

One upshot of my attending was my being invited to present on Nov 18th statewide via live audience and teleconference to the Nevada Chapter of the Health Care Financial Management Association on the subject of health policy reform, about which I've written rather extensively, e.g.,
Should be fun. One question I will pose goes to the implications of this image from the recently-released 2010 NCQA HEDIS-based report.

Zero apparent linear "Pearson-R" relationship between cost and quality of outcomes (albeit via their summary "quality vs. cost" proxy scales). The foregoing plot addresses cardiovascular disease, but the scattergrams in the report illustrating other major chronic conditions (hypertension, diabetes, COPD) evince the very same random buckshot patterns. My red rectangle upper left quadrant annotation above is meant to focus on the question this begs of me: are we, consequently, paying too much? What is it about the "above average quality/below average cost" segment that we might discern were we able to "peel the onion back" in search of relevant causal correlates? Is use of HIT a factor? And/or other organizational factors? ( The NCQA report is silent on such implications).

We have seen a similar kind of thing before, e.g.,

Evidence a decade ago of a weak inverse relationship (just a tad this side of purely random, unlike the NCQA/HEDIS data) between per capita Medicare expenditures and "quality ranking." Again, of interest to me analytically would be the lower cost/higher quality outcomes cohort.

I am again reminded of the words of Dr. Atul Gawande last year in "The Cost Conundrum" -
"Dramatic improvements and savings will take at least a decade. But a choice must be made. Whom do we want in charge of managing the full complexity of medical care? We can turn to insurers (whether public or private), which have proved repeatedly that they can’t do it. Or we can turn to the local medical communities, which have proved that they can. But we have to choose someone—because, in much of the country, no one is in charge. And the result is the most wasteful and the least sustainable health-care system in the world...

...In the sharpest economic downturn that our health system has faced in half a century, many people in medicine don’t see why they should do the hard work of organizing themselves in ways that reduce waste and improve quality if it means sacrificing revenue."


More to come...