Yet another federal public comment period announcement

“Workflow Assessment for Health IT Toolkit Evaluation.”

Proposed Project
AHRQ is a lead Federal agency in developing and disseminating evidence and evidence-based tools on how health IT can improve health care quality, safety, efficiency, and effectiveness. Understanding clinical work practices and how they will be affected by practice innovations such as implementing health IT has become a central focus of health IT research. While much of the attention of health IT research and development had been directed at the technical issues of building and deploying health IT systems, there is growing consensus that deployment of health IT has often had disappointing results, and while technical challenges remain, there is a need for greater attention to sociotechnical issues and the problems of modeling workflow.

The implementation of health IT in practice is costly in time and effort and less is known about these issues in small- and medium-sized practices where the impact of improved or disrupted workflows may have especially significant consequences because of limited resources. Practices would derive great benefit from effective tools for assessing workflow during many types of health IT implementation, such as creating disease registries, collecting quality measures, using patient portals, or implementing a new electronic health record system. To that end, in 2008, AHRQ funded the development of the Workflow Assessment for Health IT toolkit (Workflow toolkit). Through this toolkit, end users should obtain a better understanding of the impact of health IT on workflow in ambulatory care for each of the following stages of health IT implementation: (1) Determining system requirements, (2) selecting a vendor, (3) preparing for implementation, or (4) using the system post implementation. They should also be able to effectively utilize the publicly available workflow tools and methods before, during, and after health IT implementation while recognizing commonly encountered issues in health IT implementation. In the current project AFIRQ is conducting an evaluation to ensure that the newly developed Workflow toolkit is useful to small- and medium-sized ambulatory care clinic managers, clinicians, and staff...

...The evaluation will be conducted with 18 practices affiliated with one of two Practice-based Research Networks (PBRNs) in Oregon and Wisconsin, and with the Health IT Regional Extension Centers (RECs) in those States. Participants will be recruited who agree to use the Workflow toolkit in their specific health IT project for a minimum of 10 weeks. This will provide an opportunity to observe use of the Workflow toolkit amongst its intended end users, who are best positioned to provide critical feedback to improve the functionality of the Workflow toolkit...

...The estimated total cost to the Federal Government for this project is $793,456 over a 27-month period from September 23, 2011 to December 22, 2013.
__

Request for comments
In accordance with the Paperwork Reduction Act, comments on AHRQ's information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ healthcare research and healthcare information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ's estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology.

Click the workflow graphic above for the link to the HIT Workflow Toolkit.
Written comments should be submitted to:
Doris Lefkowitz, Reports Clearance Officer, AHRQ,
by email at doris.lefkowitz@AHRQ.hhs.gov.

Comment period closes May 8th, 2012

___

Well, needless to say, I found that interesting, given that I do this very kind of work for my REC, e.g.,


Click either snippet above for the link to the full jpeg for each one. The first one comes from a doc who wasted a bunch of my time (and his), only to decide he wanted to ditch his EHR contract and go back into Vendor Selection, right on the heels of our being accorded sandbox access to the product so that we could begin to map specific workflows, particularly those relating to Meaningful Use compliance.

That had to have been several grand of FTE in the commode.

The latter goes to a project now ramping up full-steam as I write this. The OMs get it, but sustaining staff buy-in / "ownership," well, we shall see. We gave them my WKFL spiel (.pps) in their HQ conference room a few weeks back, and have commenced with a paper charts-to-EHR procedure now being fleshed out. Ambitious project, 9 IM docs, 3 sites, all the docs and their MAs have variations in the way they see patients and chart the encounters (BTW, my .xls workflow data capture tool is posted here, at HealthInsight).
___

DIVERSION: THIS JUST IN


Recall my March 2nd post citation of Dr. Novella's video course on "Your Deceptive Mind: a Scientific Guide to Critical Thinking Skills"? (I continue to review it; more to come)

How about The Khan Academy?

Yeah. This drives me back down Into The Weeds':

Abstracting knowledge from the problem-solving activities to which it should relate has damaging effects at many levels. One effect is that formal education becomes focused unduly on personal intellect. Cultivating a narrow range of intellectual skills and temperaments that foster academic success, the culture of education takes interpersonal skills for granted, and devalues manual skills. And it fails to instill the intellectual behaviors and perspectives that foster effective problem-solving (see part V).


In particular, traditional schooling fails to instill high standards of achievement. In most schooling, time is the constant and achievement the variable—precisely the opposite of what true education demands. Students are allotted a fixed amount of time to learn and then permitted to pass exams and courses with a B or C or worse. Inevitably, given the widely varying abilities and inclinations of individual students, not many will have the experience of passing courses at a high level of achievement. This tolerance of lesser achievement is especially harmful when learning is cumulative, that is, when success at one level requires understanding of the material from an earlier level.


Failure to enforce high standards of quality arises from the premise that schooling should instill a fixed core of knowledge, as distinguished from a core of behavior. Teaching high standards of intellectual behavior and teaching a fixed core of knowledge are mutually exclusive when a fixed time is allotted to achieving these goals. One of the three variables (time spent, amount covered and degree of mastery) has to be held constant at a high level, and that constant should be the degree of mastery. Students would differ in the amount they master and the speed with which they do so, but not in the degree of mastery they attain.


Mastery of a core of knowledge should not be the goal of education— especially in an era when knowledge is constantly becoming obsolete and when information technology confers rapid access to more knowledge than anyone can learn and more processing power than anyone’s mind possesses. The goal should be mastering the behaviors involved in applying knowledge to solve problems effectively and efficiently.


Because these principles are ignored, many students pass through 12 or more years of schooling without ever experiencing mastery, while constantly undergoing invidious comparisons to the best students. This system is harmful even for those best students, for they may acquire elitist attitudes, superficial understanding and misplaced confidence that their academic proficiency will translate into effective problem solving. For less successful students, schooling is too often experienced as a caste system rather than a vehicle for personal development. Many students emerge from their schooling with their natural abilities undeveloped and their natural optimism defeated. To fight back, some adopt an attitude of disdain towards education and intellect. These reactions to formal education are like the reaction of dying canaries in a coal mine—highly sensitive indicators of toxic conditions. In varying degrees, many students are left without the capacities and confidence that only achievement can confer and without the expanded horizons that only education can provide.
[Medicine in Denial, pp 229-230]

UPDATE:
INTERESTING SHOOT-OUT AT THE ONC / HIT PRESS CORRAL
 Dr. Mostashari is not amused:

Apparently, doctors who order a lot of imaging tests are more likely to have electronic systems that let them view those images in their offices.

That is the finding of an article appearing in the March 2012 issue of Health Affairs by McCormick and colleagues. This is not a particularly surprising observation. What is surprising is that the authors go far beyond the scope of their research to conclude that “the federal government’s ongoing, multibillion-dollar effort to promote the adoption of health information technology may not yield anticipated cost savings from reductions in duplicative diagnostic testing. Indeed, it is possible that computerization will drive costs in this area up, not down.”...

To which the study authors publicly replied on The Health Care Blog:

Our recent Health Affairs article linking increased test ordering to electronic access to results has elicited heated responses, including a blog post by Farzad Mostashari, National Coordinator for Health IT. Some of the assertions in his blog post are mistaken. Some take us to task for claims we never made, or for studying only some of the myriad issues relevant to medical computing. And many reflect wishful thinking regarding health IT; an acceptance of deeply flawed evidence of its benefit, and skepticism about solid data that leads to unwelcome conclusions...

Place your bets, pass the popcorn...

McCormick, Bor, Woolhandler, andHillemstein conclude in rebuttal of Farzad:

Dr. Mostashari is also correct in reiterating that randomized trials are the best way to assess health IT. In fact, no randomized trial has ever been published that examines patients’ outcomes or costs associated with off-the-shelf health IT systems that dominate the U.S. market. No drug or new medical device could pass FDA review based on such thin evidence as we have on health IT. Yet his agency is disbursing $19 billion in federal funds to stimulate the adoption of this inadequately evaluated technology. Dr. Mostashari is perhaps the only person in our nation who commands the resources needed to mount a well done randomized controlled trial to fairly assess the impact of health IT, and the comparative efficacy of the various EHR options.


Finally, Dr. Mostashari’s unbridled faith in technology is mirrored by his belief that ACOs are the next panacea for health costs and quality. That health policy flavor-of-the-month also remains wholly unproven.

OK, putting aside for the moment the partisan cheap-shot questions begged by their final paragraph:

• ...no randomized trial has ever been published that examines patients’ outcomes or costs associated with off-the-shelf health IT systems that dominate the U.S. market.

Precisely how would you design and execute such a study, particularly at a time when HIT markets are ramping up at warp speed? Which "dominant" off-the-shelf systems would get included (based on what cut-off %'s of current ambulatory and inpatient market shares)?

As I write this (3/13) there are a total of 1,817 "Certified" EHR ambulatory and inpatient systems and modules (including 913 "complete" systems). Who gets "studied," who gets left out? Would these be "Dummy Clinic" mock trials, or operational CER?

What would be the "control" (i.e., stratified representative samples of equivalent paper practices and institutions)?

• No drug or new medical device could pass FDA review based on such thin evidence as we have on health IT.

Are you asserting that we simply don't know? (Rumsfeld's "unknown unknowns"?) Or that we have "evidence," (like your own reported "null" findings) but it is either equivocal or too sparse to lean on comfortably? Moreover, there are plenty of HIT press reports indicating that the FDA wants no part of this potentially radioactive oversight. They have a full plate already. Apropos, see my November 2010 blog post "First, do no Hold Harmless'."

• Dr. Mostashari is perhaps the only person in our nation who commands the resources needed to mount a well done randomized controlled trial to fairly assess the impact of health IT, and the comparative efficacy of the various EHR options.

Methinks you overstate his brief (e.g., see ARRA Title XIII, Subtitle A, Section 3001 et seq). ONC is a very small shop, as federal Beltway entities go. While you could construe Sec. 3001(b)(2)-(10) as requiring HIT "RCTs," it would seem a stretch to me.
 
But, more on the ONC mandate shortly.

Not that I disagree that we need "comparative effectiveness research" broadly -- though, it will be difficult to parse the relative contributory effects of physical workflow, HIT "information" flow, and clinical decision-making (again, see "Medicine in Denial").

• Finally, Dr. Mostashari’s unbridled faith in technology is mirrored by his belief that ACOs are the next panacea for health costs and quality. That health policy flavor-of-the-month also remains wholly unproven.

I find this to be an unhelpful ad hominem low blow. Congress, not Farzard, wrote the authorizing ACO subtitle in PPACA. Whether it turns out to be successful is yet to be determined. I've written about ACOs before. So, yeah, it's as yet "wholly unproven." Help me out here. Are we doing science, or Creation Science? Do the critics have any concrete, plausible counter-proposals? Or, is the paper chart FFS paradigm the terminus of health care progress?

Asked and Answered. You want to help improve HIT effectiveness? 

Change.The.Payment.Paradigm.
 
Not that we need not work simultaneously on HIT "usability." My record as a certifiable Crank on that issue is by now well-established.

I'm not the only one who took issue with the study's final paragraph. From the Health Affairs blog itself:

jgogek Says:
March 12th, 2012 at 6:48 pm
While in general I’m glad to see researchers vigorously defend their research, I’m left wondering about the gratuitous slam against ACOs in the last paragraph of the above blog post. Calling ACOs “flavor-of-the-month” and “wholly unproven” is simply pejorative and has nothing to do with the substance of the research in dispute. It tarnishes the authors’ objectivity. In fact, it makes them seem kind of chippy.

Indeed. One more commenter weighed in, at ModernHealthcare.com

Al Puerini
ModernHealthcare.com, Mar 15. 2012 7:07pm

I completely agree with Dr Mostashari. The only "deeply flawed evidence" is what they used to come to their study conclusions. To say that EHR vendor's "products have undergone mostly modest tweaks" since 2008 is incredibly naive and shortsighted. EHR's have changed dramatically in the last 4 years and many still have much to do to achieve meaningful use. If one cannot see the positve and deep changes in EHR's over the past 4 years as a result of the development of the PCMH and achieving Meaningful Use, then they are not paying attention to what is happening in the real world. And more to come! When hospitals, PCP's and specialists start communicating with each other through their EHR's, the savings will be huge. And that movement is well on its way. Keep up the good fight, Farzad. Your leadership is changing health IT in a strongly positive way.

Al Puerini, MD
President, RI Primary Care Physician Corporation 

___

DAVID DRANOVE WEIGHS IN

From Code Red:

...The other item that caught my attention was a study in Health Affairs showing that physicians who have electronic access to test results (via electronic medical records) order more tests than physicians who do not. The New York Times ran both an article and an editorial about the study. The finding is based on a single year of cross-section data. All first year graduate students in the social sciences should immediately recognize that this study suffers from endogeneity bias. The “treatment” (i.e., access to test results through EMR) is potentially correlated with unobservable physician characteristics that end up in the error term of the regression. As a result, causality is in doubt.

Let me offer several candidate explanations for the findings. First, it could be that access to test results encourages doctors to find ways to increase testing (the negative connotation offered by the author and the New York Times.) Second, it could be that some doctors have a technological bent. They tend to order lots of tests and they are first to have access to EMR. Thus, there is a correlation between testing and EMR but the latter does not cause the former. Third, some doctors may order an above average number of tests. Forced by managed care and Medicare to reduce costs, they seek a way to get testing under control. They install EMR which gives them access to prior test results. This allows them to reduce testing , though not by enough to put them below the average. Again, there is a correlation between testing and EMR, but in this case EMR leads to less testing. There may be other possibilities. My point is not that I know what causes what. My point is that no one knows what causes what.

Fortunately, there are research methods that can help sort out causality. I happen to be working on a similar study using these methods. In an upcoming blog, I will share some preliminary findings. My coauthors and I get a similar correlation, but reveal an interesting pattern of causality that may provide hope to those who believe that EMR can rescue the healthcare system.

AND, THE HITS JUST KEEP ON COMIN'

 

ERRATUM

Apropos of "imaging," I ran across this recent paper published by ACS. I snipped out this graphic and annotated it (click to enlarge)
 

To reduce future projected cancers from diagnostic procedures, we advocate the widespread use of evidence- based appropriateness criteria for decisions about imaging procedures; oversight of equipment to deliver reliably the minimum radiation required to attain clinical objectives; development of electronic lifetime records of imaging procedures for patients and their physicians; and commitment by medical training programs, professional societies, and radiation protection organizations to educate all stakeholders in reducing radiation from diagnostic procedures. CA Cancer J Clin 2012;62:75-100. Published 2012 American Cancer Society.

A better than 550% increase in per capita ionizing dose exposure relative to a constant natural sources denominator (2.4 mSv). Interesting. I started my white collar career in 1986 doing environmental dose/exposure related stuff in a forensic-level Rad/Mixed Waste lab in Oak Ridge, under the esteemed John A. Auxier, PhD and his stable of top-tier Health Physicists, so I found these data intriguing.
___


NEW COMMONWEALTH FUND REPORT

Executive Summary
People in the United States, regardless of where they live, deserve the same opportunities to lead long, healthy, and productive lives. Achieving that goal means that all communities should receive the very best from their local health care systems. Yet this new Scorecard on Local Health System Performance finds that where one lives has a major impact on the ability to access health care and the quality of care received. Comparing the 306 local health care areas, known as hospital referral regions, in the United States, the report finds wide variations on key indicators of health system performance. Access to care, quality of care, costs, and health outcomes all vary significantly from one local community to another, both within larger states and across states. There is often a two- to threefold variation on key indicators between leading and lagging communities. The 30 top-performing local areas include communities in the Northeast, Midwest, and a few West Coast communities—with these leading areas often doing well on multiple indicators and dimensions of care. Yet while pockets of excellence exist, there are ample opportunities for health system improvement in all communities, even among the leaders...

98 pages of incisive stuff. Below, this is pretty interesting.

As is this:

Can you say "Gawande"? You could spend a month just interpreting this stuff. Couple of words and phrases come straight to mind: "Disparities," "Health Impact Assessments."
___
More to come...