Interoperable Electronic Health Records and Quality of Care

Background

Improving health care quality depends on the ability to measure it. As Lord Kelvin, the British scientist, observed more than 100 years ago, "If you cannot measure it, you cannot improve it."(1) Indeed, many of today's health care quality metrics were chosen because they fall within the subset of quality that can be measured from existing medical records.(2) Historically, quality metrics have been selected assuming they would be measured from paper-based medical records and electronic administrative claims, with the inherent limitations of those systems. Paper-based medical records are time-consuming to review, and electronic administrative claims data lack clinical detail.

These limitations may soon change. The American Recovery and Reinvestment Act of 2009 (ARRA) could create an unprecedented opportunity to redefine goals for improving health care quality by identifying new quality metrics in advance of implementing the systems that will report them. In this commentary, we reflect on this opportunity and what it means for both quality measurement and the design of electronic systems.

ARRA allocates approximately $19 billion to physicians and hospitals for meaningful use of electronic health records that support the electronic exchange of data.(3) Electronic health records (EHRs) enable management of a patient's clinical information over time and at the point of care, aided by tools such as evidence-based decision support.(4) Electronic exchange promotes sharing of clinical information across providers in different health care settings.(5) EHRs that have the capacity for electronic health information exchange are called "interoperable" EHRs.(5)

Isolating the Effect of Interoperability on Quality

We recently published an Agency for Healthcare Research and Quality (AHRQ)-funded conceptual model and proposed metric set for isolating the effect of electronic interoperability on quality.(6) That is, we described how and when interoperability is expected to change medical decision making, and we identified metrics that could potentially capture those changes. We assumed the perspective of a board-certified, community-based primary care physician who already uses an EHR. We then asked which quality metrics could capture the potential added benefit of electronically connecting that physician's EHR to the electronic systems of hospitals, specialists, laboratories, pharmacies and other sources of clinical data. We sought metrics of this kind that could also be reported electronically, meaning that an automated process is used to collect and aggregate information across patients and providers, thereby generating the metrics.

An example of a metric sensitive to interoperability is the percentage of patients hospitalized with acute myocardial infarction who receive persistent beta-blocker treatment. In order for a physician to improve performance on this metric, he or she might benefit from having information (such as diagnosis, allergy, and medication data) from other clinical sources (such as hospitals, other physicians, and pharmacies or pharmacy benefit managers). In contrast, a metric that would be less sensitive to interoperability is the percentage of patients with acute pharyngitis who receive prescriptions for antibiotics. Performance on that metric depends largely on how physicians respond to clinical data collected from the patient at the time of the visit rather than on data from external sources.

There are a variety of ways to use metric sets of this kind as enumerated below:

• First, to evaluate the effects of interoperable EHRs on quality
• Second, to compare different electronic strategies for delivering clinical data to physicians, as the optimal strategy is not yet known
• Third, to isolate the effects of interoperable EHRs on quality in communities where other concurrent initiatives are taking place, such as implementation of the patient-centered medical home model for primary care practice
• Finally, to shape vendor products' capabilities for electronic reporting

With these purposes in mind, we identified more than 1000 individual existing ambulatory care quality metrics from a literature review, including from such sources as the National Quality Forum and the National Committee for Quality Assurance. With input from a 36-member national expert panel, we identified the 18 metrics that were most likely to be both sensitive to the effects of interoperability and suitable for electronic reporting. Examples of these metrics included the percentage of patients hospitalized with acute myocardial infarction who receive persistent beta-blocker treatment and the percentage of patients with diabetes who had at least one hemoglobin A1c measured in the reporting period below 7%. We also developed 14 new metrics to address additional areas of quality expected to be affected by interoperability. Examples of these included the percentage of tests which represented repeat tests for patients within test-specific time periods and the percentage of patients seen by their primary care physicians within 14 days of hospital discharge.

Lessons Learned

Several lessons learned in developing this metric set are highly relevant to the current national discussion on quality and electronic health records.

1. Data Representation. Information can be entered into EHRs in either free text or structured fields. A free-text field gives providers the freedom to type anything, including sentences, phrases, numbers, abbreviations and even misspellings. Free-text fields are often used for progress notes. By contrast, structured fields only accept very limited types of information; they include check-boxes and drop-down menus using controlled vocabularies. Different EHRs use different structured fields and ways of displaying those fields with their corresponding response choices. Quality reporting activities are most likely to focus on those variables captured in structured fields. Thus, the method of data capture plays an important role in shaping what is possible to measure in an electronic environment.
   
   
2. EHR Usability and Workflow. Although structured fields can be used to generate electronic reports, they are unlikely to add value unless health care providers consistently use them. Creating easy-to-use computer interfaces, restructuring clinical workflow, and having comprehensive training, configuration and ongoing technical support are all likely to affect the accuracy of electronically reported metrics.
   
   
3. Community Integration. The accuracy of electronic quality reporting for measuring interoperability will depend not only on the capacity of an EHR to receive external clinical data but also on the prevalence and completeness of community-wide data exchange. That is, because metrics designed to capture the effects of interoperability frequently require multiple types of data (such as diagnosis data plus medication or laboratory data), a well-integrated community will be able to report metrics more accurately than a less-well-integrated community. Currently, few communities are highly integrated electronically. This is especially true among communities that have fragmented health care financing and delivery, or the majority of communities in the U.S.
   
   
4. Vendor Maturity and Priorities. There is still a large gap between what one would like to report electronically and what is currently possible. For example, out of the many vendors we considered, none were able to electronically report all of the 18 metrics that our expert panel selected. Most vendors, in fact, were unable to electronically report any of the metrics without a substantial amount of expensive and time-consuming custom programming. This was true despite the fact that the 18 metrics derive from widely publicized and well-established data sets previously endorsed by national organizations, such as the National Quality Forum. Electronic reporting on the new metrics will present even more of a challenge.
   
   
5. Quality Metric Specifications. Enabling electronic reporting is partly a programming issue but also involves developing specifications (definitions for numerators, denominators, and exclusion criteria for each metric) that are applicable to EHRs, drawing on clinical data beyond the billing codes used in claims-based reporting. EHR-based reporting can yield more complete identification of patients with disease (that is, patients in the denominator), compared to claims, particularly when data from problem lists, medication lists, and laboratory values are included in the definition of disease.(7) How numerators, denominators, and exclusion criteria are defined in an electronic environment will affect not only the accuracy of electronic reporting but will also influence the results. For example, if EHR-based reporting identifies more patients with diabetes than claims-based reporting but both methods identify the same number of patients with diabetes who received HgbA1c testing, the EHR-based reporting will yield an overall proportion of patients tested that is lower than that generated from claims-based reporting. Thus, EHR-based reporting overall may yield results indicating higher or lower quality than what the medical community previously assumed.

National Context

The federal government released proposed definitions of "meaningful use" under ARRA.(8, 9) This definition consists of performance metrics for both providers and hospitals and includes: 1) systems-based process measures, related to the degree of adoption of health information technology (IT) and the degree of electronic integration for health information exchange, such as "use of computerized physician order entry for all orders" and "generate and transmit permissible prescriptions electronically," and 2) clinical process measures, such as "percent (%) of patients over 50 with annual colorectal cancer screenings." The current draft specifies all metrics for the year 2011 and some for the years 2013 and 2015, with others for 2013 and 2015 to be determined, in part, in collaboration with the National Quality Forum.

The meaningful use metrics are likely to be embraced widely as implementation and quality improvement targets. As such, they have the potential to substantially shape vendor products, provider behavior, and community-level health information exchange. Some but not all of the meaningful use metrics aim to capture the expected effects of interoperability.

The goal of reporting many clinical measures by 2011 is ambitious for most communities, and it will be important to see how this unfolds. Anecdotal reports suggest that adoption of EHRs has paradoxically slowed since the announcement of ARRA incentives, as health care providers that were on the cusp of adopting EHRs stopped to wait for the final meaningful use metrics, in order to make their own assessment of the feasibility and value of EHR adoption in the present environment.

The "bar" or threshold of meaningful use that will earn incentives appears to have been set very high, as currently only 4% of physicians and 1.5% of hospitals have fully functioning EHRs.(10, 11) If the bar was set too high, many providers may try and fail to reach it, which may adversely affect their attitudes toward future efforts at changing health care practice. Or, they may simply choose not to participate in the incentive program, thwarting the goal of EHR adoption. On the other hand, if the bar had been set too low, a lack of momentum would hinder improvement of the vendor products and community alliances that are essential for interoperability.

Summary

National efforts are underway to promote the meaningful use of interoperable EHRs. Achieving meaningful use will require integrating clinical information in unprecedented ways. This represents a unique opportunity to improve health care today, in part by changing the way quality is defined and measured.

Authors

Lisa M. Kern, MD, MPH
Weill Cornell Medical College, New York, NY

Rainu Kaushal, MD, MPH
Weill Cornell Medical College, New York, NY

Disclaimer

The views and opinions expressed are those of the author and do not necessarily state or reflect those of the National Quality Measures Clearinghouse™ (NQMC), the Agency for Healthcare Research and Quality (AHRQ), or its contractor, ECRI Institute.

Potential Conflicts of Interest

Drs. Kern and Kaushal declared no potential conflicts of interest with respect to this expert commentary.

References

1. Lord Kelvin: "If you cannot measure it, you cannot improve it." (Accessed May 19, 2010, at http://quotationsbook.com/quote/46180/ .)
2. Bates DW. The approaching revolution in quality measurement. Jt Comm J Qual Patient Saf 2009;35:358.
3. American Recovery and Reinvestment Act of 2009. Pub L, No.111-5, 123 Stat 115.
4. Healthcare Information Management and Systems Society. HIMSS Electronic Health Record Definition Model. 2003. (Accessed May 19, 2010, at http://www.himss.org/content/files/ehrattributes070703.pdf .) (PDF Help)
5. Blumenthal D, Glaser JP. Information technology comes to medicine. N Engl J Med 2007;356:2527-34.
6. Kern LM, Dhopeshwarkar R, Barron Y, Wilcox A, Pincus H, Kaushal R. Measuring the effects of health information technology on quality of care: a novel set of proposed metrics for electronic quality reporting. Jt Comm J Qual Patient Saf 2009;35:359-69.
7. Tang PC, Ralston M, Arrigotti MF, Qureshi L, Graham J. Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: implications for performance measures. J Am Med Inform Assoc 2007;14:10-5.
8. Department of Health and Human Services, Centers for Medicare and Medicaid. Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. 75 Federal Register 1844 (2010) (42 CFR Parts 412, 413, 422 and 495).
9. Department of Health and Human Services, Office of the National Coordinator for Health Information Technology. Health information technology: Initial set of standards, implementation specifications, and certification criteria for electronic health record technology; interim final rule. 75 Federal Register 2013 (2010) (45 CFR Part 170).
10. DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care--a national survey of physicians. N Engl J Med 2008;359:50-60.
11. Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in U.S. hospitals. N Engl J Med 2009;360:1628-38.

PDF Help: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader® .