NQMC - Expert Resources - Expert Commentary

Expert Commentary

Perspective

Why Are There No Efficiency Measures in the National Quality Measures Clearinghouse?

By: Peter S. Hussey, PhD
Elizabeth A. McGlynn, PhD

Concerns about rising health care costs have created tremendous pressure to measure the efficiency of health care providers and systems. Large employers, health plans, state governments, and regional coalitions are attempting to develop methods to measure and reward efficiency, integrated with their quality measurement and improvement initiatives. However, the development of health care efficiency measures has lagged far behind the development of health care quality measures. There are currently 1,762 quality measures in the National Quality Measures Clearinghouse but no efficiency measures. Why are there no efficiency measures in the Clearinghouse, despite widespread concerns about costs?

One challenge has been a lack of agreement over what constitutes an efficiency measure. A simple definition of "efficiency" is the number of inputs (or costs) needed to produce a certain output. More specifically, then, the challenge lies in establishing an agreement on what constitutes a valid comparison of inputs and outputs. To some, a valid efficiency measure must compare the production of outputs of a specified level of quality (1). This is the definition currently used by national groups including the National Quality Forum (NQF) and the Ambulatory Quality Alliance (AQA). Most measures in use are not efficiency measures by this definition because they do not explicitly incorporate a measurement of the quality of the product. These measures are often labeled "resource use" or "cost of care" measures. In this commentary, we use the term "efficiency" broadly to describe all measures comparing health care inputs and outputs, recognizing that most specific measures would likely not meet the stricter definition of efficiency used by national groups.

Many Measures Labeled as Efficiency Measures

An Agency for Healthcare Research and Quality (AHRQ)-funded review by the Southern California Evidence-Based Practice Center identified 273 measures that could be considered efficiency measures (2). Most of these measures were published in the peer-reviewed literature in studies of hospital efficiency. Some of the measures use simple ratios, such as the length of stay per discharge or the cost per discharge, adjusted for case-mix. Many use econometric and mathematical techniques developed in the field of economics, for example Stochastic Frontier Analysis and Data Envelopment Analysis, to measure the efficiency of hospitals. The measures reflect how well hospitals use resources, including physician and nurse labor, supplies, and facilities, to produce services like inpatient discharges and outpatient visits. These efficiency measures were typically developed by academic researchers and used in studies that addressed questions such as the relative efficiency of non-profit vs. for-profit hospitals.

The private sector has developed a different set of efficiency measures. These are the measures in most widespread use. They measure the efficiency with which physicians provide services to a patient for a condition or a period of time. The most commonly used measurement tools are the proprietary episode groupers, Episode Treatment Groups (ETGs) and Medical Episode Groups (MEGs), which calculate the total cost of resources used to treat a patient for an "episode of care" -- care related to a single condition or acute medical event. The measurement tools work by grouping together insurance claims for related individual services into discrete episodes of care. Episodes are then attributed to one or more physicians, and the costs of care attributed to a physician are compared with the average costs of similar care delivered by physicians in the same specialty.

The Largely Unknown State of Efficiency Measure Performance

Despite the fact that many health care efficiency measures are in use, very little is still known about how well they perform. In this way, the state of the art in efficiency measures contrasts sharply with that of the quality measures in the National Quality Measures Clearinghouse. No efficiency measures have been endorsed by the NQF. Unlike the evolution of most quality measures, current efficiency measures are not typically derived from practice standards from the research literature, professional medical associations, or expert panels. Efficiency measures have not been subjected to rigorous evaluations of their reliability and validity. Measurement scientists would prefer that steps be taken to improve these metrics in the laboratory before implementing them in operational uses. Purchasers and health plans, however, are willing to use measures without such testing under the belief that the measures will improve with use.

Several methodological questions that are important to establishing credible efficiency measurement remain. The most important questions include:

Quality: The NQF and other national groups that endorse measures have defined "efficiency" as the resource used to produce health care at a specified level of quality (1). However, most of the measures in use are not efficiency measures by this definition because they do not explicitly incorporate a measurement of the quality of the product (3). Methods for doing so are not well developed at this time.
Reliability: Reliability is an analysis of whether the variation seen in efficiency is due to measurement error or true differences in performance. The reliability of various resource use measures is largely unknown. The sample size of observations required to produce stable resource use estimates is uncertain. Health plans currently use largely arbitrary cut-offs, such as 30 episodes per physician.
Attribution: A key issue for resource measures when care is provided by multiple physicians is how to attribute accountability for the resources used. Various algorithms, mainly based on visit counts and payment amounts, have been used and assignments made to either one or multiple physicians. Different algorithms lead to different results.
Risk adjustment: Variation in efficiency measures may be driven by differences in patient risk factors that lead to high utilization and costs. While several methods for adjusting for these risk factors have been developed for certain uses, limited testing has been done in efficiency measurement applications, particularly using episodes of care.

The Development of Efficiency Measures by National Groups

There are several national groups that have been working to address these issues. The Centers for Medicare and Medicaid Services (CMS) is eveloping a method for reporting physician resource use using episode-based measures (4). CMS has been evaluating two proprietary episode groupers, ETGs and MEGs. It has also funded a study developing alternative approaches to the proprietary groupers, and is considering funding the development of new groupers for use in Medicare. Other groups, notably the Quality Alliance Steering Committee (QASC), are also developing public domain episode-based efficiency measures that will serve as alternatives to the proprietary tools (5). Finally, private-sector groups including the Leapfrog Group (6) and the National Committee on Quality Assurance (7) have developed measures for measuring hospital and health plan efficiency, respectively. All of these efforts are works in progress. Much more work will be required before the state of the art in health care efficiency measurement begins to approach that of quality measurement. Once there is greater stakeholder agreement about the definition and other properties of efficiency measures, we will likely see measures added to the National Quality Measures Clearinghouse.

Author

Peter S. Hussey, PhD
RAND Health, Arlington, Virginia

Elizabeth A. McGlynn, PhD
RAND Health, Santa Monica, California

Disclaimer

The views and opinions expressed are those of the author and do not necessarily state or reflect those of the National Quality Measures Clearinghouse™ (NQMC), the Agency for Healthcare Research and Quality (AHRQ), or its contractor ECRI Institute.

Potential Conflicts of Interest

Dr. Hussey has business/professional affiliations with RAND and the University of California, Davis -- Center for Health Management Research.

Dr. McGlynn reports that she is Secretary, Board of Directors -- Academy Health; Vice Chair, Board, Providence - Little Company of Mary Medical Centers; and Board Member -- American Board of Internal Medicine Foundation.

Drs. Hussey and McGlynn state no personal financial interests, family member conflicts of interest, or additional disclosures.

References

Adams K, Burstin H. (National Quality Forum). Voting draft report: measurement framework: evaluating efficiency across patient-focused episodes of care. 2009 Jan 12. 60 p.
McGlynn EA. Identifying, categorizing, and calculating health care efficiency measures. Final Report Final Report (prepared by the Southern California Evidence-based Practice Center-RAND Corporation, under Contract No. 282-00-0005-21). [AHRQ Publication No. 08-0030]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2008 Apr. 214 p. Also available: http://www.ahrq.gov/qual/efficiency/efficiency.pdf.
Hussey PS, de Vries H, Romley J, Wang M, Chen S, Shekelle P, McGlynn EA. A systematic review of health care efficiency measures. Health Services Research. Forthcoming.
Centers for Medicare & Medicaid Services (CMS). Medicare resource use measurement plan. Baltimore (MD): Centers for Medicare & Medicaid Services (CMS); 9 p. Also available: http://www.cms.hhs.gov/QualityInitiativesGenInfo/downloads/ResourceUse_Roadmap_OEA_1-15_508.pdf.
Quality Alliance Steering Committee (QASC). [Web site]. Washington (DC): Quality Alliance Steering Committee (QASC); [accessed 2009 Jul 10]. [various p]. Available: http://www.healthqualityalliance.org.
Leapfrog commentary. Health Services Research. Forthcoming.
National Committee on Quality Assurance. HEDIS 2008. Available at: http://www.ncqa.org/tabid/536/Default.aspx. Last accessed 3/2/09.

Expert Commentary

Search

Expert Resources

Browse

Compare