NQMC - Resources - Measure Selection

Selecting Measures

The National Quality Measures Clearinghouse™ (NQMC), sponsored by the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services, provides information on how to select, apply, and interpret a quality measure.

The NQMC Web site provides the capability of searching the database of quality measures based on various attributes found in the Template of Measure Attributes. NQMC also provides a feature that allows users to compare two or more measures in a side-by-side comparison. The attributes displayed in the comparison feature (see the Measure Comparison) were selected to address the measure selection considerations discussed below.

Measure Selection

There are many issues to consider when selecting a quality measure. The first step in selecting measures is to identify the measurement purpose and intended use of the measure. Is it to improve quality, or is it for accountability or research? Or perhaps it is a combination of these reasons (see Using Measures for more details on the uses of a measure). It is crucial to determine your assessment and improvement priorities in order to select measures that will serve those priorities. In doing so, it is also important to consider and balance the different perspectives and expectations among different stakeholders. For example, a health plan administrator may have to consider priorities of the businesses that purchase the health plan for their employees along with the priorities of the physicians who are members of their network. Businesses may be more concerned with the value of health care services whereas physicians may be more concerned with outcomes of care for their patients. The challenge is in balancing these varying expectations and perspectives. A starting point is to make clear what the varying stakeholders "value and regard as an essential mission of health care. Areas of agreement among these perspectives ought to define the central focus for quality measurement." ^{1, 2, 3}

Once the goal or intended use of the measure has been determined, the following areas should be considered in selecting an appropriate measure for the desired purpose(s): ^{4, 5}

Does the measure possess the desirable attributes of a measure as outlined by the conceptual areas of importance, scientific soundness, and feasibility of a measure? (Refer to NQMC's Desirable Measure Attributes section of Using Measures and the Complete Summary for more details.)
What data sources are available? What is the possibility and expense of collecting additional data? The three most common data sources that are used are administrative databases (often referred to as "claims" data), medical records, and survey data. Administrative data is usually readily available in electronic form and relatively inexpensive to use whereas medical records data, though richer in detail, are expensive to obtain and require personnel and time-intensive data abstraction methods. ⁵
Does the measure apply to the desired setting of care and to the providers that give care that you wish to assess? ⁵
Is the measure selected from the appropriate domain of measurement that will produce the type of data appropriate for use? For example, an organization wishing to focus on the perceptions of patients should use Patient Experience measures since the information is collected directly from the patient. Data from these measures will provide insight into patients' satisfaction and perception of the care they receive. Likewise, an organization wishing to collect data that identifies processes that may affect patient care, and are in need of improvement, should select measures that assess processes of care (i.e., Process measures). ⁵
Have considerations been made for comparisons? When selecting a quality measure, it is important to determine an appropriate comparison in order to make reliable assessments of quality. For example, an organization that wishes to assess their performance in relation to a prescriptive standard, such as Healthy People 2010, should select measures for which prescriptive standards of comparison are available. Some comparisons, such as national benchmarks, may require the additional consideration of risk or case-mix adjustment of factors that contribute to differences in results but are not related to quality of care. ² Alternatively, comparisons can be made by stratifying results to examine potential disparities in care among different subgroups.

Applying Quality Measures

Quality measurement can be used to drive performance improvement. The quality improvement process is often iterative and therefore measurement may be repeated over time. The first measurement can be used to identify problems and to establish baseline results. Subsequent measurements allow an evaluation of the impact of quality improvement efforts and make it possible to monitor and sustain improvement. ^{2, 6, 7}

Once users identify appropriate quality measures for implementation, complete measure specifications should be obtained from the submitting organization. The measure summaries provided in NQMC are intended to provide an abstraction of the important attributes of a measure incorporating the information provided by the submitting organization(s). NQMC identifies contact information for requesting measure documentation in the "MEASURE AVAILABILITY" field. When possible, information regarding electronic (including hypertext links to full-text) and print copies is provided.

Interpreting Quality Measures

Quality measures often require summing data about the health care given to many patients and expressing the results as a rate, ratio, frequency distribution, or score for average performance. ^{4, 8} The measure result is often composed of a number and unit of measure. The number provides the magnitude, and the unit provides a context for interpreting the number. It is difficult to interpret the result of a quality measure as good or poor unless there is a standard of comparison by which it can be compared. The different types of comparisons are: external comparison to similar providers at a single point in time, external comparison to similar providers over time, internal comparison over time (comparing scores before and after quality improvement efforts), and prescriptive standard (e.g., goals set by Healthy People 2010). Well-established standards of care exist for some areas of health care treatment and services. In these cases, it is possible to conclude that a quality problem does or does not exist. Where there is no standard of care, results can be meaningful if compared to set goals such as those set by the organization implementing the measure or to national goals such as Healthy People 2010. ^{5, 9}

If you have any questions or comments, e-mail NQMC at info@qualitymeasures.ahrq.gov.

¹ McGlynn EA. Six challenges in measuring the quality of health care. Health Aff (Millwood) 1997 May-Jun;16(3):7-21.
² Hermann RC, Leff HS, Lagodmos G. Selecting process measures for quality improvement in mental healthcare. Cambridge (MA): Human Services Research Institute (HSRI); 2002 Jul 1. 58 p. Also available: http://www.hsri.org/docs/pn47.PDF.
³ Hermann RC, Palmer RH. Common ground: a framework for selecting core quality measures for mental health and substance abuse care. Psychiatr Serv 2002 Mar;53(3):281-7.
⁴ Lawthers AW, Palmer H. In search of a few good performance measures. In: Seltzer J, Nash DB, editors. Models for measuring quality in managed care: analysis and impact. New York (NY): Faulkner & Gray's Healthcare Information Center; 1997. p. 121-150. (Medical outcomes & practice guidelines library II).
⁵ Palmer RH, Miller MR. Methodologic challenges in developing and implementing measures of quality for child health care. Ambul Pediatr 2001;1(1):39-52.
⁶ Palmer RH. Using clinical performance measures to drive quality improvement. Total Qual Manage 1997;8(5):305-11.
⁷ Palmer RH, Peterson LE. Development and testing of performance measures for quality improvement in clinical preventive services. Am J Prev Med 1995;11(6):402-6.
⁸ Center for Health Policy Studies, Harvard School of Public Health, Center for Quality of Care Research and Education. Understanding and choosing clinical performance measures for quality improvement: development of typology: final report. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 1995 Jan 31. Various p.
⁹ Agency for Healthcare Research and Quality (AHRQ). Child health toolbox: measuring performance in child health programs. Understanding performance measurement. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2001 Mar [cited 2001 May 09]. [10 p]. Available: http://www.ahrq.gov/chtoolbx/understn.htm.

Selecting Measures

Measure Selection

Applying Quality Measures

Interpreting Quality Measures

Search

Resources

Browse

Compare