NQMC - Resources - Measure Use

Using Measures

The National Quality Measures Clearinghouse™ (NQMC), sponsored by the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services, provides information on how to use a quality measure. Before learning about the various uses of a quality measure, it is important to first define a quality measure and to discuss its desirable attributes.

What is a Quality Measure?

In order to define a quality measure, it is important to define quality of care. NQMC relies on the Institute of Medicine's (IOMs) definition of quality of care as "the degree to which health care services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge." ¹ A quality measure is a mechanism that enables the user to quantify the quality of a selected aspect of care by comparing it to a criterion. A subtype of a quality measure is a clinical performance measure. Specifically, a clinical performance measure is a mechanism for assessing the degree to which a provider competently and safely delivers clinical services that are appropriate for the patient in the optimal time period. ^{2, 3}

Each domain of measurement in NQMC (i.e., Access, Outcome, Patient Experience, Process, and Structure) offers a different insight into health care quality. Provided below are descriptions of the different types of information that are captured by each domain of measurement.

Access - an access measure assesses the patient's attainment of timely and appropriate health care. Barriers to access may include inability to pay for health care, difficulty traveling to health care facilities, unavailability of health care facilities, lack of a "medical home," cultural and health beliefs that prevent recognition of the need for and benefits of health care, and disparities in responding to persons seeking health care.

Outcome - an outcome of care is a health state of a patient resulting from health care. An outcome measure can be used to assess quality of care to the extent that health care services influence the likelihood of desired health outcomes. Outcome-based measures of quality reflect the cumulative impact of multiple processes of care. Outcome measures may suggest specific areas of care that may require quality improvement, but further investigation is typically necessary to determine the specific structures or processes that should be changed. ^{4, 5}

Patient Experience - a patient experience measure aggregates reports of patients about their observations of and participation in health care. These measures provide the patient perspective on quality of care.

Process - a process measure assesses a health care service provided to, or on behalf of, a patient. Process measures are often used to assess adherence to recommendations for clinical practice based on evidence or consensus. To a greater extent than outcome measures, process measures can identify specific areas of care that may require improvement. ^{4, 5}

Structure - a structure measure is a feature of a health care organization or clinician relevant to its capacity to provide health care. Structure data describe the capability of organizations or professionals rather than care provided to, or results achieved for, specific patients or groups of patients. For example, nurse/patient ratio is a structure-based measure because it does not describe care given to specific patients or specific groups of patients.

Desirable Measure Attributes

NQMC conducted a comprehensive review of existing frameworks of desirable attributes of quality measures from national and international organizations committed to measuring and improving quality. Frameworks were reviewed from the National Committee on Quality Assurance (NCQA), the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), Foundation for Accountability (FACCT), the Institute of Medicine (IOM), the U.S. Department of Health & Human Services (DHHS), the Performance Measures Coordinating Council, the Physician Consortium, Australia's National Health Performance Committee, United Kingdom's National Health System, and the German Agency for Quality in Medicine. Common conceptual areas were identified in reviewing these frameworks and were used to inform the Inclusion Criteria and the Template of Measure Attributes for NQMC.

The desirable attributes of a measure can be grouped according to three key broad conceptual areas within which narrower categories provide more detail. These three areas are (1) the importance of a measure, (2) the scientific soundness of a measure, and (3) the feasibility of a measure.

Importance of the Measure

Relevance to stakeholders - the topic area of the measure is of significant interest, and financially and strategically important to stakeholders (e.g., businesses, clinicians, patients).

Health importance - the aspect of health the measure addresses is clinically important as defined by high prevalence or incidence, and a significant effect on the burden of illness (i.e., effect on the mortality and morbidity of a population).

Applicable to measuring the equitable distribution of health care - the measure can be stratified, or analyzed by subgroup to examine whether disparities in care exist among a population of patients.

Potential for improvement - there is evidence indicating that there is overall poor quality or variations of quality among organizations indicating a need for the measure.

Susceptibility to being influenced by the health care system - the results of the measure can be operationalized into actions or interventions that are under the control of the user, leading to improvements that are known to be feasible.

Scientific Soundness: Clinical Logic

Explicitness of evidence - the evidence supporting the measure is explicitly stated.

Strength of evidence - the topic area of the measure is strongly supported by the evidence (i.e., indicated to be of great importance for improving quality of care).

Scientific Soundness: Measure Properties

Reliability - the results of the measure should be reproducible and reflect results of action when implemented over time; reliability testing should be documented.

Validity - the measure is associated with what it purports to measure; validity testing should be documented.

Allowance for patient/consumer factors as required - the measure allows for stratification or case-mix adjustment.

Comprehensible - the results of the measure should be understandable for the user who will be acting on the data.

Feasibility

Explicit specification of numerator and denominator - a measure should have explicit and detailed specifications for the numerator and denominator; statements of the requirements for data collection should be understandable and implementable.

Data availability - the data source that is needed to implement the measure should be available, accessible, and timely. The burden of measurement should also be considered, where the costs of abstracting and collecting data are justified by the potential for improvement in care.

These important characteristics of a measure are captured in NQMC's Template of Measure Attributes and represented by the Complete Summary.

How are Quality Measures Used?

Quality measures are used for three general purposes: quality improvement, accountability, and research. Descriptions and examples of each of these uses of a measure are provided below. ^{6, 7, 8}

Quality Improvement

Quality measures can be used for both quality improvement within an institution or system of care (internal quality improvement) or across institutions or systems of care (external quality improvement).

Using measures for internal quality improvement involves three basic steps: identifying problems or opportunities for improvement, selecting appropriate measures and using them to obtain a baseline assessment of current practices, and using them to reassess or monitor the effect of improvement efforts on measure performance. Baseline quality measure results can be used to better understand a quality problem, provide motivation for change, and establish a basis for comparison across institutional units or over time. Baseline results also enable the user to prioritize areas for quality improvement. Results from repeated measurements of clinical performance can be used by internal quality improvement programs to assess whether performance has changed after improvement efforts have been implemented. ^{6, 7}

Quality measures may be used for external quality improvement in programs operated by state, regional, or national entities; other quality improvement organizations; or professional organizations. These organizations may coordinate cycles of clinical performance measurement and reporting of comparative performance to stimulate health care institutions to undertake internal quality improvement efforts. The usual audiences for results of external quality improvement are the participating institutions or providers of care within institutions. External agencies frequently collect the performance measurement data, verify their accuracy, and report quality performance results among providers of care in a format that allows direct comparison of providers. External agencies may also provide "benchmark" results that can be used to encourage providers to strive to perform at the best level shown to be achievable. ^{6, 8}

Accountability

Uses of quality measures for the purpose of accountability include purchaser and/or consumer decision-making, accreditation and external quality oversight. Although the use of quality measures for accountability may be quite similar to their use for external quality improvement, and the same set of organizations may conduct measurement for both purposes, the requirements for validity and reliability are higher when using measures for accountability. Greater validity and reliability demand that each provider collect data in the exact same way using standardized and detailed specifications. This ensures that comparisons are fair or that predefined measure performance has indeed been achieved.

The usual audiences for accountability data are entities other than those that provide care such as purchasers of health care, payers, or patients. Their primary interest is in using accountability data to guide the selection of providers or set financial rewards to providers for performance. They will use results to compare provider groups, select providers based on performance levels in priority areas of clinical practice and consumer service, or establish financial rewards. Some providers supply report cards displaying clinical performance measurements for consumers to make decisions based on quality.^{6, 7}

Although purchasers or consumers may be the key audience for accountability measures, health care providers may nevertheless use such measures to implement internal quality improvement programs. In some instances, accreditating organizations expect providers to use accountability measures in this way. For example, "accreditation is offered by the Joint Commission on Accreditation of Health Care Organizations (JCAHO) for hospitals, ambulatory surgery centers, long-term care facilities, home-care programs, and health care networks and by the National Committee on Quality Assurance (NCQA) for managed care organizations. Both programs require an internal quality improvement component within each accredited organization, and both have developed sets of clinical performance measures to be used for internal quality improvement." ^{6, 7, 8}

Research

The primary use of quality measures in research is to develop or produce new knowledge about the health care system that is generalizable to a wide range of settings and useful in setting health policy. Quality-of-care research is often conducted to evaluate programs and assess the impact of policy changes on health care quality. Compared with their use for other purposes, the use of quality measures for research purposes may require larger sample sizes, longer time horizons, more detailed data collection, the merging of multiple sources of data, and more complex analyses. However, quality measures applied for other purposes are becoming increasingly useful in a research context. ^{9, 10, 11, 12}

Quality measures may be used in multiple ways. The current uses of each quality measure, as indicated by the submitting organization, are captured in the "Current Use" field of the Complete Measure Summary. NQMC has divided the broader categories of measure uses (i.e., Quality Improvement, Accountability, and Research) into more detailed and specific categories. Examples of "Current Use" include accreditation, internal quality improvement, decision-making, external quality oversight, quality of care reporting, and research.

Send your questions or comments regarding measure uses to NQMC at info@qualitymeasures.ahrq.gov.

¹ Institute of Medicine, Committee to Design a Strategy for Quality Review and Assurance in Medicine. Lohr KN, editors. Medicare: a strategy for quality assurance / Committee to Design a Strategy for Quality Review and Assurance in Medicare, Division of Health Care Services, Institute of Medicine. Vol. 1, Report of a study. Washington (DC): National Academy Press; 1990 May. 468 p.
² Center for Health Policy Studies, Harvard School of Public Health, Center for Quality of Care Research and Education. Understanding and choosing clinical performance measures for quality improvement: development of typology: final report. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 1995 Jan 31. Various p.
³ Lawthers AW, Palmer H. In search of a few good performance measures. In: Seltzer J, Nash DB, editors. Models for measuring quality in managed care: analysis and impact. New York (NY): Faulkner & Gray's Healthcare Information Center; 1997. p. 121-150. (Medical outcomes & practice guidelines library II).
⁴ Rubin HR, Pronovost P, Diette GB. The advantages and disadvantages of process-based measures of health care quality. Int J Qual Health Care 2001 Dec 1;13(6):469-74.
⁵ Mant J. Process versus outcome indicators in the assessment of quality health care. Int J Qual Health Care 2001 Dec;13(6):475-80.
⁶ Palmer RH. Measuring clinical performance to provide information for quality improvement. Qual Manag Health Care 1996;4(2):1-6.
⁷ Solberg LI, Mosser G, McDonald S. The three faces of performance measurement: improvement, accountability, and research. Jt Comm J Qual Improv 1997 Mar;23(3):135-47.
⁸ Agency for Healthcare Research and Quality (AHRQ). CONQUEST: overview: a computerized needs-oriented quality measurement evaluation system. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 1999 Mar [cited 2001 May 10]. [17 p].
⁹ Ellerbeck EF, Jencks SF, Radford MJ, et al. Quality of care for Medicare patients with acute myocardial infarction. A four-state pilot study from the Cooperative Cardiovascular Project. JAMA. May 17 1995;273(19):1509-1514.
¹⁰ O'Connor GT, Quinton HB, Traven ND, et al. Geographic variation in the treatment of acute myocardial infarction: the Cooperative Cardiovascular Project. JAMA. Feb 17 1999;281(7):627-633.
¹¹ Schneider E, Zaslavsky A, Epstein A. Racial disparities in the quality of care for enrollees in Medicare managed care. JAMA. 2002;287(10):1288-1294.
¹² Zaslavsky AM, Hochheimer JN, Schneider EC, et al. Impact of sociodemographic case mix on the HEDIS measures of health plan quality. Med Care. 2000;38(10):981-992.