menu-iconMore mobile-close-icon
Skip Navigation
Skip Navigation
PrintDownload PDFGet Adobe ReaderDownload to WordDownload as HTMLDownload as XMLCitation Manager
Save to Favorites
Measure Summary
This measure is endorsed by the National Quality Forum
Title
Behavioral health care patients' experiences: percentage of adult patients who rated how much improvement they perceived in themselves.
Source(s)
CAHPS® surveys and tools to advance patient-centered care [https://www.cahps.ahrq.gov/default.asp]. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [updated 2008 Feb 21]; [accessed 2004 Apr 15]. [3 p].

ECHO® Survey and Reporting Kit 2004. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2004. Various p.

Measure Domain

Primary Measure Domain
Clinical Quality Measures: Patient Experience
Secondary Measure Domain
Clinical Quality Measure: Outcome

Brief Abstract

Description

This composite measure indicates the percentage of adult patients who rated how much improvement they perceived in themselves ("Much better," "A little better," "About the same," "A little worse," or "Much worse"). The "Perceived Improvement" composite measure is based on four questions on the Experience of Care and Health Outcomes (ECHO) Survey.

Note: A composite score is calculated in which a higher score indicates better quality. Composite scores are intended for consumer-level reporting. Additionally, frequency distributions are available for plans or providers to use for quality improvement purposes.

Rationale

Consumer ratings and reports of their behavioral health care and the health plan or managed behavioral health care organization (MBHO) that provide that care are important measures of the quality of behavioral health treatment. National efforts to develop quality indicators for behavioral health care by treatment providers, researchers and policymakers have all included consumer ratings. Moreover, many behavioral health care treatment providers use consumer ratings as a component of their quality improvement processes.

The Experience of Care and Health Outcomes (ECHO) Survey is designed to collect information on patients' experiences with behavioral health care. It produces data that can be useful for patients, clinicians, managed behavioral healthcare organizations (MBHO), health care plans, purchasers, States, and Federal agencies. This ECHO Survey is part of the CAHPS family of surveys, which are developed and supported by a public-private consortium of researchers sponsored by the Agency for Healthcare Research and Quality (AHRQ).

The ECHO Survey can be used for the following purposes:

  • To satisfy external standards.
  • To improve the quality of mental health and substance abuse services.
  • To monitor the quality of behavioral health organizations.
  • To hold providers accountable through public reporting.
Evidence for Rationale
ECHO Development Team. Shaul JA, Eisen SV, Clarridge BR, Stringfellow VL, Fowler FJ Jr, Cleary PD. Experience of care and health outcomes (ECHO) survey. Field test report: survey evaluation. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2001 May 8. 100 p. [13 references]

Hall LL, Flynn LM. NAMI'S managed care report card. National Alliance for the Mentally Ill. Eval Rev. 1997 Jun;21(3):352-6. [9 references] PubMed External Web Site Policy

Hermann RC, Regner JL, Erickson P, Yang D. Developing a quality management system for behavioral health care: the Cambridge Health Alliance Experience. Harv Rev Psychiatry. 2000 Nov;8(5):251-60. PubMed External Web Site Policy

Ross EC. Managed behavioral health care premises, accountable systems of care, and AMBHA'S perms. American Managed Behavioral Healthcare Association. Eval Rev. 1997 Jun;21(3):318-21. PubMed External Web Site Policy

Shaul JA, Eisen SV, Stringfellow VL, Clarridge BR, Hermann RC, Nelson D, Anderson E, Kubrin AI, Leff HS, Cleary PD. Use of consumer ratings for quality improvement in behavioral health insurance plans. Jt Comm J Qual Improv. 2001 Apr;27(4):216-29. PubMed External Web Site Policy

Shaul JA, Eisen SV, Stringfellow VL, et al. Methodological considerations for collecting health plan and treatment ratings about behavioral health care. Submitted Health Services Research. 2001.

Teague GB, Ganju V, Hornik JA, Johnson JR, McKinney J. The MHSIP mental health report card. A consumer-oriented approach to monitoring the quality of mental health plans. Mental Health Statistics Improvement Program. Eval Rev. 1997 Jun;21(3):330-41. [34 references] PubMed External Web Site Policy
Primary Health Components

Behavioral health care; patients' experiences; perceived improvement

Denominator Description

Health plan members age 18 years and older, enrolled in a managed care organization (MCO) or managed behavioral health organization (MBHO) for the previous 12 months with no more than one break in enrollment of up to 45 days during the enrollment period, who received behavioral health care services and who answered the "Perceived Improvement" questions on the Experience of Care and Health Outcomes (ECHO) Survey

Numerator Description

The number of health plan members from the denominator who indicated "Much better," "A little better," "About the same," "A little worse," or "Much worse" on the four questions regarding their perceived improvement

Evidence Supporting the Measure

Type of Evidence Supporting the Criterion of Quality for the Measure
  • A formal consensus procedure, involving experts in relevant clinical, methodological, public health and organizational sciences
Additional Information Supporting Need for the Measure

Unspecified

Extent of Measure Testing

The ECHO Survey 3.0 is the product of nearly six years of research and testing, as well as extensive consultations with behavioral health care experts and consumers.

The ECHO Development Team produced a draft instrument based on the Consumer Assessment of Behavioral Health Services (CABHS) instrument and the Mental Health Statistics Improvement Program (MHSIP) survey. The draft was reviewed by multiple stakeholder groups; it was also evaluated by survey experts, administrators, mental health providers, and consumers.

The team conducted literacy testing of the draft instrument as well as cognitive testing with individuals who received behavioral health care services. Recommended revisions to the survey based on this testing were reviewed with the ECHO Development Team to produce the draft of the survey that was field tested. The current version of the survey tested at a reading level between 7th and 8th grade.

The team then undertook a field test study to further evaluate the instrument and to refine it for use as a standardized measure of behavioral health care.

Evidence for Extent of Measure Testing
Daniels AS, Shaul JA, Greenberg P, Cleary PD. The Experience of Care and Health Outcomes Survey (ECHO): a consumer survey to collect ratings of treatment, outcomes and plans. In: Maruish ME, editor(s). The use of psychological testing for treatment planning and outcome assessment. 3rd ed. Mahwah (NJ): Lawrence Erlbaum Associates; 2004.

ECHO Development Team. Shaul JA, Eisen SV, Clarridge BR, Stringfellow VL, Fowler FJ Jr, Cleary PD. Experience of care and health outcomes (ECHO) survey. Field test report: survey evaluation. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2001 May 8. 100 p. [13 references]

State of Use of the Measure

State of Use
Current routine use
Current Use
Accreditation
Internal quality improvement

Application of the Measure in its Current Use

Measurement Setting
Behavioral Health Care
Managed Care Plans
Substance Use Treatment Programs/Centers
Professionals Involved in Delivery of Health Services
Does not apply to this measure (e.g., measure is not provider specific)
Least Aggregated Level of Services Delivery Addressed
Individual Clinicians or Public Health Professionals
Statement of Acceptable Minimum Sample Size
Specified
Target Population Age

Age greater than or equal to 18 years

Target Population Gender
Either male or female

National Strategy for Quality Improvement in Health Care

National Quality Strategy Aim
Better Care
National Quality Strategy Priority
Person- and Family-centered Care
Prevention and Treatment of Leading Causes of Mortality

Institute of Medicine (IOM) National Health Care Quality Report Categories

IOM Care Need
Getting Better
Living with Illness
IOM Domain
Effectiveness
Patient-centeredness

Data Collection for the Measure

Case Finding Period

12 months

Denominator Sampling Frame
Enrollees or beneficiaries
Denominator (Index) Event or Characteristic
Encounter
Patient/Individual (Consumer) Characteristic
Therapeutic Intervention
Denominator Time Window
Time window brackets index event
Denominator Inclusions/Exclusions

Inclusions
Health plan members age 18 years and older, enrolled in a managed care organization (MCO) or managed behavioral health organization (MBHO) for the previous 12 months with no more than one break in enrollment of up to 45 days during the enrollment period, who

  • received ambulatory or outpatient and day/night behavioral health care services during the evaluation period including outpatient visits or treatment sessions, medications, partial treatment, or day or night treatment, and
  • answered the "Perceived Improvement" questions on the Experience of Care and Health Outcomes (ECHO) Survey.

Include refusals, non-response, and bad addresses/phone numbers.

Refer to the original measure documentation for further details including administrative codes developed to identify patients who have received services.

Exclusions

  • Deceased
  • Ineligible (not enrolled in the plan)
Exclusions/Exceptions
Does not apply to this measure
Numerator Inclusions/Exclusions

Inclusions
The number of health plan members from the denominator who indicated "Much better," "A little better," "About the same," "A little worse," or "Much worse" on the four questions regarding their perceived improvement

Exclusions
Unspecified

Numerator Search Strategy
Fixed time period or point in time
Data Source
Administrative clinical data
Patient/Individual survey
Type of Health State
Functional Status
Instruments Used and/or Associated with the Measure

ECHO® Survey 3.0

Computation of the Measure

Measure Specifies Disaggregation
Does not apply to this measure
Scoring
Composite/Scale
Interpretation of Score
Desired value is a higher score
Allowance for Patient or Population Factors
Case-mix adjustment
Description of Allowance for Patient or Population Factors

CAHPS recommends adjusting the data for respondent age, education, and general health status.

If the sample size is sufficient, responses may be analyzed for specific sub-populations, such as respondents with chronic conditions.

Standard of Comparison
External comparison at a point in, or interval of, time
Internal time comparison

Identifying Information

Original Title

Perceived improvement.

Measure Collection Name
Submitter
Agency for Healthcare Research and Quality - Federal Government Agency [U.S.]
Developer
Agency for Healthcare Research and Quality - Federal Government Agency [U.S.]
CAHPS Consortium - Health Care Quality Collaboration
Harvard Medical School - Academic Institution
Funding Source(s)

The Evaluation Center at the Human Services Research Institute, Cambridge, Massachusetts, (Cooperative Agreement U97 SM52004 with The Center for Mental Health Services, NIMH), the Agency for Healthcare Research and Quality (AHRQ) (Cooperative Agreement HS-0925), and the MacArthur Foundation

Composition of the Group that Developed the Measure

Harvard Medical School

Financial Disclosures/Other Potential Conflicts of Interest

None

Endorser
National Quality Forum
NQF Number
0008
Date of Endorsement

2012 Sep 17

Adaptation

This measure was adapted from the Consumer Assessment of Behavioral Health Services (CABHS) and the Mental Health Statistics Improvement Program (MHSIP).

Date of Most Current Version in NQMC
2004 Aug
Measure Maintenance

Unspecified

Date of Next Anticipated Revision

Unspecified

Measure Status

This is the current release of the measure.

The measure developer reaffirmed the currency of this measure in July 2014.

Source(s)
CAHPS® surveys and tools to advance patient-centered care [https://www.cahps.ahrq.gov/default.asp]. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [updated 2008 Feb 21]; [accessed 2004 Apr 15]. [3 p].

ECHO® Survey and Reporting Kit 2004. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2004. Various p.
Measure Availability

The individual measure, "Perceived Improvement," is published in the "ECHO Survey and Reporting Kit 2004."

For more information, contact CAHPS Technical Assistance.

Companion Documents

The following is available:

  • ECHO - the CAHPS behavioral health survey. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2004 [cited 2004 Apr 19].

For more information, contact CAHPS Technical Assistance.

NQMC Status

This NQMC summary was completed by ECRI on December 13, 2004. The information was verified by the measure developer on February 11, 2005. This NQMC summary was retrofitted into the new template on April 22, 2011. The information was reaffirmed by the measure developer on July 10, 2014.

Copyright Statement

No copyright restrictions apply.

Disclaimer

NQMC Disclaimer

The National Quality Measures Clearinghouse™ (NQMC) does not develop, produce, approve, or endorse the measures represented on this site.

Read full disclaimer...