Skip to main content
  • Measure Summary
  • NQMC:000846
  • Aug 2004
  • NQF-Endorsed Measure

Behavioral health care patients' satisfaction: adult patients' overall rating of their health plan for counseling or treatment.

CAHPS® surveys and tools to advance patient-centered care [https://www.cahps.ahrq.gov/default.asp]. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [updated 2008 Feb 21]; [accessed 2004 Apr 15]. [3 p].
ECHO® Survey and Reporting Kit 2004. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2004. Various p.

This is the current release of the measure.

The measure developer reaffirmed the currency of this measure in May 2016.

Primary Measure Domain

Clinical Quality Measures: Patient Experience

Secondary Measure Domain

Does not apply to this measure

Description

This measure indicates adult patients' perceptions of their health plan for counseling or treatment. Patients rate the health plan on a scale from 0 to 10, where 0 is the worst plan possible and 10 is the best plan possible. Responses are grouped by rating: percentages are reported for ratings of 9 and 10, 7 and 8, and 6 and lower.

Note: The percentage of ratings for the group 9 and 10 (or sometimes 8, 9, and 10) is used for consumer-level reporting; higher percentages indicate better quality. Additionally, frequency distributions encompassing all groups of ratings (i.e., 9 and 10, 7 and 8, 6 or lower) are available for plans or providers to use for quality improvement purposes.

Rationale

Consumer ratings and reports of their behavioral health care and the health plan or managed behavioral health care organization (MBHO) that provide that care are important measures of the quality of behavioral health treatment. National efforts to develop quality indicators for behavioral health care by treatment providers, researchers and policymakers have all included consumer ratings. Moreover, many behavioral health care treatment providers use consumer ratings as a component of their quality improvement processes.

The Experience of Care and Health Outcomes (ECHO) Survey is designed to collect information on patients' experiences with behavioral health care. It produces data that can be useful for patients, clinicians, managed behavioral healthcare organizations (MBHO), health care plans, purchasers, States, and Federal agencies. This ECHO Survey is part of the CAHPS family of surveys, which are developed and supported by a public-private consortium of researchers sponsored by the Agency for Healthcare Research and Quality (AHRQ).

The ECHO Survey can be used for the following purposes:

  • To satisfy external standards.
  • To improve the quality of mental health and substance abuse services.
  • To monitor the quality of behavioral health organizations.
  • To hold providers accountable through public reporting.

Evidence for Rationale

ECHO Development Team. Shaul JA, Eisen SV, Clarridge BR, Stringfellow VL, Fowler FJ Jr, Cleary PD. Experience of care and health outcomes (ECHO) survey. Field test report: survey evaluation. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2001 May 8. 100 p. [13 references]

Hall LL, Flynn LM. NAMI'S managed care report card. National Alliance for the Mentally Ill. Eval Rev. 1997 Jun;21(3):352-6. [9 references] PubMed External Web Site Policy

Hermann RC, Regner JL, Erickson P, Yang D. Developing a quality management system for behavioral health care: the Cambridge Health Alliance Experience. Harv Rev Psychiatry. 2000 Nov;8(5):251-60. PubMed External Web Site Policy

Ross EC. Managed behavioral health care premises, accountable systems of care, and AMBHA'S perms. American Managed Behavioral Healthcare Association. Eval Rev. 1997 Jun;21(3):318-21. PubMed External Web Site Policy

Shaul JA, Eisen SV, Stringfellow VL, Clarridge BR, Hermann RC, Nelson D, Anderson E, Kubrin AI, Leff HS, Cleary PD. Use of consumer ratings for quality improvement in behavioral health insurance plans. Jt Comm J Qual Improv. 2001 Apr;27(4):216-29. PubMed External Web Site Policy

Shaul JA, Eisen SV, Stringfellow VL, et al. Methodological considerations for collecting health plan and treatment ratings about behavioral health care. Submitted Health Services Research. 2001.

Teague GB, Ganju V, Hornik JA, Johnson JR, McKinney J. The MHSIP mental health report card. A consumer-oriented approach to monitoring the quality of mental health plans. Mental Health Statistics Improvement Program. Eval Rev. 1997 Jun;21(3):330-41. [34 references] PubMed External Web Site Policy

Primary Health Components

Behavioral health care; patients' experiences; rating of the health plan for counseling or treatment

Denominator Description

Health plan members age 18 years and older, enrolled in a managed care organization (MCO) for the previous 12 months with no more than one break in enrollment of up to 45 days during the enrollment period, who received behavioral health care services and who answered the "Overall Rating of the Health Plan" question on the Experience of Care and Health Outcomes (ECHO) Survey

Numerator Description

To report ratings of 9 and 10:
The number of health plan members from the denominator who rated their health plan for counseling or treatment a 9 or 10

To report ratings of 7 and 8:
The number of health plan members from the denominator who rated their health plan for counseling or treatment a 7 or 8

To report ratings of 6 or lower:
The number of health plan members from the denominator who rated their health plan for counseling or treatment a 6 or lower

Type of Evidence Supporting the Criterion of Quality for the Measure

  • A formal consensus procedure, involving experts in relevant clinical, methodological, public health and organizational sciences

Additional Information Supporting Need for the Measure

Unspecified

Extent of Measure Testing

The ECHO Survey 3.0 is the product of nearly six years of research and testing, as well as extensive consultations with behavioral health care experts and consumers.

The ECHO Development Team produced a draft instrument based on the Consumer Assessment of Behavioral Health Services (CABHS) instrument and the Mental Health Statistics Improvement Program (MHSIP) survey. The draft was reviewed by multiple stakeholder groups; it was also evaluated by survey experts, administrators, mental health providers, and consumers.

The team conducted literacy testing of the draft instrument as well as cognitive testing with individuals who received behavioral health care services. Recommended revisions to the survey based on this testing were reviewed with the ECHO Development Team to produce the draft of the survey that was field tested. The current version of the survey tested at a reading level between 7th and 8th grade.

The team then undertook a field test study to further evaluate the instrument and to refine it for use as a standardized measure of behavioral health care.

Evidence for Extent of Measure Testing

Daniels AS, Shaul JA, Greenberg P, Cleary PD. The Experience of Care and Health Outcomes Survey (ECHO): a consumer survey to collect ratings of treatment, outcomes and plans. In: Maruish ME, editor(s). The use of psychological testing for treatment planning and outcome assessment. 3rd ed. Mahwah (NJ): Lawrence Erlbaum Associates; 2004.

ECHO Development Team. Shaul JA, Eisen SV, Clarridge BR, Stringfellow VL, Fowler FJ Jr, Cleary PD. Experience of care and health outcomes (ECHO) survey. Field test report: survey evaluation. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2001 May 8. 100 p. [13 references]

State of Use

Current routine use

Current Use

Accreditation

Internal quality improvement

Measurement Setting

Behavioral Health Care

Managed Care Plans

Substance Use Treatment Programs/Centers

Professionals Involved in Delivery of Health Services

Does not apply to this measure (e.g., measure is not provider specific)

Least Aggregated Level of Services Delivery Addressed

Individual Clinicians or Public Health Professionals

Statement of Acceptable Minimum Sample Size

Specified

Target Population Age

Age greater than or equal to 18 years

Target Population Gender

Either male or female

National Quality Strategy Aim

Better Care

National Quality Strategy Priority

Person- and Family-centered Care

IOM Care Need

Getting Better

Living with Illness

IOM Domain

Patient-centeredness

Case Finding Period

12 months

Denominator Sampling Frame

Enrollees or beneficiaries

Denominator (Index) Event or Characteristic

Encounter

Patient/Individual (Consumer) Characteristic

Therapeutic Intervention

Denominator Time Window

Time window brackets index event

Denominator Inclusions/Exclusions

Inclusions
Health plan members age 18 years and older, enrolled in a managed care organization (MCO) for the previous 12 months with no more than one break in enrollment of up to 45 days during the enrollment period, who

  • received ambulatory or outpatient and day/night behavioral health care services during the evaluation period including outpatient visits or treatment sessions, medications, partial treatment, or day or night treatment, and
  • answered the "Overall Rating of the Health Plan" question on the Experience of Care and Health Outcomes (ECHO) Survey.

Include refusals, non-response, and bad addresses/phone numbers.

Refer to the original measure documentation for further details including administrative codes developed to identify patients who have received services.

Exclusions

  • Deceased
  • Ineligible (not enrolled in the plan)

Exclusions/Exceptions

Does not apply to this measure

Numerator Inclusions/Exclusions

Inclusions
To report ratings of 9 and 10:
The number of health plan members from the denominator who rated their health plan for counseling or treatment a 9 or 10

To report ratings of 7 and 8:
The number of health plan members from the denominator who rated their health plan for counseling or treatment a 7 or 8

To report ratings of 6 or lower:
The number of health plan members from the denominator who rated their health plan for counseling or treatment a 6 or lower

Exclusions
Unspecified

Numerator Search Strategy

Fixed time period or point in time

Data Source

Administrative clinical data

Patient/Individual survey

Type of Health State

Does not apply to this measure

Instruments Used and/or Associated with the Measure

ECHO® Survey 3.0

Measure Specifies Disaggregation

Does not apply to this measure

Scoring

Composite/Scale

Interpretation of Score

Desired value is a higher score

Allowance for Patient or Population Factors

Case-mix adjustment

Description of Allowance for Patient or Population Factors

CAHPS recommends adjusting the data for respondent age, education, and general health status.

If the sample size is sufficient, responses may be analyzed for specific sub-populations, such as respondents with chronic conditions.

Standard of Comparison

External comparison at a point in, or interval of, time

Internal time comparison

Original Title

Overall rating of the health plan for counseling or treatment.

Measure Collection Name

ECHO® Survey 3.0

Measure Set Name

ECHO® Survey 3.0 Adult Questionnaire

Submitter

Agency for Healthcare Research and Quality

Developer

Agency for Healthcare Research and Quality

CAHPS Consortium

Harvard Medical School

Funding Source(s)

The Evaluation Center at the Human Services Research Institute, Cambridge, Massachusetts, (Cooperative Agreement U97 SM52004 with The Center for Mental Health Services, NIMH), the Agency for Healthcare Research and Quality (AHRQ) (Cooperative Agreement HS-0925), and the MacArthur Foundation

Composition of the Group that Developed the Measure

Harvard Medical School

Financial Disclosures/Other Potential Conflicts of Interest

None

Endorser

National Quality Forum

NQF Number

0008

Date of Endorsement

2012 Sep 17

Adaptation

This measure was adapted from the Consumer Assessment of Behavioral Health Services (CABHS) and the Mental Health Statistics Improvement Program (MHSIP).

Date of Most Current Version in NQMC

2004 Aug

Measure Maintenance

Unspecified

Date of Next Anticipated Revision

Unspecified

Measure Status

This is the current release of the measure.

The measure developer reaffirmed the currency of this measure in May 2016.

Source(s)

CAHPS® surveys and tools to advance patient-centered care [https://www.cahps.ahrq.gov/default.asp]. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); [updated 2008 Feb 21]; [accessed 2004 Apr 15]. [3 p].

ECHO® Survey and Reporting Kit 2004. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2004. Various p.

Measure Availability

ECHO Survey and Reporting Kit 2004 available from the CAHPS Web site External Web Site Policy.

For more information, contact CAHPS Technical Assistance.

Companion Documents

The following is available:

  • ECHO - the CAHPS behavioral health survey. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2004 [cited 2004 Apr 19].

For more information, contact CAHPS Technical Assistance.

NQMC Status

This NQMC summary was completed by ECRI on December 13, 2004. The information was verified by the measure developer on February 11, 2005.

This NQMC summary was retrofitted into the new template on April 21, 2011.

The information was reaffirmed by the measure developer on May 20, 2016.

Copyright Statement

No copyright restrictions apply.

NQMC Disclaimer

The National Quality Measures Clearinghouse™ (NQMC) does not develop, produce, approve, or endorse the measures represented on this site.

All measures summarized by NQMC and hosted on our site are produced under the auspices of medical specialty societies, relevant professional associations, public and private organizations, other government agencies, health care organizations or plans, individuals, and similar entities.

Measures represented on the NQMC Web site are submitted by measure developers, and are screened solely to determine that they meet the NQMC Inclusion Criteria.

NQMC, AHRQ, and its contractor ECRI Institute make no warranties concerning the content or its reliability and/or validity of the quality measures and related materials represented on this site. Moreover, the views and opinions of developers or authors of measures represented on this site do not necessarily state or reflect those of NQMC, AHRQ, or its contractor, ECRI Institute, and inclusion or hosting of measures in NQMC may not be used for advertising or commercial endorsement purposes.

Readers with questions regarding measure content are directed to contact the measure developer.