Outcome Measurement Program Case Studies

Policy Context

The Centers for Medicare and Medicaid Services provides funding for the Home and Community-Based Services waiver.  HCBS is the primary public resource for individualized long-term service and support (LTSS), enabling more than three million people with disabilities to live in their communities.  The HCBS mission is to help LTSS users enjoy the same access and opportunities for social, civic, career, educational, and home life as non-disabled peers.  Accurate information about the effectiveness of this publically funded program is important for the responsible government agencies, elected officials, users of HCBS-funded supports, advocates, and the public to obtain data needed for evidence-based decision-making.

Measurement Program

The Independent Monitoring for Quality program (IM4Q), piloted in 2002, was designed to improve everyday living outcomes for individuals with intellectual disability and their families who are receiving services through Pennsylvania’s Office of Developmental Programs (ODP; Feinstein & Caruso, 2003). IM4Q teams conduct personal interviews using a standardized survey to gather information and produce reports that will enable ODP, administrative entities (AEs), and others to evaluate the effectiveness of services and individual outcomes. The participation of self-advocates, family members, advocates, and citizens from the community as interviewers assures an objective and participant-centered approach to evaluate the outcomes of services and experiences of individuals.  

Key Messages

Strengths of IM4Q in Measuring HCBS Outcomes:

1. Independence in monitoring ensures that data is collected by organizations that do not have a conflict of interest in the service delivery system.
2. IM4Q has a longitudinal component that allows for monitoring the effects of policy changes.
3. Administrative entities and local agencies receive high-quality technical assistance regarding any implementation questions that arose during the monitoring process.
4. Allows for data-driven policy and programming decisions.
5. Provide opportunities for individuals to participate in the interview process that directly relates to the policy at the system level.
6. The extent to which the measurement program is woven into the state’s Quality Assurance processes.

Challenges  of IM4Q in Measuring HCBS Outcomes:

1. Reliability and validity testing of the Essential Data Elements (EDE) and other aspects of the IM4Q program are not robust.