Outcome Measurement Program Case Studies

The  three measurement programs examined in these case studies, National Core Indicators-In-person survey (NCI-IPS), Personal Outcome Measures (POMs), and Independent Monitoring for Quality program (IM4Q) are used to measure Home- and Community-based Services outcomes for adults with intellectual and developmental disabilities (IDD). While each of these programs has a different focus (systemic vs. individual), the case studies identified key factors that can be applied to create a robust measurement program that can be used to evaluate service quality and make data-driven decisions about quality improvement for the individuals with IDD receiving Medicaid funded HCBS supports.

Lessons Learned

Commitment from Leadership

Agency leaders who commit to outcome measurement programs also commit to developing more effective program of services. Successful measurement programs are built on a foundation of adequate financial investments and staff time for effective implementation. Leadership also needs to communicate the importance of collecting high-quality data and incorporating this information into quality assurance programs and data-driven decision-making. Systemic measurement programs implemented by states or large organizations may have an easier time committing resources to these programs without diverting resources from other projects.  Smaller agencies with more limited resources will need to be creative to design and implement a robust measurement program.  This will require committed leadership to finding innovative ways to use available resources to develop or adopt, and implement an outcome measurement program.

Stakeholder Involvement and Buy-In

Stakeholders, including agency staff and recipients of HCBS and their families, all need to be educated about the purpose and importance of an outcome measurement (OM) program.  Some case study participants across the three case studies mentioned that individual service recipients and other stakeholders were not always aware of the measurement program's purpose and how their information will be used. Educating stakeholders to help them understand the purpose of the measurement program, what was learned from the information they provided, and how that information was used is an important step in engaging individuals in the measurement process, including in survey participation and other data collection efforts.  Measurement programs implemented at the provider level may have an easier time helping the participant make the connection between their participation in the data collection and how it could lead to improved services.

Training and Technical Assistance

Training and technical assistance for interviewers and measurement program coordinators are essential to ensuring the fidelity of program implementation. Effective training includes time for interviewers to role play and practice interviewing to apply and solidify skills.  Also, effective training involves interviewers in shadowing experienced interviewers and builds in observations  and feedback of interviewers to ensure fidelity.  Refresher training opportunities (such as videos) can be useful in maintaining interview fidelity.

Technical assistance included support for interviewers in the field, with programs having people available by phone to answer interviewer questions and troubleshoot concerns.  Other technical assistance examples include technical assistance for local coordinators to ensure program fidelity and for state agencies that included sampling, data collection, and using data effectively.

Data from Measurement Programs Drives Decision-Making

Effective measurement programs start with a clear understanding of why the program is being implemented, what questions need to be answered, and how the data will improve services.  Measurement programs that are woven into quality assurance systems and provide data that can inform decision-making by organizational leadership and other stakeholders are more successful in improving the quality of HCBS supports.