Outcome Measurement Program Case Studies
Appendix C
Case Study Protocol
Case Study Purpose
The RTC/OM was created with federal funds to assist in the development of effective HCBS outcome measurement tools, and this case study will make an important contribution to that goal. Several case studies will be undertaken to examine how commonly used HCBS outcome measurement tools are implemented, and to explore factors that influence their implementation. Through cross-case study analysis, the research team will seek to identify common measures and best practices in implementation to inform stakeholders on the effective measurement of HCBS.
Process
With input from the RTC/OM National Advisory Group (NAG), the research team identified several nationally recognized HCBS outcome measurement tools to study that are frequently used by states and organizations that operate HCBS.
In partnership with the organizations that offer the measurement tools, the research team will plan and schedule study activities including: identification of critical program components, collection of materials that describe the tool and its implementation procedures, visits to observe aspects of tool implementation, and interviews with key informants including the tool developers and site leaders, tool users including service organizations and government agencies, advocates for people receiving HCBS and people receiving HCBS.
Methods
Research Questions (Please note that research questions may change in response to certain project findings):
- What are the strengths and challenges of various outcome measurement programs?
- How do these strengths and challenges impact measure administration fidelity?
- What methodological components need to be in place to ensure measure administration fidelity in the implementation of HCBS outcome measures?
- To what extent have known factors that are important to fidelity been attended to in the programs reviewed (e.g., interviewer training, criterion testing, protection against/identification of response bias)?
- What are the similarities and differences in implementing various outcome measurement programs?
- What are the likely/potential implications for validity and reliability of data gathered that might derive from the differences?
- What factors most facilitate or distract from effective implementation of programs regarding community living and participation outcome measurement?
- What methodological components need to be in place to ensure measure administration fidelity in the implementation of HCBS outcome measures?
- What are the strengths and challenges of various outcome measurement programs and how do these impact measure administration fidelity?
- What are the similarities and differences in implementing various outcome measurement programs?
- What factors most facilitate or distract from effective implementation of programs regarding community living and participation outcome measurement?
Data Collection
Identification of critical program components in partnership with tool developers:
- Review of written materials including: procedure manuals, site leader selection process and training materials; marketing and end-user materials; description of technology used to support implementation and measurement; descriptions of technical assistance provided to measurement sites; description of methods for documentation of measurement data, description of methods for analysis and reports or feedback of measurement data, and any current technical and research reports focused on the measurement tool.
- In-depth interviews with key informants;
- On-site observation;
- Review of draft case study findings with the measurement organization; and,
- Development of a written case study report.