RTC/OM HCBS Measurement Education Modules

Reliability Types - Inter-rater Reliability

Purpose:

  • Determine how consistently interviewers or observers rate and record participant responses.
  • To what degree do interviewers/observers ratings differ due to measurement error?

Application:

  • Multiple interviewers/observers assess participants on a set of tasks or outcomes

Analysis Method:

  • Coefficient alpha: similar to internal consistency, but examines the consistency between observers instead of between items

Example:

  • Multiple interviewers observe several HCBS recipients responding about the quality of their transportation services. For each recipient, do interviewers reach similar conclusions about the quality of transportation the person receives?