Annual Adjustments Improve Indicator Accuracy
Why do Quality of Care and Quality of Life Indicator (QCLI) thresholds change?
The April 2009 issue of Provider describes how and why the thresholds for QCLIs are set in the QIS.
To review briefly, the thresholds used to determine whether a Stage 2 investigation will be conducted are analogous to the values for diagnostic screens that doctors use to decide whether to conduct more in-depth diagnostic tests. These initial screening tests are brief and not very invasive, but they are also not definitive.
What they do is indicate the need for further tests that are typically more invasive and resource intensive. Stage 1 of the QIS is similar to the screening tests, while Stage 2 is similar to a more invasive medical work-up.
The QCLI thresholds provide the values for Stage 1 screening that are used to determine whether further, more in-depth Stage 2 investigations are warranted. Setting these thresholds uses the same method that is used to set the “normal” values for medical diagnostic tests, sensitivity, and specificity analysis. The goal is to ensure that the QCLI is sensitive enough to recognize when deficient practice may be present and yet specific enough so that in-depth Stage 2 investigations are not conducted in numerous cases where there is not a reasonable potential for deficient practice.
If the thresholds are set too low, surveyors will spend their resources investigating numerous aspects of care where no problem exists. On the other may result in surveyors missing an important care problem. It is important not to confuse the thresholds with benchmarks or percentiles that indicate where a facility stands in relation to others. Every facility can be below the threshold just like every person can have a medical diagnostic test result within the normal range.
As more QIS surveys are conducted, data are collected that can be analyzed to determine how sensitive and how specific each QCLI is at the threshold that is being used. With over 4,000 QIS surveys conducted to date, and large resident samples obtained, ample data have accumulated on each QCLI for the analysis.
The University of Colorado in Denver performs the threshold analysis for the Centers for Medicare & Medicaid Services (CMS) on an annual basis. CMS then makes decisions on threshold value changes to improve sensitivity or specificity.
Not every threshold is adjusted each time. Most recently, CMS made changes to 14 of the 128 QCLI thresholds, and these changes were effective on March 29, 2010. Of the 14 QCLIs with threshold changes, seven thresholds were lowered to be more sensitive, and seven thresholds were raised to be more specific.