Don’t be Fooled by the Federal Nursing Home Five-Star Quality Rating System

In 2014, The New York Times reported that nursing facilities were gaming the Five-Star Quality Rating System on Nursing Home Compare and that “even nursing homes with a history of poor care rate highly in the areas that rely on self-reported data."[1]  The Times reported that nearly two-thirds of 50 facilities on CMS's watch list of poorly performing facilities (known as  Special Focus Facilities, or SFFs) achieve four and five stars (on a five-point scale) in staffing and quality measures.  Shortly thereafter, both the White House and the Centers for Medicare & Medicaid Services (CMS) announced plans to revise the Rating System in order both to reduce reliance on self-reported data and to make additional improvements to the federal website.[2]  In a 2014 report to CMS analyzing the first five years of the Five-Star Quality Rating System, Abt Associates confirmed that dramatic upward changes in facilities’ overall ratings reflected improved scores in self-reported staffing and quality measures, not in public survey results.[3]

A new research study finds that the composite or overall ratings, added to Nursing Home Compare in 2008, seem to matter to the public in making admission decisions.[4]  Although Nursing Home Compare had included considerable and increasing amounts of information about nursing home performance since President Bill Clinton first announced development of the public website in 1998, the information did not appear to influence admission decisions in the website’s initial years.  However, following the addition of the Five-Star Quality Rating System, which combined the separate data into five-point scales, researchers found that admissions to five-star nursing facilities increased, while admissions to one-star nursing facilities declined.[5]  The researchers found that two-, three-, and four-star ratings did not affect admissions decisions.

Reflecting on the new research findings that shifts in admissions appear to reflect star ratings, the Center for Medicare Advocacy (the Center) repeated its December 2011 analysis of Special Focus Facilities and Nursing Home Compare[6] to determine whether nursing facilities are continuing to report staffing and assessment information that lead to better reported results on the federal website.  In its new analysis of SFFs identified by CMS in August 2016, the Center found that nearly 50% of SFFs had four and five star ratings in staffing and nearly 40% of SFFs had four and five star ratings in their quality measures.  Even more troubling, however, nearly 45% of SFFs had overall ratings of two stars.  High scores in self-reported staffing or quality measures boosted nearly half of SFFs to two-star overall ratings.  The Center’s analysis indicates that data manipulation continues and, as reflected in Wenner’s research, this manipulation is likely influencing admission decisions. 

Nursing Home Compare, the Five Star Quality Rating System, and Special Focus Facilities

CMS’s public website, Nursing Home Compare, reports information about nursing homes.  Over the years, CMS has expanded the information that it publicly reports on the site.  Three distinct domains are covered – health inspections (which are based on unannounced annual and complaint surveys[7] conducted by state survey agencies to determine facilities’ compliance with federal standards of care, called Requirements of Participation[8]) and two self-reported areas, quality measures (which are based on resident assessment information) and nurse staffing.  As required by the Affordable Care Act, nursing facilities have now begun to report staffing data electronically, "based on payroll and other verifiable and auditable data,”[9] although the staffing data reported on the website are still based on self-reported information.  When CMS begins to use payroll-based data in Nursing Home Compare, however, gaming in facilities’ staffing domain should decline.

In 2008, CMS created a Five Star Quality Rating System to assign stars to each of the three domains and to establish a fourth composite rating that combines the three separate domains into a single overall score.  The composite rating begins with the health inspections domain and adjusts a facility’s composite score upwards or downwards by one star, depending on the facility’s five-star or one-star performance on the staffing and quality domains.[10]

Each month, CMS identifies nursing facilities as SFFs.[11]  These facilities, chosen by CMS and states, are among the facilities that provide the poorest care to residents, as determined by facilities’ three-year survey history.  SFFs are surveyed at least twice each year (compared to other facilities, which have a single annual survey) and are subject to enhanced enforcement.  CMS separately reports newly-identified SFFs, SFFs that have not improved, SFFs that have improved but are still SFFs, SFFs that have “graduated,” and SFFs that no longer participate in Medicare and Medicaid.

The Center’s Analysis of Special Focus Facilities

The Center looked at 42 current SFFs, as reported on CMS’s updated list on August 18, 2016[12] – all 16 facilities that were recently added and all 26 facilities that had not improved

Evaluating these 42 SFFs, the Center found:

• 19 of 42 (45%) SFFs received two stars in their overall score, although all but two of the 42 SFFs received one-star on health inspections (one SFF had a 3-star overall rating and no ratings were available for another SFF).  The higher overall score means that SFFs reported sufficiently high star ratings in either staffing or quality measures to boost their overall rating from one star to two stars. 
• Of the 19 facilities with 2 stars as their overall rating,
  • 15 (79%) had 5 or 4 stars on staffing (1 had 5 stars on staffing; 14 had 4 stars on staffing)
    • Of the 8 newly-added SFFs, 1 had 5 stars on staffing; 5 had 4 stars on staffing)
    • Of the 11 not improved SFFs, 9 had 4 stars on staffing
  • 11 (58%) had 5 or 4 stars on Quality Measures (8 had 5 stars on quality measures; 3 had 4 stars on quality measures)
    • Of the 8 newly-added SFFs, 4 had 5 stars on quality measures
    • Of the 11 not improved SFFs, 4 had 5 stars on quality measures; 3 had 4 stars on quality measures
• Of the 42 newly-added and not improved SFFs, 39 (93%) had remedies imposed
  • 25 had both civil money penalties and denials of payment for new admissions (DPNA) imposed
  • 9 had only CMPs imposed
  • 5 had only DPNAs imposed
  • 3 had neither CMPs nor DPNAs imposed
• Of the 42 facilities, 32 (76%) are for-profit, 8 (19%) are not-for-profit, and 2 (5%) are government-owned.  In 2011, 96% of 47 SFFs analyzed were for-profit.

Five-Star Facilities

Five-star facilities also use their staffing and quality measure domains to boost their overall ratings to five stars.  An analysis of the 113 skilled nursing facilities (SNFs) that have had five stars as their overall rating for all seven years since the Five Star Quality Rating System was introduced in December 2008 found that 96% of the 113 SNFs had their scores boosted to five stars by their staffing or quality domain scores or both

• 69 of the SNFs got an extra star in their overall rating from the staffing measure
• 16 of the SNFs got an extra star in their overall rating from the quality measure
• 21 of the SNFs got an extra star in their overall rating from both the staffing and quality measures.[13]​

The federal enforcement system seems to be using the SFF program, as intended, to focus on facilities that have had serious quality problems over an extended period of time, as reflected by the high CMPs and frequent DPNAs imposed against them in the prior three years.  Only three of the 42 SFFs analyzed by the Center had not had either a CMP or DPNA, or (most commonly) both, imposed in the three-year period.

However, facilities are continuing to self-report information on both staffing and quality measures that are sufficiently high to boost their overall scores from one star to two stars.  This finding is troubling for two reasons.  First, it continues to demonstrate manipulation of the federal website.  Second, it moves facilities with one star in health surveys into a two-star overall rating.  Facilities boosting their overall scores by reporting high staffing or quality measures prevent both the stigma of being a one-star facility and avoidance by families making placement decisions.  But these boosted scores seriously mislead the public when SFFs, which, by definition, are among the very poorest performing facilities in the country, are perceived as two-star facilities.

Similarly, at the other end of the spectrum, facilities misrepresent their five-star status when their overall ratings are boosted by their star ratings in the staffing and quality measure domains.

Providing composite scores is intended to help consumers make sense of the large amounts of material on Nursing Home Compare, but composite scores do a disservice to the public when nursing facilities’ self-reported information boosts facilities’ ratings into higher levels.

October 5, 2016 – T. Edelman