• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Donate Now
  • Sign Up

Center for Medicare Advocacy

Advancing Access to Medicare and Healthcare

  • Eligibility/Enrollment
  • Coverage/Appeals
    • Medicare Costs (2021)
    • Self Help Materials – Toolkits & More
  • Topics
    • Basic Introduction to Medicare
    • COVID-19 and Medicare
    • Medicare Costs (2021)
    • Home Health Care
    • Improvement Standard and Jimmo News
    • Nursing Home / Skilled Nursing Facility Care
    • Outpatient Observation Status
    • Part B
    • Part D / Prescription Drug Benefits
    • Medicare for People Under 65
    • Medicare “Reform”
    • All Other Topics
    • Resources
      • Infographics
  • Publications
    • CMA Alerts
    • Fact Sheets & Issue Briefs
    • Infographics
    • The Medicare Handbook
    • SNF Enforcement Newsletter
    • Elder Justice Newsletter
    • Medicare Facts & Fiction
    • Annual Report
    • Articles by Topic
  • Litigation
    • Litigation News
    • Cases
    • Litigation Archive
    • Amicus Curiae Activities
  • Newsroom
    • Press Releases
    • Editorials & Letters to the Editor
    • CMA Comments, Responses, and Letters
    • Medicare Facts & Fiction
    • CMA in the News
  • About Us
    • Mission Statement
    • CMA FAQs
    • Personnel & Boards
    • The Center for Medicare Advocacy Founder’s Circle
    • Connecticut Dually Eligible Appeals Project
    • Ossen Medicare Outreach, Education and Advocacy Project
    • National Medicare Advocates Alliance
    • National Voices of Medicare Summit
    • CMA Webinars
    • Products & Services
    • Testimonials
    • Career, Fellowship & Internship Opportunities
    • Contact Us
  • Support Our Work
    • Donate Now
    • Join the Center for Medicare Advocacy Founder’s Circle
    • Take Action
    • Share Your Health Care Story
    • Tell Congress to Protect Our Care
    • Listen to Medicare & Health Care Stories
    • Sign Up

Nursing Facilities’ “Quality Measures” Do Not Reflect Actual Quality of Care Provided to Residents

August 9, 2018

Print Friendly, PDF & Email

The Center for Medicare Advocacy wanted to determine whether nursing facilities that had one-star in their health survey ratings on Nursing Home Compare were able to boost their overall ratings from one star to two stars through the designation of five stars in the self-reported quality measure domain.  The finding – that many facilities in the five states reviewed were able to boost their overall ratings in this way – undermines the validity of “quality measures” as reflecting the actual quality of care provided to residents by nursing facilities.

When facilities that are challenged with poor performance cite their “quality measures,” beware.  Quality measures do not reflect the quality of care residents receive.

Background

The Centers for Medicare & Medicaid Services’ (CMS’) website Nursing Home Compare provides public information about nursing facilities that participate in Medicare or Medicaid or both.  It provides information about three domains – health surveys, staffing, and quality measures – and an overall rating. 

  • The health survey rating is based on unannounced annual and complaint surveys conducted by state survey agencies, under contract with CMS;
  • The nurse staffing rating is based on total nurse staffing (registered nurses (RNs), licensed practical nurses and nurse aides) and RN staffing; and
  • The quality measures rating is based primarily on self-reported and unaudited resident assessment information provided by facilities.   

CMS assigns star ratings to each of the three domains, and an overall score which starts with the health survey rating and revises the overall rating downward or upward to reflect, respectively, one-star and five-star ratings on the staffing and quality measure domains.  Ratings in each domain and in the overall rating range from one to five stars, with one star reflecting the lowest performance (“much below average”) and five stars, the highest performance (“much above average”).

The three domains are separately scored, each with highly complex scoring rules.  As described in CMS’s Technical Users’ Guide,[1] the rating system is based on the following principles:

  • The health survey domain “calculates a health inspection score based on points assigned to deficiencies identified in each active provider’s two most recent recertification health inspections prior to November 28, 2017, as well as on deficiency findings from the most recent two years of complaint inspections prior to November 28, 2017.”[2]  Ratings are “fixed” for each state: the top 10% receive 5 stars; the middle 70% of facilities receive a rating of two, three, or four stars, with an equal number (approximately 23.33%) in each star rating category; and the bottom 20% receive one star.[3]  

    CMS “froze” the survey domain for a year, effective November 27, 2017, to reflect implementation of the new federal survey process.[4]  Deficiencies that are cited in surveys conducted after November 27, 2017 are reported on Nursing Home Compare, but they are not used in calculating the health survey rating.
     

  • The staffing domain reports two components – total nurse staffing (RN, licensed practical nurses, and certified nurse aides) and on RN staffing, which both are case-mix adjusted.[5]  Since May 2018, the staffing domain has been based on payroll-based journal information (PBJ), not, as before, on self-reported information.[6] 

    Scoring uses “cut points,” [7] which CMS adjusted so that the ratings looked the same after April 2018, when CMS first began to report PBJ-reported staffing data.  The New York Times reported that 70% of nurse facilities had lower staffing numbers, averaging 12% lower under PBJ than under the prior self-reported system, and that, in the last quarter of 2017, a quarter of all nursing facilities reported at least one day in the three-month period when there was no RN present in the facility.[8] 
     

  • Nursing Home Compare reports 27 quality measures (24 assessment-based quality measures that are self-reported by facilities and 3 measures that are both assessment-based and claims data-based),[9] but the quality measure rating is based on only 16 of the 27 measures (9 long-stay self-reported measures for residents in the facility for 100 days or more; 4 short-stay self-reported measures for residents in the facility for fewer than 100 days; and 3 claims-based short-stay measures).[10]

    Each measure is separately scored and assigned points (20-100) based on the national distribution of the measure.[11]  CMS then groups facilities “into quintiles based on the national distribution of the QM,” with the lowest-performing facilities receiving 20 points and the highest-performing facilities, 100 points.[12]  The point scores for each of the 16 measures are added together and ratings assigned according to cut points.[13]
     

  • A five-star rating in the quality measure domain boosts a facility’s overall rating by one star, under the scoring rules used by CMS.[14] 

This Analysis

The Center began by identifying nursing facilities listed on Nursing Home Compare on August 7, 2018 with overall ratings of two stars.  Next the Center looked at these two-star facilities to determine how many had five stars in the quality measure domain.  Finally, the Center calculated how many facilities had one star in the health survey domain and five stars in the quality measure domain. 

Facilities with overall ratings of two stars report five stars in quality measures, from 39% of the two-star facilities in Iowa to 80% of the two-star facilities in California. 

Even more disturbing, many facilities with overall ratings of two stars had one star in health surveys but five stars in quality measures, with the quality measure rating boosting the overall rating to two stars.  These boosts occurred in 28 facilities (33%) in Iowa and in 167 facilities (71%) in California.

State

Number of facilities with 2 stars in overall rating

Number of 2-star facilities with 5 stars in quality measure domain

Percentage of 2-star facilities with 5 stars in quality measure domain

Number of 2-star facilities with 1 star in health survey domain and 5 stars in quality measure domain

Percentage of 2-star facilities that boosted their rating from 1 to 2 stars, based on their quality measure rating

California

235

188

80%

167

71%

Connecticut

  47

  28

60%

  26

55%

Iowa

  84

  33

39%

  28

33%

New York

115

  65

56%

  46

40%

Wisconsin

  68

  35

51%

  33

48%

Conclusion

These findings confirm that the quality measure domain overstates the “quality” of a facility, is completely unreliable, and should not be used in determining facilities’ overall ratings.  As The New York Times reported in 2014, facilities “game” the rating system, particularly in their misuse of the quality measure rating.[15]

 

August 9, 2018 – T. Edelman


[1] CMS, Design for Nursing Home Compare Five-Star Quality Rating System: Technical Users’ Guide (July 2018) [hereafter Technical Users’ Guide], https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/CertificationandComplianc/Downloads/usersguide.pdf. 
[2] Id. 5.
[3] Id. 8.
[4] CMS, “Temporary Enforcement Delays for Certain Phase 2 F-Tags and Changes to Nursing Home Compare,” S&C: 18-04-NH (Nov. 24, 2017), https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/Downloads/Survey-and-Cert-Letter-18-04.pdf.
[5] Technical Users’ Guide, supra note 1, 9.
[6] CMS, “Transition to Payroll-Based Journal (PBJ) Staffing Measures on the Nursing Home Compare tool on Medicare.gov and the Five Star Quality Rating System,” QSO-18-17-NH (Apr. 6, 208), https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/Downloads/QSO18-17-NH.pdf. 
[7] Technical Users’ Guide, supra note 1, 14 (Table 5).
[8] Jordan Rau, “‘It’s Almost Like a Ghost Town.’ Most Nursing Homes Overstated Staffing for Years,” The New York Times (Jul. 7, 2018), https://www.nytimes.com/2018/07/07/health/nursing-homes-staffing-medicare.html.
[9] Technical Users’ Guide, supra note 1, 12-13.
[10] Id. 16-17 (Table 6).
[11] Id. 16.
[12] Id. 16.
[13] Id. 20 (Table 7).
[14] Id. 18-19.
[15] Katie Thomas, “Medicare Star Ratings Allow Nursing Homes to Game the System,” The New York Times (Aug. 14, 2014), https://www.nytimes.com/2014/08/25/business/medicare-star-ratings-allow-nursing-homes-to-game-the-system.html.

 

Filed Under: Article Tagged With: alert, Quality of Care, Skilled Nursing Facility, snf, Weekly Alert

Primary Sidebar

Easy Access to Understanding Medicare

The Center for Medicare Advocacy produces a range of informative materials on Medicare-related topics. Check them out:

  • Medicare Basics
  • CMA Alerts
  • CMA Webinars
  • Connecticut Info & Projects
  • Health Care Stories
  • Se habla Español

Sign Up for CMA Alerts

Jimmo v. Sebelius

Medicare covers skilled care to maintain or slow decline as well as to improve.

Improvement Isn’t Required. It’s the law!

Read more.

Medicare: Build Back Better

By prioritizing Medicare beneficiaries and the health systems that serve them, we can avoid drastic national consequences. The Center for Medicare Advocacy proposes a five-part plan that will make Medicare a bulwark against the worsening health and economic challenges facing the American people.

learn more.

Latest Tweets

  • .@CMAorg is grateful to @johnahartford for the many years of ongoing support and for sponsoring this year’s Sen. Ja… https://t.co/5m2bEvignf, Mar 5
@CMAorg

Footer

Stay Connected:

  • Contact Us
  • Sitemap
  • Products & Services
  • Copyright/Privacy

© 2021 · Center for Medicare Advocacy