Additional stars in the rating system for dialysis facilities may not reflect increases in quality of care, according to an analysis presented at the National Kidney Foundation 2016 Spring Clinical Meetings.

The research examines the Dialysis Facility Compare Star Rating Program, a rating system developed by Medicare that assigns 1 to 5 stars to dialysis facilities based on the health and treatment of the patients in their clinics compared to patients in other dialysis facilities across the country. It is intended to help patients find and compare dialysis facilities.

Read also: Finding consistency in Medicare’s quality ratings for dialysis providers 

Statistical analysis from Prima Health Analytics in Weymouth, MA, found that the dialysis centers that dramatically improved their star ratings likely benefitted from chance and reporting improvements, rather than improvements in clinical care.

“The facilities with the greatest improvement in star ratings may have achieved this with very little change in their actual clinical practices,” said Mark Stephens, Principal/Owner at Prima Health Analytics. “Most of the improvement was probably due to random chance, data reporting improvements, and statistical artifacts of the rating methodology.”

Read also: A comparison of the new dialysis facility star ratings to the original

“The flip side of this, of course, is that facilities that are actually making real-world quality of care improvements may not see this reflected in their star ratings,” Stephens said.

Ten facilities were included in the case study; six went from two to five stars, four went from one to four stars. The three main drivers of these dramatic improvements, according to Stephens, were:

1)Luck – The average improvement in standardized mortality (SMR), hospitalization (SHR) and transfusion (STrR) ratio scores was 38%. Only 3% of these standardized scores (pooled analysis across both years) had statistically significant odds of being different from the national average (p < 0.05).

2) Better reporting – Two facilities improved their dialysis adequacy (Kt/V) scores from the lowest half-percentile nationally to above average scores in a single year. Better claims coding in 2014 may account for this dramatic improvement.

3) Maturity – 5 of the 10 facilities were Medicare-certified after the beginning of the first star rating experience period (January 2010), resulting in missing measures in the first round of star ratings. The addition of high-ranking STrR and SMR scores helped 3 facilities score higher in round 2.

“There is no patient input to the quality ratings being used by Medicare for dialysis centers. Quality ratings that at least partially consider patient satisfaction with their dialysis treatments might produce very different rankings,” Stephens said. “Our hope is that we can elevate the dialogue about fixing the problems in the system, and hopefully improve the education of both clinicians and patients about how to interpret these ratings.”