OBJECTIVES: The goal of this review was to evaluate the effectiveness of public reporting of health care quality information as a quality improvement strategy. We sought to determine if public reporting results in improvements in health care delivery and patient outcomes. We also considered whether public reporting affects the behavior of patients or of health care providers. Finally we assessed whether the characteristics of the public reports and the context affect the impact of public reports.
DATA SOURCES: Articles available between 1980 and 2011 were identified through searches of the following bibliographical databases: MEDLINE®, Embase, EconLit, PsychINFO, Business Source Premier, CINAHL, PAIS, Cochrane Database of Systematic Reviews, EPOC Register of Studies, DARE, NHS EED, HEED, NYAM Grey Literature Report database, and other sources (experts, reference lists, and gray literature).
REVIEW METHODS: We screened citations based on inclusion and exclusion criteria developed based on our definition of public reporting. We initially did not exclude any studies based on study design. Of the 11,809 citations identified through title and abstract triage, we screened and reviewed 1,632 articles. A total of 97 quantitative and 101 qualitative studies were included, abstracted, entered into tables, and evaluated. The heterogeneity of outcomes as well as methods prohibited formal quantitative synthesis. Systematic reviews were used to identify studies, but their conclusions were not incorporated into this review.
RESULTS: For most of the outcomes, the strength of the evidence available to assess the impact of public reporting was moderate. This was due in part to the methodological challenges researchers face in designing and conducting research on the impact of population-level interventions. Public reporting is associated with improvement in health care performance measures such as those included in Nursing Home Compare. Almost all identified studies found no evidence or only weak evidence that public reporting affects the selection of health care providers by patients or their representatives. Studies of health care providers' response to public reports suggest they engage in activities to improve quality when performance data are made public. Characteristics of public reports and the context, which are likely to be important when considering the diffusion of quality improvement activities, were rarely studied or even described.
CONCLUSIONS: The heterogeneity of the outcomes and the moderate strength of evidence for most outcomes make it difficult to draw definitive conclusions. However, some observations were supported by existing research. Public reporting is more likely to be associated with changes in health care provider behaviors than with selection of health services providers by patients or families. Quality measures that are publicly reported improve over time. Although the potential for harms is frequently cited by commentators and critics of public reporting, the amount of research on harms is limited and most studies do not confirm the potential harm.
BACKGROUND: Previous reviews have shown inconsistent effects of publicly reported performance data on quality of care, but many new studies have become available in the 7 years since the last systematic review. PURPOSE: To synthesize the evidence for using publicly reported performance data to improve quality. DATA SOURCES: Web of Science, MEDLINE, EconLit, and Wilson Business Periodicals (1999-2006) and independent review of articles (1986-1999) identified in a previous systematic review. Only sources published in English were included. STUDY SELECTION: Peer-reviewed articles assessing the effects of public release of performance data on selection of providers, quality improvement activity, clinical outcomes (effectiveness, patient safety, and patient-centeredness), and unintended consequences. DATA EXTRACTION: Data on study participants, reporting system or level, study design, selection of providers, quality improvement activity, outcomes, and unintended consequences were extracted. DATA SYNTHESIS: Forty-five articles published since 1986 (27 of which were published since 1999) evaluated the impact of public reporting on quality. Many focus on a select few reporting systems. Synthesis of data from 8 health plan-level studies suggests modest association between public reporting and plan selection. Synthesis of 11 studies, all hospital-level, suggests stimulation of quality improvement activity. Review of 9 hospital-level and 7 individual provider-level studies shows inconsistent association between public reporting and selection of hospitals and individual providers. Synthesis of 11 studies, primarily hospital-level, indicates inconsistent association between public reporting and improved effectiveness. Evidence on the impact of public reporting on patient safety and patient-centeredness is scant. LIMITATIONS: Heterogeneity made comparisons across studies challenging. Only peer-reviewed, English-language articles were included. CONCLUSION: Evidence is scant, particularly about individual providers and practices. Rigorous evaluation of many major public reporting systems is lacking. Evidence suggests that publicly releasing performance data stimulates quality improvement activity at the hospital level. The effect of public reporting on effectiveness, safety, and patient-centeredness remains uncertain.
The goal of this review was to evaluate the effectiveness of public reporting of health care quality information as a quality improvement strategy. We sought to determine if public reporting results in improvements in health care delivery and patient outcomes. We also considered whether public reporting affects the behavior of patients or of health care providers. Finally we assessed whether the characteristics of the public reports and the context affect the impact of public reports.
DATA SOURCES:
Articles available between 1980 and 2011 were identified through searches of the following bibliographical databases: MEDLINE®, Embase, EconLit, PsychINFO, Business Source Premier, CINAHL, PAIS, Cochrane Database of Systematic Reviews, EPOC Register of Studies, DARE, NHS EED, HEED, NYAM Grey Literature Report database, and other sources (experts, reference lists, and gray literature).
REVIEW METHODS:
We screened citations based on inclusion and exclusion criteria developed based on our definition of public reporting. We initially did not exclude any studies based on study design. Of the 11,809 citations identified through title and abstract triage, we screened and reviewed 1,632 articles. A total of 97 quantitative and 101 qualitative studies were included, abstracted, entered into tables, and evaluated. The heterogeneity of outcomes as well as methods prohibited formal quantitative synthesis. Systematic reviews were used to identify studies, but their conclusions were not incorporated into this review.
RESULTS:
For most of the outcomes, the strength of the evidence available to assess the impact of public reporting was moderate. This was due in part to the methodological challenges researchers face in designing and conducting research on the impact of population-level interventions. Public reporting is associated with improvement in health care performance measures such as those included in Nursing Home Compare. Almost all identified studies found no evidence or only weak evidence that public reporting affects the selection of health care providers by patients or their representatives. Studies of health care providers' response to public reports suggest they engage in activities to improve quality when performance data are made public. Characteristics of public reports and the context, which are likely to be important when considering the diffusion of quality improvement activities, were rarely studied or even described.
CONCLUSIONS:
The heterogeneity of the outcomes and the moderate strength of evidence for most outcomes make it difficult to draw definitive conclusions. However, some observations were supported by existing research. Public reporting is more likely to be associated with changes in health care provider behaviors than with selection of health services providers by patients or families. Quality measures that are publicly reported improve over time. Although the potential for harms is frequently cited by commentators and critics of public reporting, the amount of research on harms is limited and most studies do not confirm the potential harm.
Systematic Review Question»Systematic review of interventions
