BACKGROUND: To assess both qualitatively and quantitatively the impact of Public Reporting (PR) on clinical outcomes, we carried out a systematic review of published studies on this topic.
METHODS: Pubmed, Web of Science and SCOPUS databases were searched to identify studies published from 1991 to 2014 that investigated the relationship between PR and clinical outcomes. Studies were considered eligible if they investigated the relationship between PR and clinical outcomes and comprehensively described the PR mechanism and the study design adopted. Among the clinical outcomes identified, meta-analysis was performed for overall mortality rate which quantitative data were exhaustively reported in a sufficient number of studies. Two reviewers conducted all data extraction independently and disagreements were resolved through discussion. The same reviewers evaluated also the quality of the studies using a GRADE approach.
RESULTS: Twenty-seven studies were included. Mainly, the effect of PR on clinical outcomes was positive. Meta-analysis regarding overall mortality included, in a context of high heterogeneity, 10 studies with a total of 1,840,401 experimental events and 3,670,446 control events and resulted in a RR of 0.85 (95 % CI, 0.79-0.92).
CONCLUSIONS: The introduction of PR programs at different levels of the healthcare sector is a challenging but rewarding public health strategy. Existing research covering different clinical outcomes supports the idea that PR could, in fact, stimulate providers to improve healthcare quality.
OBJECTIVES: The goal of this review was to evaluate the effectiveness of public reporting of health care quality information as a quality improvement strategy. We sought to determine if public reporting results in improvements in health care delivery and patient outcomes. We also considered whether public reporting affects the behavior of patients or of health care providers. Finally we assessed whether the characteristics of the public reports and the context affect the impact of public reports.
DATA SOURCES: Articles available between 1980 and 2011 were identified through searches of the following bibliographical databases: MEDLINE®, Embase, EconLit, PsychINFO, Business Source Premier, CINAHL, PAIS, Cochrane Database of Systematic Reviews, EPOC Register of Studies, DARE, NHS EED, HEED, NYAM Grey Literature Report database, and other sources (experts, reference lists, and gray literature).
REVIEW METHODS: We screened citations based on inclusion and exclusion criteria developed based on our definition of public reporting. We initially did not exclude any studies based on study design. Of the 11,809 citations identified through title and abstract triage, we screened and reviewed 1,632 articles. A total of 97 quantitative and 101 qualitative studies were included, abstracted, entered into tables, and evaluated. The heterogeneity of outcomes as well as methods prohibited formal quantitative synthesis. Systematic reviews were used to identify studies, but their conclusions were not incorporated into this review.
RESULTS: For most of the outcomes, the strength of the evidence available to assess the impact of public reporting was moderate. This was due in part to the methodological challenges researchers face in designing and conducting research on the impact of population-level interventions. Public reporting is associated with improvement in health care performance measures such as those included in Nursing Home Compare. Almost all identified studies found no evidence or only weak evidence that public reporting affects the selection of health care providers by patients or their representatives. Studies of health care providers' response to public reports suggest they engage in activities to improve quality when performance data are made public. Characteristics of public reports and the context, which are likely to be important when considering the diffusion of quality improvement activities, were rarely studied or even described.
CONCLUSIONS: The heterogeneity of the outcomes and the moderate strength of evidence for most outcomes make it difficult to draw definitive conclusions. However, some observations were supported by existing research. Public reporting is more likely to be associated with changes in health care provider behaviors than with selection of health services providers by patients or families. Quality measures that are publicly reported improve over time. Although the potential for harms is frequently cited by commentators and critics of public reporting, the amount of research on harms is limited and most studies do not confirm the potential harm.
BACKGROUND: Previous reviews have shown inconsistent effects of publicly reported performance data on quality of care, but many new studies have become available in the 7 years since the last systematic review. PURPOSE: To synthesize the evidence for using publicly reported performance data to improve quality. DATA SOURCES: Web of Science, MEDLINE, EconLit, and Wilson Business Periodicals (1999-2006) and independent review of articles (1986-1999) identified in a previous systematic review. Only sources published in English were included. STUDY SELECTION: Peer-reviewed articles assessing the effects of public release of performance data on selection of providers, quality improvement activity, clinical outcomes (effectiveness, patient safety, and patient-centeredness), and unintended consequences. DATA EXTRACTION: Data on study participants, reporting system or level, study design, selection of providers, quality improvement activity, outcomes, and unintended consequences were extracted. DATA SYNTHESIS: Forty-five articles published since 1986 (27 of which were published since 1999) evaluated the impact of public reporting on quality. Many focus on a select few reporting systems. Synthesis of data from 8 health plan-level studies suggests modest association between public reporting and plan selection. Synthesis of 11 studies, all hospital-level, suggests stimulation of quality improvement activity. Review of 9 hospital-level and 7 individual provider-level studies shows inconsistent association between public reporting and selection of hospitals and individual providers. Synthesis of 11 studies, primarily hospital-level, indicates inconsistent association between public reporting and improved effectiveness. Evidence on the impact of public reporting on patient safety and patient-centeredness is scant. LIMITATIONS: Heterogeneity made comparisons across studies challenging. Only peer-reviewed, English-language articles were included. CONCLUSION: Evidence is scant, particularly about individual providers and practices. Rigorous evaluation of many major public reporting systems is lacking. Evidence suggests that publicly releasing performance data stimulates quality improvement activity at the hospital level. The effect of public reporting on effectiveness, safety, and patient-centeredness remains uncertain.
The public release of health care-quality data into more formalized consumer health report cards is intended to educate consumers, improve quality of care, and increase competition in the marketplace The purpose of this review is to evaluate the evidence on the impact of consumer report cards on the behavior of consumers, providers, and purchasers. Studies were selected by conducting database searches in Medline and Healthstar to identify papers published since 1995 in peer-review journals pertaining to consumer report cards on health care. The evidence indicates that consumer report cards do not make a difference in decision making, improvement of quality, or competition. The research to date suggests that perhaps we need to rethink the entire endeavor of consumer report cards. Consumers desire information that is provider specific and may be more likely to use information on rates of errors and adverse outcomes. Purchasers may be in a better position to understand and use information about health plan quality to select high-quality plans to offer consumers and to design premium contributions to steer consumers, through price, to the highest-quality plans.
- There are some pressures for acute services to be concentrated in hospitals with larger volume.
- Much research examining the relationship between hospitals or clinician volume and health outcomes is of poor quality and does not make adequate adjustment for differences in patient case-mix.
- The best research suggests that there is no general relationship between volume and quality. However, in some specialities there appear to be quality gains associated with increased hospital or clinician volume.
- There is no evidence that cost savings can be secured merely by increasing scale in acute hospitals beyond 200 beds and it is likely that large hospitals (above 600 beds) display diseconomies of scale, though these inefficiencies may be offset in other
ways.
- There is evidence that utilisation of some health services is lower for patients living further away. When services are concentrated, some of the costs are shifted from the health service to patients and their carers
OBJECTIVES: To carry out a systematic review of the literature in order to examine the evidence for a relationship between the volume of activity of a hospital and patient outcomes in a range of procedures. DATA SOURCES: Search of MEDLINE (from 1985 to 1994) and of science Citation Index on the Bath Information and Data Service (BIDS). Both letters and editorials were excluded. The reference list of identified articles were also searched. Key relevant journals were also hand-searched: Medical Care from 1971 to 1994 and Health Policy from 1986 to 1994. Researchers in the field in the United Kingdom and the United States were consulted to identify published and unpublished evidence. STUDY SELECTION: Studies were included if they empirically assessed the relationship between the frequency with which procedures were carried out, or patients with a particular diagnosis were treated, and health outcomes such as mortality or morbidity. Individual studies were judged to be relevant by one reviewer. Data extraction was checked by another reviewer. DATA EXTRACTION: For studies of coronary artery bypass graft surgery (CABG) which examined the effect of volume on outcome, details of the source of data and the date of data collection, number of hospitals and patients, volume cut-off points used, statistical adjustements made in the analyses, together with the relationship found, were recorded. DATA SYNTHESIS: A narrative overview of most of the evidence was carried out. In the case of CABG surgery a meta-analysis by regression was used. the studies were pooled in order to assess the degree to which the estimates of the effect of volume on outcome were affected by the degree of adjustment for case-mix. Factors adjusted for included age, sex, severity of diagnoses and comorbidity. In addition the analysis investigated the relationship between the estimates of volume effect and year of data. RESULTS: There is a large literature, primarily from North America which has examined the relationship between volume and outcome for a wide range of (mainly surgical) procedures. Almost all of the research in this area has used an observational study design with hospital mortality rates as the measure of quality. The methods used in each study differed considerably and ranged from simple comparisons of outcomes by hospital volume to complex simultaneous-equation models. A wide range of levels of volume of activity have been used to categorise hospitals as high or low volume.
A positive relationship between volume and outcome (where higher volumes were associated with lower mortality rates) has been reported for most procedures: abdominal aortic aneurysm, vascular surgery, biliary tract surgery, cardiac catheterization and angiography, CABG surgery, total hip replacement, prostatectomy, intestinal operations, hysterectomy, acute myocardial infarction, perinatal ilness, hernia and trauma care. In contrast, studies of treatment for fracture of the femur, appendicectomy, stomach operations and burn care have been reported to have either no relationship or a negative relationship (where higher volumes were associated with higher mortality rates) between volume and outcome.
Few of the studies adjusted for the effects of case-mix differences between high and low volume hospitals. The 15 identified studies which examined CABG surgery differed with respect to the extent of adjustement for confounding. The analysis carried out suggests that the greater the adjustment for the effects fo case-mix, the smaller the size of the estimate of benefit associated with increased volumes of activity.
The available studies did not shed much light on the possible causes of any volume-outcome relationship and the level (e.g. hospital, ward, clinician) at which it might operate. Thus it was difficult to disentangle any direction of cause i.e. whether increased volume may have generated better outcomes of if better units attracted more patients. CONCLUSIONS: Whilst most studies report a positive relationship between hospital volume and outcome for several elective procedures, they may have biased estimates of the size of the effect of volume because of poor adjustment for the effect of differences in case-mix between high and low volume hospitals. There is also a paucity of information as to why high volume might be associated with better outcomes. For example, there is little evidence whether clinician-related elements such as better judgement about high and low risk patients may be related to volume or whether high volume clinicians keep up with the literature and develop an effective protocol of practice. If the latter were true then the adoption of clinical guidelines by other clinicians may improve quality of care and close the gap between smaller and larger units. Alternatively, there may be institution related elements such as the level of facilities which produce better outcomes. There is little evidence as to whether merging hospitals to create larger units will result in a change in outcomes over time. Because of the uncertainty in both the size and the interpretation of any effect of volume reported, caution should be exercised in using the research literature to justify policies of reorganisation of health care delivery. Therefore, the main recommendation is that policy-makers should be cautious when invoking the assumed improvements in outcome achieved by volume as a key argument for centralisation of services.
OBJECTIVES: To examine the evidence for a relation between volume of coronary artery bypass graft surgery and hospital death rates, and to assess the degree to which this could be due to confounding because of differences in case mix.
SUBJECTS: People receiving coronary artery bypass graft surgery in the United States.
DESIGN: A systematic review of empirical studies examining the relation between volume and outcome of coronary artery bypass graft surgery. Studies were scored according to degree of adjustment for case mix. Above 200 procedures a year was regarded as high volume.
RESULTS: Fifteen studies were identified, all of which used observational data from the United States for 1972-92. Six were included in the analysis, one was included in a sensitivity analysis, and eight were excluded because of duplicate analyses of data sources and methods of reporting results. The seven studies analysed reported a reduced mortality with increased volume. Studies with better adjustment for case mix, however, indicated less reduction in mortality with increased volume (P = 0.04). The apparent advantages of higher volume also decreased over time (P < 0.001).
CONCLUSIONS: The evidence for reduced mortality in hospitals with a high volume of coronary artery bypass graft surgery is based entirely on observational studies. These studies may have over-estimated the benefit of increased volume because of poor adjustment for case mix. It signals the need for caution in interpreting the results of observational studies that examine the relation between volume and outcome.
To assess both qualitatively and quantitatively the impact of Public Reporting (PR) on clinical outcomes, we carried out a systematic review of published studies on this topic.
METHODS:
Pubmed, Web of Science and SCOPUS databases were searched to identify studies published from 1991 to 2014 that investigated the relationship between PR and clinical outcomes. Studies were considered eligible if they investigated the relationship between PR and clinical outcomes and comprehensively described the PR mechanism and the study design adopted. Among the clinical outcomes identified, meta-analysis was performed for overall mortality rate which quantitative data were exhaustively reported in a sufficient number of studies. Two reviewers conducted all data extraction independently and disagreements were resolved through discussion. The same reviewers evaluated also the quality of the studies using a GRADE approach.
RESULTS:
Twenty-seven studies were included. Mainly, the effect of PR on clinical outcomes was positive. Meta-analysis regarding overall mortality included, in a context of high heterogeneity, 10 studies with a total of 1,840,401 experimental events and 3,670,446 control events and resulted in a RR of 0.85 (95 % CI, 0.79-0.92).
CONCLUSIONS:
The introduction of PR programs at different levels of the healthcare sector is a challenging but rewarding public health strategy. Existing research covering different clinical outcomes supports the idea that PR could, in fact, stimulate providers to improve healthcare quality.
