OBJECTIVES: To assess the impact of pay for performance programmes on inequalities in the quality of health care in relation to age, sex, ethnicity and socioeconomic status.
METHODS: Systematic search and appraisal of experimental or observational studies that assessed quantitatively the impact of a monetary incentive on health care inequalities. We searched published articles in English identified in the MEDLINE, EMBASE, PsycINFO and Cochrane databases.
RESULTS: Twenty-two studies were identified, 20 of which were conducted in the United Kingdom and examined the impact of the Quality and Outcomes Framework. Sixteen studies used practice level data rather than patient level data. Socioeconomic status was the most frequently examined inequality; age, sex and ethnic inequalities were less frequently assessed. There was some weak evidence that the use of financial incentives reduced inequalities in chronic disease management between socioeconomic groups. Inequalities in chronic disease management between age, sex and ethnic groups persisted after the use of such incentives.
CONCLUSION: Inequalities in chronic disease management have largely persisted after the introduction of the Quality and Outcome Framework. Pay for performance programmes should be designed to reduce inequalities as well as improve the overall quality of care.
OBJECTIVE: To determine (1) how medical registries provide information feedback to health care professionals, (2) whether this feedback has any effect on the quality of care and (3) what the barriers and success factors are to the effectiveness of feedback.
DATA SOURCES: Original articles in English found in MEDLINE Pubmed covering the period January 1990 to August 2007.
REVIEW METHOD: Titles and abstracts of 6223 original articles were independently screened by two reviewers to determine relevance for further review.
DATA EXTRACTION AND ANALYSIS: We used a standardized data abstraction form to collect information on the feedback initiatives and their effectiveness. The effect of the feedback was only described for analytic papers, i.e. papers that attempted to objectively quantify the effect on the quality of care and to relate this effect to feedback as an intervention. For analysis of the effectiveness, we categorized the initiatives based on the number of elements added to the feedback.
RESULTS: We included 53 papers, describing 50 feedback initiatives, of which 39 were part of a multifaceted approach. Our results confirm previous research findings that adding elements to a feedback strategy positively influences its effectiveness. We found 22 analytic studies, four of which found a positive effect on all outcome measures, eight found a mix of positive- and no effects and ten did not find any effects (neither positive nor negative). Of the 43 process of care measures evaluated in the analytic studies, 26 were positively affected by the feedback initiative. Of the 36 evaluated outcome of care measures, five were positively affected. The most frequently mentioned factors influencing the effectiveness of the feedback were: (trust in) quality of the data, motivation of the recipients, organizational factors and outcome expectancy of the feedback recipients.
CONCLUSIONS: The literature on methods and effects of information feedback by medical registries is heterogeneous, making it difficult to draw definite conclusions on its effectiveness. However, the positive effects cannot be discarded. Although our review confirms findings from previous studies that process of care measures are more positively influenced by feedback than outcome of care measures, further research should attempt to identify outcome of care measures that are sensitive to behaviour change as a result of feedback strategies. Furthermore, future studies evaluating the effectiveness of feedback should include a more extensive description of their intervention in order to increase the reproducibility of feedback initiatives and the generalizability of the results.
BACKGROUND: Audit and feedback (A&F) has long been used to improve quality of care, albeit with variable results. This meta-analytic study tested whether Feedback Intervention Theory, a framework from industrial/organizational psychology, explains the observed variability in health care A&F research.
METHOD: None
DATA SOURCE: studies cited by Jamtvedt's 2006 Cochrane systematic review of A&F, followed by database searches using the Cochrane review's search strategy to identify more recent studies.
INCLUSION CRITERIA: Cochrane review criteria, plus: presence of a treatment group receiving only A & F; a control group receiving no intervention; a quantitatively measurable outcome; minimum n of 10 per arm; sufficient statistics for effect size calculations. Moderators: presence of discouragement and praise; correct solution, attainment level, velocity, frequency, and normative information; feedback format (verbal, textual, graphic, public, computerized, group vs. individual); goal setting activity.
PROCEDURE: meta-analytic procedures using the Hedges-Olkin method.
RESULTS: Of 519 studies initially identified, 19 met all inclusion criteria. Studies were most often excluded due to the lack of a feedback-only arm. A&F has a modest, though significant positive effect on quality outcomes (d = 0.40, 95% confidence interval = +/-0.20); providing specific suggestions for improvement, written, and more frequent feedback strengthened this effect, whereas graphical and verbal feedback attenuated this effect.
CONCLUSIONS: A&F effectiveness is improved when feedback is delivered with specific suggestions for improvement, in writing, and frequently. Other feedback characteristics could also potentially improve effectiveness; however, research with stricter experimental controls is needed to identify the specific feedback characteristics that maximize its effectiveness.
BACKGROUND: Previous reviews have shown inconsistent effects of publicly reported performance data on quality of care, but many new studies have become available in the 7 years since the last systematic review. PURPOSE: To synthesize the evidence for using publicly reported performance data to improve quality. DATA SOURCES: Web of Science, MEDLINE, EconLit, and Wilson Business Periodicals (1999-2006) and independent review of articles (1986-1999) identified in a previous systematic review. Only sources published in English were included. STUDY SELECTION: Peer-reviewed articles assessing the effects of public release of performance data on selection of providers, quality improvement activity, clinical outcomes (effectiveness, patient safety, and patient-centeredness), and unintended consequences. DATA EXTRACTION: Data on study participants, reporting system or level, study design, selection of providers, quality improvement activity, outcomes, and unintended consequences were extracted. DATA SYNTHESIS: Forty-five articles published since 1986 (27 of which were published since 1999) evaluated the impact of public reporting on quality. Many focus on a select few reporting systems. Synthesis of data from 8 health plan-level studies suggests modest association between public reporting and plan selection. Synthesis of 11 studies, all hospital-level, suggests stimulation of quality improvement activity. Review of 9 hospital-level and 7 individual provider-level studies shows inconsistent association between public reporting and selection of hospitals and individual providers. Synthesis of 11 studies, primarily hospital-level, indicates inconsistent association between public reporting and improved effectiveness. Evidence on the impact of public reporting on patient safety and patient-centeredness is scant. LIMITATIONS: Heterogeneity made comparisons across studies challenging. Only peer-reviewed, English-language articles were included. CONCLUSION: Evidence is scant, particularly about individual providers and practices. Rigorous evaluation of many major public reporting systems is lacking. Evidence suggests that publicly releasing performance data stimulates quality improvement activity at the hospital level. The effect of public reporting on effectiveness, safety, and patient-centeredness remains uncertain.
PURPOSE: The purpose of this study was to identify and analyze research into accreditation and accreditation processes.
DATA SOURCES: A multi-method, systematic review of the accreditation literature was conducted from March to May 2007. The search identified articles researching accreditation. Discussion or commentary pieces were excluded.
STUDY SELECTION: From the initial identification of over 3000 abstracts, 66 studies that met the search criteria by empirically examining accreditation were selected. DATA EXTRACTION AND RESULTS OF DATA SYNTHESIS: The 66 studies were retrieved and analyzed. The results, examining the impact or effectiveness of accreditation, were classified into 10 categories: professions' attitudes to accreditation, promote change, organizational impact, financial impact, quality measures, program assessment, consumer views or patient satisfaction, public disclosure, professional development and surveyor issues.
RESULTS: The analysis reveals a complex picture. In two categories consistent findings were recorded: promote change and professional development. Inconsistent findings were identified in five categories: professions' attitudes to accreditation, organizational impact, financial impact, quality measures and program assessment. The remaining three categories-consumer views or patient satisfaction, public disclosure and surveyor issues-did not have sufficient studies to draw any conclusion. The search identified a number of national health care accreditation organizations engaged in research activities.
CONCLUSION: The health care accreditation industry appears to be purposefully moving towards constructing the evidence to ground our understanding of accreditation.
Despite more than a decade of bench-marking and public reporting of quality problems in the health care sector, changes in medical practice have been slow to materialize. To accelerate quality improvement, many private and public payers have begun to offer financial incentives to physicians and hospitals based on their performance on clinical and service quality measures. The authors review the empirical literature on paying for quality in health care and comparable interventions in other sectors. They find little evidence to support the effectiveness of paying for quality. The absence of findings for an effect may be attributable to the small size of the bonuses studied and the fact that payers often accounted for only a fraction of the targeted provider's panel. Even in non-health settings, however, where the institutional features are more favorable to a positive impact, the literature contains mixed results on the effectiveness of analogous pay-for-performance schemes.
BACKGROUND: Most physicians and hospitals are paid the same regardless of the quality of the health care they provide. This produces no financial incentives and, in some cases, produces disincentives for quality. Increasing numbers of programs link payment to performance. PURPOSE: To systematically review studies assessing the effect of explicit financial incentives for improved performance on measures of health care quality. DATA SOURCES: PubMed search of English-language literature (1 January 1980 to 14 November 2005), and reference lists of retrieved articles. STUDY SELECTION: Empirical studies of the relationship between explicit financial incentives designed to improve health care quality and a quantitative measure of health care quality. DATA EXTRACTION: The authors categorized studies according to the level of the incentive (individual physician, provider group, or health care payment system) and the type of quality measure rewarded. DATA SYNTHESIS: Thirteen of 17 studies examined process-of-care quality measures, most of which were for preventive services. Five of the 6 studies of physician-level financial incentives and 7 of the 9 studies of provider group-level financial incentives found partial or positive effects on measures of quality. One of the 2 studies of incentives at the payment-system level found a positive effect on access to care, and 1 showed evidence of a negative effect on access to care for the sickest patients. In all, 4 studies suggested unintended effects of incentives. The authors found no studies examining the optimal duration of financial incentives for quality or the persistence of their effects after termination. Only 1 study addressed cost-effectiveness. LIMITATIONS: Few empirical studies of explicit financial incentives for quality were available for review. CONCLUSIONS: Ongoing monitoring of incentive programs is critical to determine the effectiveness of financial incentives and their possible unintended effects on quality of care. Further research is needed to guide implementation of financial incentives and to assess their cost-effectiveness.
PURPOSE: The relative effectiveness of the diverse approaches used to promote preventive care activities, such as cancer screening and adult immunization, is unknown. Despite many high-quality published studies, practices and policymakers attempting to improve preventive care have little definitive information on which to base decisions. Thus, we quantitatively assessed the relative effectiveness of previously studied approaches for improving adherence to adult immunization and cancer screening guidelines. DATA SOURCES: MEDLINE, the Cochrane Effective Practice and Organization of Care Review Group register, previous systematic reviews, and the Medicare Health Care Quality Improvement Project database. STUDY SELECTION: Controlled clinical trials that assessed interventions to increase use of immunizations for influenza and pneumococcal pneumonia and screening for colon, breast, and cervical cancer in adults. DATA EXTRACTION: Two reviewers independently extracted data on characteristics and outcomes from unmasked articles. Intervention components to increase use of services were classified as reminder, feedback, education, financial incentive, legislative action, organizational change, or mass media campaign. DATA SYNTHESIS: Of 552 abstracts and articles, 108 met the inclusion criteria. To assess the effect of intervention components, meta-regression models were developed for immunizations and each cancer screening service by using 81 studies with a usual care or control group. The most potent intervention types involved organizational change (the adjusted odds ratios for increased use of services from organizational change ranged from 2.47 to 17.6). Organizational change interventions included the use of separate clinics devoted to prevention, use of a planned care visit for prevention, or designation of nonphysician staff to do specific prevention activities. The next most effective intervention components were patient financial incentives (adjusted odds ratios, 1.82 to 3.42) and patient reminders (adjusted odds ratios, 1.74 to 2.75); the adjusted odds ratios ranged from 1.29 to 1.53 for patient education and from 1.10 to 1.76 for feedback. CONCLUSIONS: Rates of adult immunization and cancer screening are most likely to improve when a health care organization supports performance of these activities through organizational changes in staffing and clinical procedures. Involving patients in self-management through patient financial incentives and reminders is also likely to positively affect performance.
CONTEXT: Information about the performance of hospitals, health professionals, and health care organizations has been made public in the United States for more than a decade. The expected gains of public disclosure have not been made clear, and both the benefits and potential risks have received minimal empirical investigation.
OBJECTIVE: To summarize the empirical evidence concerning public disclosure of performance data, relate the results to the potential gains, and identify areas requiring further research.
DATA SOURCES: A literature search was conducted on MEDLINE and EMBASE databases for articles published between January 1986 and October 1999 in peer-reviewed journals. Review of citations, public documents, and expert advice was conducted to identify studies not found in the electronic databases.
STUDY SELECTION: Descriptive, observational, or experimental evaluations of US reporting systems were selected for inclusion.
DATA EXTRACTION: Included studies were organized based on use of public data by consumers, purchasers, physicians, and hospitals; impact on quality of care outcomes; and costs.
DATA SYNTHESIS: Seven US reporting systems have been the subject of published empirical evaluations. Descriptive and observational methods predominate. Consumers and purchasers rarely search out the information and do not understand or trust it; it has a small, although increasing, impact on their decision making. Physicians are skeptical about such data and only a small proportion makes use of it. Hospitals appear to be most responsive to the data. In a limited number of studies, the publication of performance data has been associated with an improvement in health outcomes.
CONCLUSIONS: There are several potential gains from the public disclosure of performance data, but use of the information by provider organizations for quality improvement may be the most productive area for further research.
To assess the impact of pay for performance programmes on inequalities in the quality of health care in relation to age, sex, ethnicity and socioeconomic status.
METHODS:
Systematic search and appraisal of experimental or observational studies that assessed quantitatively the impact of a monetary incentive on health care inequalities. We searched published articles in English identified in the MEDLINE, EMBASE, PsycINFO and Cochrane databases.
RESULTS:
Twenty-two studies were identified, 20 of which were conducted in the United Kingdom and examined the impact of the Quality and Outcomes Framework. Sixteen studies used practice level data rather than patient level data. Socioeconomic status was the most frequently examined inequality; age, sex and ethnic inequalities were less frequently assessed. There was some weak evidence that the use of financial incentives reduced inequalities in chronic disease management between socioeconomic groups. Inequalities in chronic disease management between age, sex and ethnic groups persisted after the use of such incentives.
CONCLUSION:
Inequalities in chronic disease management have largely persisted after the introduction of the Quality and Outcome Framework. Pay for performance programmes should be designed to reduce inequalities as well as improve the overall quality of care.
