Background Up to 30% of diagnostic imaging (DI) tests may be unnecessary, leading to increased healthcare costs and the possibility of patient harm. The primary objective of this systematic review was to assess the effect of audit and feedback (AF) interventions directed at healthcare providers on reducing image ordering. The secondary objective was to examine the effect of AF on the appropriateness of DI ordering. Methods Studies were identified using MEDLINE, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials and ClinicalTrials.gov registry on December 22nd, 2022. Studies were included if they were randomized control trials (RCTs), targeted healthcare professionals, and studied AF as the sole intervention or as the core component of a multi-faceted intervention. Risk of bias for each study was evaluated using the Cochrane risk of bias tool. Meta-analyses were completed using RevMan software and results were displayed in forest plots. Results Eleven RCTs enrolling 4311 clinicians or practices were included. AF interventions resulted in 1.5 fewer image test orders per 1000 patients seen than control interventions (95% confidence interval (CI) for the difference -2.6 to -0.4, p-value = 0.009). The effect of AF on appropriateness was not statistically significant, with a 3.2% (95% CI -1.5 to 7.7%, p-value = 0.18) greater likelihood of test orders being considered appropriate with AF vs control interventions. The strength of evidence was rated as moderate for the primary objective but was very low for the appropriateness outcome because of risk of bias, inconsistency in findings, indirectness, and imprecision. Conclusion AF interventions are associated with a modest reduction in total DI ordering with moderate certainty, suggesting some benefit of AF. Individual studies document effects of AF on image order appropriateness ranging from a non-significant trend toward worsening to a highly significant improvement, but the weighted average effect size from the meta-analysis is not statistically significant with very low certainty.
BACKGROUND: Musculoskeletal conditions require particular management skills. Identification of interventions which are effective in equipping general practitioners (GPs) with such necessary skills could translate to improved health outcomes for patients and reduced healthcare and societal costs.
OBJECTIVES: To determine the effectiveness of professional interventions for GPs that aim to improve the management of musculoskeletal conditions in primary care. SEARCH METHODS: We searched the Cochrane Central Register of Controlled Trials (CENTRAL), 2010, Issue 2; MEDLINE, Ovid (1950 - October 2013); EMBASE, Ovid (1980 - Ocotber 2013); CINAHL, EbscoHost (1980 - November 2013), and the EPOC Specialised Register. We conducted cited reference searches using ISI Web of Knowledge and Google Scholar; and handsearched selected issues of Arthritis and Rheumatism and Primary Care-Clinics in Office Practice. The latest search was conducted in November 2013.
SELECTION CRITERIA: We included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-and-after studies (CBAs) and interrupted time series (ITS) studies of professional interventions for GPs, taking place in a community setting, aiming to improve the management (including diagnosis and treatment) of musculoskeletal conditions and reporting any objective measure of GP behaviour, patient or economic outcomes. We considered professional interventions of any length, duration, intensity and complexity compared with active or inactive controls.
DATA COLLECTION AND ANALYSIS: Two review authors independently abstracted all data. We calculated the risk difference (RD) and risk ratio (RR) of compliance with desired practice for dichotomous outcomes, and the mean difference (MD) and standardised mean difference (SMD) for continuous outcomes. We investigated whether the direction of the targeted behavioural change affects the effectiveness of interventions.
MAIN RESULTS: Thirty studies met our inclusion criteria.From 11 studies on osteoporosis, meta-analysis of five studies (high-certainty evidence) showed that a combination of a GP alerting system on a patient's increased risk of osteoporosis and a patient-directed intervention (including patient education and a reminder to see their GP) improves GP behaviour with regard to diagnostic bone mineral density (BMD) testing and osteoporosis medication prescribing (RR 4.44; (95% confidence interval (CI) 3.54 to 5.55; 3 studies; 3,386 participants)) for BMD and RR 1.71 (95% CI 1.50 to 1.94; 5 studies; 4,223 participants) for osteoporosis medication. Meta-analysis of two studies showed that GP alerting on its own also probably improves osteoporosis guideline-consistent GP behaviour (RR 4.75 (95% CI 3.62 to 6.24; 3,047 participants)) for BMD and RR 1.52 (95% CI 1.26 to 1.84; 3.047 participants) for osteoporosis medication) and that adding the patient-directed component probably does not lead to a greater effect (RR 0.94 (95% CI 0.81 to 1.09; 2,995 participants)) for BMD and RR 0.93 (95% CI 0.79 to 1.10; 2,995 participants) for osteoporosis medication.Of the 10 studies on low back pain, seven showed that guideline dissemination and educational opportunities for GPs may lead to little or no improvement with regard to guideline-consistent GP behaviour. Two studies showed that the combination of guidelines and GP feedback on the total number of investigations requested may have an effect on GP behaviour and result in a slight reduction in the number of tests, while one of these studies showed that the combination of guidelines and GP reminders attached to radiology reports may result in a small but sustained reduction in the number of investigation requests.Of the four studies on osteoarthritis, one study showed that using educationally influential physicians may result in improvement in guideline-consistent GP behaviour. Another study showed slight improvements in patient outcomes (pain control) after training GPs on pain management.Of three studies on shoulder pain, one study reported that there may be little or no improvement in patient outcomes (functional capacity) after GP education on shoulder pain and injection training.Of two studies on other musculoskeletal conditions, one study on pain management showed that there may be worse patient outcomes (pain control) after GP training on the use of validated assessment scales.The 12 remaining studies across all musculoskeletal conditions showed little or no improvement in GP behaviour and patient outcomes.The direction of the targeted behaviour (i.e. increasing or decreasing a behaviour) does not seem to affect the effectiveness of an intervention. The majority of the studies did not investigate the potential adverse effects of the interventions and only three studies included a cost-effectiveness analysis.Overall, there were important methodological limitations in the body of evidence, with just a third of the studies reporting adequate allocation concealment and blinded outcome assessments. While our confidence in the pooled effect estimate of interventions for improving diagnostic testing and medication prescribing in osteoporosis is high, our confidence in the reported effect estimates in the remaining studies is low.
AUTHORS' CONCLUSIONS: There is good-quality evidence that a GP alerting system with or without patient-directed education on osteoporosis improves guideline-consistent GP behaviour, resulting in better diagnosis and treatment rates.Interventions such as GP reminder messages and GP feedback on performance combined with guideline dissemination may lead to small improvements in guideline-consistent GP behaviour with regard to low back pain, while GP education on osteoarthritis pain and the use of educationally influential physicians may lead to slight improvement in patient outcomes and guideline-consistent behaviour respectively. However, further studies are needed to ascertain the effectiveness of such interventions in improving GP behaviour and patient outcomes.
BACKGROUND: Rates of imaging for low-back pain are high and are associated with increased health care costs and radiation exposure as well as potentially poorer patient outcomes. We conducted a systematic review to investigate the effectiveness of interventions aimed at reducing the use of imaging for low-back pain.
METHODS: We searched MEDLINE, Embase, CINAHL and the Cochrane Central Register of Controlled Trials from the earliest records to June 23, 2014. We included randomized controlled trials, controlled clinical trials and interrupted time series studies that assessed interventions designed to reduce the use of imaging in any clinical setting, including primary, emergency and specialist care. Two independent reviewers extracted data and assessed risk of bias. We used raw data on imaging rates to calculate summary statistics. Study heterogeneity prevented meta-analysis.
RESULTS: A total of 8500 records were identified through the literature search. Of the 54 potentially eligible studies reviewed in full, 7 were included in our review. Clinical decision support involving a modified referral form in a hospital setting reduced imaging by 36.8% (95% confidence interval [CI] 33.2% to 40.5%). Targeted reminders to primary care physicians of appropriate indications for imaging reduced referrals for imaging by 22.5% (95% CI 8.4% to 36.8%). Interventions that used practitioner audits and feedback, practitioner education or guideline dissemination did not significantly reduce imaging rates. Lack of power within some of the included studies resulted in lack of statistical significance despite potentially clinically important effects.
INTERPRETATION: Clinical decision support in a hospital setting and targeted reminders to primary care doctors were effective interventions in reducing the use of imaging for low-back pain. These are potentially low-cost interventions that would substantially decrease medical expenditures associated with the management of low-back pain.
Abstract Laboratory and radiographic tests are often ordered unnecessarily. This excess testing has financial costs and is a burden on patients. We performed a systematic review to determine the effectiveness interventions to reduce test utilization by physicians. The MEDLINE and EMBASE databases were searched for the years 1946 through to September 2013 for English articles that had themes of test utilization and cost containment or optimization. Bibliographies of included papers were scanned to identify other potentially relevant studies. Our search resulted in 3236 articles of which 109 met the inclusion criteria of having an intervention aimed at reducing test utilization with results that could be expressed as a percent reduction in test use relative to the comparator. Each intervention was categorized into one or more non-exclusive category of education, audit and feedback, system based, or incentive or penalty. A rating of study quality was also performed. The percent reductions in test use ranged from a 99.7% reduction to a 27.7% increase in test use. Each category of intervention was effective in reducing test utilization. Heterogeneity between interventions, poor study quality, and limited time horizons makes generalizations difficult and calls into question the validity of results. Very few studies measure any patient safety or quality of care outcomes affected by reduced test use. There are numerous studies that use low investment strategies to reduce test utilization with one time changes in the ordering system. These low investment strategies are the most promising for achievable and durable reductions in inappropriate test use.
BACKGROUND: Demand management describes any method used to monitor, direct or regulate patient referrals. Several strategies have been developed to manage the referral of patients to secondary care, with interventions targeting primary care, specialist services, or infrastructure.
OBJECTIVE: This research aimed to conduct an inclusive systematic review and logic model synthesis in order to better understand factors impacting on the effectiveness of interventions targeting referral between primary and secondary medical health care.
DESIGN: The approach combined systematic review with logic modelling synthesis techniques to develop an evidence-based framework of factors influencing the pathway between interventions and system-wide changes.
SETTING: Primary health care.
MAIN OUTCOME MEASURES: Referral from primary to secondary care.
REVIEW METHODS: Systematic searches were undertaken to identify recent, relevant studies. Quality of individual studies was appraised, with consideration of overall strength of evidence. A narrative synthesis and logic model summary of the data was completed.
RESULTS: From a database of 8327 unique papers, 290 were included in the review. The intervention studies were grouped into four categories of education interventions (n = 50); process change interventions (n = 49); system change interventions (n = 38); and patient-focused interventions (n = 3). Effectiveness was assessed variously in these papers; however, there was a gap regarding the mechanisms whereby these interventions lead to demand management impacts. The findings suggest that, although individual-level interventions may be popular, the stronger evidence relates only to peer-review and feedback interventions. Process change interventions appeared to be more effective when the change resulted in the specialist being provided with more or better quality information about the patient. System changes including the community provision of specialist services by general practitioners, outreach provision by specialists and the return of inappropriate referrals appeared to have evidence of effect. The pathway whereby interventions might lead to service-wide impact was complex, with multiple factors potentially acting as barriers or facilitators to the change process. Factors related, first, to the doctor (including knowledge, attitudes and beliefs, and previous experiences of a service), second, to the patient (including condition and social factors) and, third, to the influence of the doctor–patient relationship. We also identified a number of potentially influential factors at a local level, such as perceived waiting times and the availability of a specialist. These elements are key factors in the pathway between an intervention and intended demand management outcomes influencing both applicability and effectiveness.
CONCLUSIONS: The findings highlight the complexity of the referral process and multiple elements that will impact on intervention outcomes and applicability to a local area. Any interventions seeking to change referral practice need to address factors relating to the individual practitioner, the patient and also the situation in which the referral is taking place. These conclusions apply especially to referral management in a UK context where this whole range of factors/issues lies well within the remit of the NHS. This work highlights that intermediate outcomes are important in the referral pathway. It is recommended that researchers include measure of these intermediate outcomes in their evaluation of intervention effectiveness in order to determine where blocks to or facilitators of system-wide impact may be occurring.
STUDY REGISTRATION: The study is registered as PROSPERO CRD42013004037.
FUNDING: The National Institute for Health Research Health Services and Delivery Research programme.
BACKGROUND: Imaging is commonly performed for musculoskeletal conditions. Identifying interventions to improve the appropriate use of imaging for musculoskeletal conditions could potentially result in improved health outcomes for patients and reduced health care costs. OBJECTIVES: To determine the effects of interventions that aim to improve the appropriate use of imaging for people with musculoskeletal conditions. SEARCH STRATEGY: We searched the Cochrane Effective Practice and Organisation of Care Group Specialised Register (June 2007), The Cochrane Central Register of Controlled Trials (The Cochrane Library 2007, Issue 2), MEDLINE (January 1966 to June 2007), EMBASE (January 1980 to June 2007) and CINAHL (January 1982 to June 2007). We also searched reference lists of included studies and relevant reviews. We undertook citation searches of all included studies, contacted authors of included studies, and contacted other experts in the field of effective professional practice. SELECTION CRITERIA: Randomised controlled trials, non-randomised controlled clinical trials and interrupted time-series analyses that evaluated interventions designed to improve the use of imaging for musculoskeletal symptoms. DATA COLLECTION AND ANALYSIS: Two review authors independently assessed risk of bias and extracted data. We contacted study authors for additional information. MAIN RESULTS: Twenty eight studies met our inclusion criteria. The majority of the studies were for the management of osteoporosis or low back pain, and most evaluated interventions aimed at health professionals. To improve the use of imaging in the management of osteoporosis, the effect of any type of intervention compared to no-intervention controls was modest (absolute improvement in bone mineral density test ordering +10%, IQR 0.0 to +27.7). Patient mediated, reminder, and organisational interventions appeared to have most potential for improving imaging use in osteoporosis. For low back pain studies, the most common intervention evaluated was distribution of educational materials and this showed varying effects. Other interventions in low back pain studies also showed variable effects. For other musculoskeletal conditions, distribution of educational materials, educational meetings and audit and feedback were not shown to be effective for changing imaging ordering behaviour. Across all conditions, increasing the number of intervention components did not increase effect. AUTHORS' CONCLUSIONS: For improving the use of imaging in osteoporosis, most professional interventions demonstrated benefit, and patient mediated, reminder, and organisational interventions appeared to have most potential for benefit. For low back pain studies interventions showed varying effects. For other musculoskeletal conditions, no firm conclusions can be drawn.
BACKGROUND: Audit and feedback (A&F) has long been used to improve quality of care, albeit with variable results. This meta-analytic study tested whether Feedback Intervention Theory, a framework from industrial/organizational psychology, explains the observed variability in health care A&F research.
METHOD: None
DATA SOURCE: studies cited by Jamtvedt's 2006 Cochrane systematic review of A&F, followed by database searches using the Cochrane review's search strategy to identify more recent studies.
INCLUSION CRITERIA: Cochrane review criteria, plus: presence of a treatment group receiving only A & F; a control group receiving no intervention; a quantitatively measurable outcome; minimum n of 10 per arm; sufficient statistics for effect size calculations. Moderators: presence of discouragement and praise; correct solution, attainment level, velocity, frequency, and normative information; feedback format (verbal, textual, graphic, public, computerized, group vs. individual); goal setting activity.
PROCEDURE: meta-analytic procedures using the Hedges-Olkin method.
RESULTS: Of 519 studies initially identified, 19 met all inclusion criteria. Studies were most often excluded due to the lack of a feedback-only arm. A&F has a modest, though significant positive effect on quality outcomes (d = 0.40, 95% confidence interval = +/-0.20); providing specific suggestions for improvement, written, and more frequent feedback strengthened this effect, whereas graphical and verbal feedback attenuated this effect.
CONCLUSIONS: A&F effectiveness is improved when feedback is delivered with specific suggestions for improvement, in writing, and frequently. Other feedback characteristics could also potentially improve effectiveness; however, research with stricter experimental controls is needed to identify the specific feedback characteristics that maximize its effectiveness.
In all areas of health research, including the alcohol and other drugs (AOD) field, funds are committed to developing and evaluating research and resources, yet little is invested into helping potential resource users understand, adopt and implement innovations. This study evaluated the effectiveness of two professional practice change interventions (reminders and feedback) that are designed to bridge the 'research-practice gap' by increasing knowledge and changing behaviour of health-care professionals and specialist AOD workers. We conducted a systematic review of general health, AOD and mental health literature (1966 to March 2005). Fourteen existing systematic reviews and 15 primary studies were assessed. Because few studies evaluated the effectiveness of reminders and feedback in the AOD context, evidence is drawn largely from the general health-care literature. Use of reminders and feedback is supported for a range of health behaviours. AOD-specific clinical behaviours that are most likely to be improved with the use of reminders or feedback include pharmacotherapy prescribing, AOD education, screening and counselling and monitoring/management of AOD treatment and/or related problems (e.g. depression). Reminders and feedback are effective strategies to facilitate professional practice change and have potential in the AOD field. However, further well-designed empirical studies are needed to assess fully the effectiveness of these professional practice change strategies in AOD-specific contexts.
Background: Many people advocate audit and feedback as a strategy for improving professional practice. The main results of an update of a Cochrane review on the effects of audit and feedback are reported. Data sources: The Cochrane Effective Practice and Organisation of Care Group's register up to January 2004 was searched. Randomised trials of audit and feedback that reported objectively measured professional practice in a healthcare setting or healthcare outcomes were included. Review methods: Data were independently extracted and the quality of studies were assessed by two reviewers. Quantitative, visual and qualitative analyses were undertaken. Main results: 118 trials are included in the review. In the primary analysis, 88 comparisons from 72 studies were included that compared any intervention in which audit and feedback was a component to no intervention. For dichotomous outcomes, the median-adjusted risk difference of compliance with desired practice was 5% (interquartile range 3-11). For continuous outcomes, the median-adjusted percentage change relative to control was 16% (interquartile range 5-37). Low baseline compliance with recommended practice and higher intensity of audit and feedback appeared to predict the effectiveness of audit and feedback. Conclusions: Audit and feedback can be effective in improving professional practice. The effects are generally small to moderate. The absolute effects of audit and feedback are likely to be larger when baseline adherence to recommended practice is low and intensity of audit and feedback is high.
BACKGROUND AND CONTEXT: There is a basis for the assumption that feedback can be used to enhance physicians' performance. Nevertheless, the findings of empirical studies of the impact of feedback on clinical performance have been equivocal.
OBJECTIVES: To summarize evidence related to the impact of assessment and feedback on physicians' clinical performance.
SEARCH STRATEGY: The authors searched the literature from 1966 to 2003 using MEDLINE, HealthSTAR, the Science Citation Index and eight other electronic databases. A total of 3702 citations were identified.
INCLUSION AND EXCLUSION CRITERIA: Empirical studies were selected involving the baseline measurement of physicians' performance and follow-up measurement after they received summaries of their performance.
DATA EXTRACTION: Data were extracted on research design, sample, dependent and independent variables using a written protocol.
DATA SYNTHESIS: A group of 220 studies involving primary data collection was identified. However, only 41 met all selection criteria and evaluated the independent effect of feedback on physician performance. Of these, 32 (74%) demonstrated a positive impact. Feedback was more likely to be effective when provided by an authoritative source over an extended period of time. Another subset of 132 studies examined the effect of feedback combined with other interventions such as educational programmes, practice guidelines and reminders. Of these, 106 studies (77%) demonstrated a positive impact. Two additional subsets of 29 feedback studies involving resident physicians in training and 18 studies examining proxy measures of physician performance across clinical sites or groups of patients were reviewed. The majority of these two subsets also reported that feedback had positive effects on performance.
HEADLINE RESULTS: Feedback can change physicians' clinical performance when provided systematically over multiple years by an authoritative, credible source.
CONCLUSIONS: The effects of formal assessment and feedback on physician performance are influenced by the source and duration of feedback. Other factors, such as physicians' active involvement in the process, the amount of information reported, the timing and amount of feedback, and other concurrent interventions, such as education, guidelines, reminder systems and incentives, also appear to be important. However, the independent contributions of these interventions have not been well documented in controlled studies. It is recommended that the designers of future theoretical as well as practical studies of feedback separate the effects of feedback from other concurrent interventions.
Background Up to 30% of diagnostic imaging (DI) tests may be unnecessary, leading to increased healthcare costs and the possibility of patient harm. The primary objective of this systematic review was to assess the effect of audit and feedback (AF) interventions directed at healthcare providers on reducing image ordering. The secondary objective was to examine the effect of AF on the appropriateness of DI ordering. Methods Studies were identified using MEDLINE, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials and ClinicalTrials.gov registry on December 22nd, 2022. Studies were included if they were randomized control trials (RCTs), targeted healthcare professionals, and studied AF as the sole intervention or as the core component of a multi-faceted intervention. Risk of bias for each study was evaluated using the Cochrane risk of bias tool. Meta-analyses were completed using RevMan software and results were displayed in forest plots. Results Eleven RCTs enrolling 4311 clinicians or practices were included. AF interventions resulted in 1.5 fewer image test orders per 1000 patients seen than control interventions (95% confidence interval (CI) for the difference -2.6 to -0.4, p-value = 0.009). The effect of AF on appropriateness was not statistically significant, with a 3.2% (95% CI -1.5 to 7.7%, p-value = 0.18) greater likelihood of test orders being considered appropriate with AF vs control interventions. The strength of evidence was rated as moderate for the primary objective but was very low for the appropriateness outcome because of risk of bias, inconsistency in findings, indirectness, and imprecision. Conclusion AF interventions are associated with a modest reduction in total DI ordering with moderate certainty, suggesting some benefit of AF. Individual studies document effects of AF on image order appropriateness ranging from a non-significant trend toward worsening to a highly significant improvement, but the weighted average effect size from the meta-analysis is not statistically significant with very low certainty.
