BACKGROUND: Clinical practice does not always reflect best practice and evidence, partly because of unconscious acts of omission, information overload, or inaccessible information. Reminders may help clinicians overcome these problems by prompting them to recall information that they already know or would be expected to know and by providing information or guidance in a more accessible and relevant format, at a particularly appropriate time. This is an update of a previously published review.
OBJECTIVES: To evaluate the effects of reminders automatically generated through a computerized system (computer-generated) and delivered on paper to healthcare professionals on quality of care (outcomes related to healthcare professionals' practice) and patient outcomes (outcomes related to patients' health condition).
SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, six other databases and two trials registers up to 21 September 2016 together with reference checking, citation searching and contact with study authors to identify additional studies.
SELECTION CRITERIA: We included individual- or cluster-randomized and non-randomized trials that evaluated the impact of computer-generated reminders delivered on paper to healthcare professionals, alone (single-component intervention) or in addition to one or more co-interventions (multi-component intervention), compared with usual care or the co-intervention(s) without the reminder component.
DATA COLLECTION AND ANALYSIS: Review authors working in pairs independently screened studies for eligibility and abstracted data. For each study, we extracted the primary outcome when it was defined or calculated the median effect size across all reported outcomes. We then calculated the median improvement and interquartile range (IQR) across included studies using the primary outcome or median outcome as representative outcome. We assessed the certainty of the evidence according to the GRADE approach.
MAIN RESULTS: We identified 35 studies (30 randomized trials and five non-randomized trials) and analyzed 34 studies (40 comparisons). Twenty-nine studies took place in the USA and six studies took place in Canada, France, Israel, and Kenya. All studies except two took place in outpatient care. Reminders were aimed at enhancing compliance with preventive guidelines (e.g. cancer screening tests, vaccination) in half the studies and at enhancing compliance with disease management guidelines for acute or chronic conditions (e.g. annual follow-ups, laboratory tests, medication adjustment, counseling) in the other half.Computer-generated reminders delivered on paper to healthcare professionals, alone or in addition to co-intervention(s), probably improves quality of care slightly compared with usual care or the co-intervention(s) without the reminder component (median improvement 6.8% (IQR: 3.8% to 17.5%); 34 studies (40 comparisons); moderate-certainty evidence).Computer-generated reminders delivered on paper to healthcare professionals alone (single-component intervention) probably improves quality of care compared with usual care (median improvement 11.0% (IQR 5.4% to 20.0%); 27 studies (27 comparisons); moderate-certainty evidence). Adding computer-generated reminders delivered on paper to healthcare professionals to one or more co-interventions (multi-component intervention) probably improves quality of care slightly compared with the co-intervention(s) without the reminder component (median improvement 4.0% (IQR 3.0% to 6.0%); 11 studies (13 comparisons); moderate-certainty evidence).We are uncertain whether reminders, alone or in addition to co-intervention(s), improve patient outcomes as the certainty of the evidence is very low (n = 6 studies (seven comparisons)). None of the included studies reported outcomes related to harms or adverse effects of the intervention.
AUTHORS' CONCLUSIONS: There is moderate-certainty evidence that computer-generated reminders delivered on paper to healthcare professionals probably slightly improves quality of care, in terms of compliance with preventive guidelines and compliance with disease management guidelines. It is uncertain whether reminders improve patient outcomes because the certainty of the evidence is very low. The heterogeneity of the reminder interventions included in this review also suggests that reminders can probably improve quality of care in various settings under various conditions.
OBJECTIVES: To identify factors that differentiate between effective and ineffective computerised clinical decision support systems in terms of improvements in the process of care or in patient outcomes.
DESIGN: Meta-regression analysis of randomised controlled trials.
DATA SOURCES: A database of features and effects of these support systems derived from 162 randomised controlled trials identified in a recent systematic review. Trialists were contacted to confirm the accuracy of data and to help prioritise features for testing.
MAIN OUTCOME MEASURES: "Effective" systems were defined as those systems that improved primary (or 50% of secondary) reported outcomes of process of care or patient health. Simple and multiple logistic regression models were used to test characteristics for association with system effectiveness with several sensitivity analyses.
RESULTS: Systems that presented advice in electronic charting or order entry system interfaces were less likely to be effective (odds ratio 0.37, 95% confidence interval 0.17 to 0.80). Systems more likely to succeed provided advice for patients in addition to practitioners (2.77, 1.07 to 7.17), required practitioners to supply a reason for over-riding advice (11.23, 1.98 to 63.72), or were evaluated by their developers (4.35, 1.66 to 11.44). These findings were robust across different statistical methods, in internal validation, and after adjustment for other potentially important factors.
CONCLUSIONS: We identified several factors that could partially explain why some systems succeed and others fail. Presenting decision support within electronic charting or order entry systems are associated with failure compared with other ways of delivering advice. Odds of success were greater for systems that required practitioners to provide reasons when over-riding advice than for systems that did not. Odds of success were also better for systems that provided advice concurrently to patients and practitioners. Finally, most systems were evaluated by their own developers and such evaluations were more likely to show benefit than those conducted by a third party.
PURPOSE: Influenza and pneumococcal vaccination rates remain below national targets. We systematically reviewed the effectiveness of quality improvement interventions for increasing the rates of influenza and pneumococcal vaccinations among community-dwelling adults.
METHODS: We included randomized and nonrandomized studies with a concurrent control group. We estimated pooled odds ratios using random effects models, and used the Downs and Black tool to assess the quality of included studies.
RESULTS: Most studies involved elderly primary care patients. Interventions were associated with improvements in the rates of any vaccination (111 comparisons in 77 studies, pooled odds ratio [OR] = 1.61, 95% CI, 1.49-1.75), and influenza (93 comparisons, 65 studies, OR = 1.46, 95% CI, 1.35-1.57) and pneumococcal (58 comparisons, 35 studies, OR = 2.01, 95% CI, 1.72-2.3) vaccinations. Interventions that appeared effective were patient financial incentives (influenza only), audit and feedback (influenza only), clinician reminders, clinician financial incentives (influenza only), team change, patient outreach, delivery site changes (influenza only), clinician education (pneumococcus only), and case management (pneumococcus only). Patient outreach was more effective if personal contact was involved. Team changes were more effective where nurses administered influenza vaccinations independently. Heterogeneity in some pooled odds ratios was high, however, and funnel plots showed signs of potential publication bias. Study quality varied but was not associated with outcomes.
CONCLUSIONS: Quality improvement interventions, especially those that assign vaccination responsibilities to nonphysician personnel or that activate patients through personal contact, can modestly improve vaccination rates in community-dwelling adults. To meet national policy targets, more-potent interventions should be developed and evaluated.
BACKGROUND: Despite increasing emphasis on the role of clinical decision-support systems (CDSSs) for improving care and reducing costs, evidence to support widespread use is lacking.
PURPOSE: To evaluate the effect of CDSSs on clinical outcomes, health care processes, workload and efficiency, patient satisfaction, cost, and provider use and implementation.
DATA SOURCES: MEDLINE, CINAHL, PsycINFO, and Web of Science through January 2011.
STUDY SELECTION: Investigators independently screened reports to identify randomized trials published in English of electronic CDSSs that were implemented in clinical settings; used by providers to aid decision making at the point of care; and reported clinical, health care process, workload, relationship-centered, economic, or provider use outcomes.
DATA EXTRACTION: Investigators extracted data about study design, participant characteristics, interventions, outcomes, and quality.
DATA SYNTHESIS: 148 randomized, controlled trials were included. A total of 128 (86%) assessed health care process measures, 29 (20%) assessed clinical outcomes, and 22 (15%) measured costs. Both commercially and locally developed CDSSs improved health care process measures related to performing preventive services (n= 25; odds ratio [OR], 1.42 [95% CI, 1.27 to 1.58]), ordering clinical studies (n= 20; OR, 1.72 [CI, 1.47 to 2.00]), and prescribing therapies (n= 46; OR, 1.57 [CI, 1.35 to 1.82]). Few studies measured potential unintended consequences or adverse effects.
LIMITATIONS: Studies were heterogeneous in interventions, populations, settings, and outcomes. Publication bias and selective reporting cannot be excluded.
CONCLUSION: Both commercially and locally developed CDSSs are effective at improving health care process measures across diverse settings, but evidence for clinical, economic, workload, and efficiency outcomes remains sparse. This review expands knowledge in the field by demonstrating the benefits of CDSSs outside of experienced academic centers.
PRIMARY FUNDING SOURCE: Agency for Healthcare Research and Quality.
BACKGROUND: Underuse and overuse of diagnostic tests have important implications for health outcomes and costs. Decision support technology purports to optimize the use of diagnostic tests in clinical practice. The objective of this review was to assess whether computerized clinical decision support systems (CCDSSs) are effective at improving ordering of tests for diagnosis, monitoring of disease, or monitoring of treatment. The outcome of interest was effect on the diagnostic test-ordering behavior of practitioners.
METHODS: We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews database, Inspec, and reference lists for eligible articles published up to January 2010. We included randomized controlled trials comparing the use of CCDSSs to usual practice or non-CCDSS controls in clinical care settings. Trials were eligible if at least one component of the CCDSS gave suggestions for ordering or performing a diagnostic procedure. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of test ordering outcomes.
RESULTS: Thirty-five studies were identified, with significantly higher methodological quality in those published after the year 2000 (p = 0.002). Thirty-three trials reported evaluable data on diagnostic test ordering, and 55% (18/33) of CCDSSs improved testing behavior overall, including 83% (5/6) for diagnosis, 63% (5/8) for treatment monitoring, 35% (6/17) for disease monitoring, and 100% (3/3) for other purposes. Four of the systems explicitly attempted to reduce test ordering rates and all succeeded. Factors of particular interest to decision makers include costs, user satisfaction, and impact on workflow but were rarely investigated or reported.
CONCLUSIONS: Some CCDSSs can modify practitioner test-ordering behavior. To better inform development and implementation efforts, studies should describe in more detail potentially important factors such as system design, user interface, local context, implementation strategy, and evaluate impact on user satisfaction and workflow, costs, and unintended consequences.
BACKGROUND: The use of computerized clinical decision support systems (CCDSSs) may improve chronic disease management, which requires recurrent visits to multiple health professionals, ongoing disease and treatment monitoring, and patient behavior modification. The objective of this review was to determine if CCDSSs improve the processes of chronic care (such as diagnosis, treatment, and monitoring of disease) and associated patient outcomes (such as effects on biomarkers and clinical exacerbations).
METHODS: We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews database, Inspec, and reference lists for potentially eligible articles published up to January 2010. We included randomized controlled trials that compared the use of CCDSSs to usual practice or non-CCDSS controls. Trials were eligible if at least one component of the CCDSS was designed to support chronic disease management. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of relevant outcomes.
RESULTS: Of 55 included trials, 87% (n = 48) measured system impact on the process of care and 52% (n = 25) of those demonstrated statistically significant improvements. Sixty-five percent (36/55) of trials measured impact on, typically, non-major (surrogate) patient outcomes, and 31% (n = 11) of those demonstrated benefits. Factors of interest to decision makers, such as cost, user satisfaction, system interface and feature sets, unique design and deployment characteristics, and effects on user workflow were rarely investigated or reported.
CONCLUSIONS: A small majority (just over half) of CCDSSs improved care processes in chronic disease management and some improved patient health. Policy makers, healthcare administrators, and practitioners should be aware that the evidence of CCDSS effectiveness is limited, especially with respect to the small number and size of studies measuring patient outcomes.
PURPOSE: Influenza and pneumococcal vaccination rates remain below national targets. We systematically reviewed the effectiveness of quality improvement interventions for increasing the rates of influenza and pneumococcal vaccinations among community-dwelling adults. METHODS: We included randomized and nonrandomized studies with a concurrent control group. We estimated pooled odds ratios using random effects models, and used the Downs and Black tool to assess the quality of included studies. RESULTS: Most studies involved elderly primary care patients. Interventions were associated with improvements in the rates of any vaccination (111 comparisons in 77 studies, pooled odds ratio [OR] = 1.61, 95% CI, 1.49-1.75), and influenza (93 comparisons, 65 studies, OR = 1.46, 95% CI, 1.35-1.57) and pneumococcal (58 comparisons, 35 studies, OR = 2.01, 95% CI, 1.72-2.3) vaccinations. Interventions that appeared effective were patient financial incentives (influenza only), audit and feedback (influenza only), clinician reminders, clinician financial incentives (influenza only), team change, patient outreach, delivery site changes (influenza only), clinician education (pneumococcus only), and case management (pneumococcus only). Patient outreach was more effective if personal contact was involved. Team changes were more effective where nurses administered influenza vaccinations independently. Heterogeneity in some pooled odds ratios was high, however, and funnel plots showed signs of potential publication bias. Study quality varied but was not associated with outcomes. CONCLUSIONS: Quality improvement interventions, especially those that assign vaccination responsibilities to nonphysician personnel or that activate patients through personal contact, can modestly improve vaccination rates in community-dwelling adults. To meet national policy targets, more-potent interventions should be developed and evaluated.
Most major medical organizations recommend routine screening for breast, cervical, and colorectal cancers. Screening can lead to early detection of these cancers, resulting in reduced mortality. Yet, not all people who should be screened are screened regularly or, in some cases, ever. This report presents results of systematic reviews of effectiveness, applicability, economic efficiency, barriers to implementation, and other harms or benefits of provider reminder/recall interventions to increase screening for breast, cervical, and colorectal cancers. These interventions involve using systems to inform healthcare providers when individual clients are due (reminder) or overdue (recall) for specific cancer screening tests. Evidence in this review of studies published from 1986 through 2004 indicates that reminder/recall systems can effectively increase screening with mammography, Pap, fecal occult blood tests, and flexible sigmoidoscopy. Additional research is needed to determine if provider reminder/recall systems are effective in increasing colorectal cancer screening by colonoscopy. Specific areas for further research are also suggested.
Preventive care measures remain underutilized despite recommendations to increase their use. The objective of this review was to examine the characteristics, types, and effects of paper- and computer-based interventions for preventive care measures. The study provides an update to a previous systematic review. We included randomized controlled trials that implemented a physician reminder and measured the effects on the frequency of providing preventive care. Of the 1,535 articles identified, 28 met inclusion criteria and were combined with the 33 studies from the previous review. The studies involved 264 preventive care interventions, 4,638 clinicians and 144,605 patients. Implementation strategies included combined paper-based with computer generated reminders in 34 studies (56%), paper-based reminders in 19 studies (31%), and fully computerized reminders in 8 studies (13%). The average increase for the three strategies in delivering preventive care measures ranged between 12% and 14%. Cardiac care and smoking cessation reminders were most effective. Computer-generated prompts were the most commonly implemented reminders. Clinician reminders are a successful approach for increasing the rates of delivering preventive care; however, their effectiveness remains modest. Despite increased implementation of electronic health records, randomized controlled trials evaluating computerized reminder systems are infrequent.
Introduction: We undertook a meta-analysis of the Continuing Medical Education (CME) outcome literature to examine the effect of moderator variables on physician knowledge, performance, and patient outcomes. Methods: A literature search of MEDLINE and ERIC was conducted for randomized controlled trials and experimental design studies of CME outcomes in which physicians were a major group. CME moderator variables included the types of intervention, the types and number of participants, time, and the number of intervention sessions held over time. Results: Thirty-one studies met the eligibility criteria, generating 61 interventions. The overall sample-size weighted effect size for all 61 interventions was r = 0.28 (0.18). The analysis of CME moderator variables showed that active and mixed methods had medium effect sizes (r = 0.33 [0.33], r = 0.33 [0.26], respectively), and passive methods had a small effect size (r = 0.20 [0.16], confidence interval 0.15, 0.26). There was a positive correlation between the effect size and the length of the interventions (r = 0.33) and between multiple interventions over time (r=0.36). There was a negative correlation between the effect size and programs that involved multiple disciplines (r=-0.18) and the number of participants (r=-0.13). The correlation between the effect size and the length of time for outcome assessment was negative (r = -0.31). Discussion: The meta-analysis suggests that the effect size of CME on physician knowledge is a medium one; however, the effect size is small for physician performance and patient outcome. The examination of moderator variables shows there is a larger effect size when the interventions are interactive, use multiple methods, and are designed for a small group of physicians from a single discipline.
Clinical practice does not always reflect best practice and evidence, partly because of unconscious acts of omission, information overload, or inaccessible information. Reminders may help clinicians overcome these problems by prompting them to recall information that they already know or would be expected to know and by providing information or guidance in a more accessible and relevant format, at a particularly appropriate time. This is an update of a previously published review.
OBJECTIVES:
To evaluate the effects of reminders automatically generated through a computerized system (computer-generated) and delivered on paper to healthcare professionals on quality of care (outcomes related to healthcare professionals' practice) and patient outcomes (outcomes related to patients' health condition).
SEARCH METHODS:
We searched CENTRAL, MEDLINE, Embase, six other databases and two trials registers up to 21 September 2016 together with reference checking, citation searching and contact with study authors to identify additional studies.
SELECTION CRITERIA:
We included individual- or cluster-randomized and non-randomized trials that evaluated the impact of computer-generated reminders delivered on paper to healthcare professionals, alone (single-component intervention) or in addition to one or more co-interventions (multi-component intervention), compared with usual care or the co-intervention(s) without the reminder component.
DATA COLLECTION AND ANALYSIS:
Review authors working in pairs independently screened studies for eligibility and abstracted data. For each study, we extracted the primary outcome when it was defined or calculated the median effect size across all reported outcomes. We then calculated the median improvement and interquartile range (IQR) across included studies using the primary outcome or median outcome as representative outcome. We assessed the certainty of the evidence according to the GRADE approach.
MAIN RESULTS:
We identified 35 studies (30 randomized trials and five non-randomized trials) and analyzed 34 studies (40 comparisons). Twenty-nine studies took place in the USA and six studies took place in Canada, France, Israel, and Kenya. All studies except two took place in outpatient care. Reminders were aimed at enhancing compliance with preventive guidelines (e.g. cancer screening tests, vaccination) in half the studies and at enhancing compliance with disease management guidelines for acute or chronic conditions (e.g. annual follow-ups, laboratory tests, medication adjustment, counseling) in the other half.Computer-generated reminders delivered on paper to healthcare professionals, alone or in addition to co-intervention(s), probably improves quality of care slightly compared with usual care or the co-intervention(s) without the reminder component (median improvement 6.8% (
IQR:
3.8% to 17.5%); 34 studies (40 comparisons); moderate-certainty evidence).Computer-generated reminders delivered on paper to healthcare professionals alone (single-component intervention) probably improves quality of care compared with usual care (median improvement 11.0% (IQR 5.4% to 20.0%); 27 studies (27 comparisons); moderate-certainty evidence). Adding computer-generated reminders delivered on paper to healthcare professionals to one or more co-interventions (multi-component intervention) probably improves quality of care slightly compared with the co-intervention(s) without the reminder component (median improvement 4.0% (IQR 3.0% to 6.0%); 11 studies (13 comparisons); moderate-certainty evidence).We are uncertain whether reminders, alone or in addition to co-intervention(s), improve patient outcomes as the certainty of the evidence is very low (n = 6 studies (seven comparisons)). None of the included studies reported outcomes related to harms or adverse effects of the intervention.
AUTHORS' CONCLUSIONS:
There is moderate-certainty evidence that computer-generated reminders delivered on paper to healthcare professionals probably slightly improves quality of care, in terms of compliance with preventive guidelines and compliance with disease management guidelines. It is uncertain whether reminders improve patient outcomes because the certainty of the evidence is very low. The heterogeneity of the reminder interventions included in this review also suggests that reminders can probably improve quality of care in various settings under various conditions.
