BACKGROUND: Search filters are standardised sets of search terms, with validated performance, that are designed to retrieve studies with specific characteristics. A cost-utility analysis (CUA) is the preferred type of economic evaluation to underpin decision-making at the National Institute for Health and Care Excellence (NICE). Until now, when searching for economic evidence for NICE guidelines, we have used a broad set of health economic-related search terms, even when the reviewer's interest is confined to CUAs alone.
METHODS: We developed search filters to retrieve CUAs from MEDLINE and Embase. Our aim was to achieve recall of 90% or better across both databases while reducing the overall yield compared with our existing broad economic filter. We used the relative recall method along with topic expert input to derive and validate 3 pairs of filters, assessed by their ability to identify a gold-standard set of CUAs that had been used in published NICE guidelines. We developed and validated MEDLINE and Embase filters in pairs (testing whether, when used together, they find target studies in at least 1 database), as this is how they are used in practice. We examined the proxy-precision of our new filters by comparing their overall yield with our previous approach using publications indexed in a randomly selected year (2010).
RESULTS: All 3 filter-pairs exceeded our target recall and led to substantial improvements in search proxy-precision. Our paired 'sensitive' filters achieved 100% recall (95% CI 99.0 to 100%) in the validation set. Our paired 'precise' filters also had very good recall (97.6% [95%CI: 95.4 to 98.9%]). We estimate that, compared with our previous search strategy, using the paired 'sensitive' filters would reduce reviewer screening burden by a factor of 5 and the 'precise' versions would do so by a factor of more than 20.
CONCLUSIONS: Each of the 3 paired cost-utility filters enable the identification of almost all CUAs from MEDLINE and Embase from the validation set, with substantial savings in screening workload compared to our previous search practice. We would encourage other researchers who regularly use multiple databases to consider validating search filters in combination as this will better reflect how they use databases in their everyday work.
BACKGROUND: Delivery arrangements include changes in who receives care and when, who provides care, the working conditions of those who provide care, coordination of care amongst different providers, where care is provided, the use of information and communication technology to deliver care, and quality and safety systems. How services are delivered can have impacts on the effectiveness, efficiency and equity of health systems. This broad overview of the findings of systematic reviews can help policymakers and other stakeholders identify strategies for addressing problems and improve the delivery of services.
OBJECTIVES: To provide an overview of the available evidence from up-to-date systematic reviews about the effects of delivery arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on delivery arrangements and informing refinements of the framework for delivery arrangements outlined in the review.
METHODS: We searched Health Systems Evidence in November 2010 and PDQ-Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of delivery arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use, healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty or employment) and that were published after April 2005. We excluded reviews with limitations important enough to compromise the reliability of the findings. Two overview authors independently screened reviews, extracted data, and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence), and assessments of the relevance of findings to low-income countries.
MAIN RESULTS: We identified 7272 systematic reviews and included 51 of them in this overview. We judged 6 of the 51 reviews to have important methodological limitations and the other 45 to have only minor limitations. We grouped delivery arrangements into eight categories. Some reviews provided more than one comparison and were in more than one category. Across these categories, the following intervention were effective; that is, they have desirable effects on at least one outcome with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects. Who receives care and when: queuing strategies and antenatal care to groups of mothers. Who provides care: lay health workers for caring for people with hypertension, lay health workers to deliver care for mothers and children or infectious diseases, lay health workers to deliver community-based neonatal care packages, midlevel health professionals for abortion care, social support to pregnant women at risk, midwife-led care for childbearing women, non-specialist providers in mental health and neurology, and physician-nurse substitution. Coordination of care: hospital clinical pathways, case management for people living with HIV and AIDS, interactive communication between primary care doctors and specialists, hospital discharge planning, adding a service to an existing service and integrating delivery models, referral from primary to secondary care, physician-led versus nurse-led triage in emergency departments, and team midwifery. Where care is provided: high-volume institutions, home-based care (with or without multidisciplinary team) for people living with HIV and AIDS, home-based management of malaria, home care for children with acute physical conditions, community-based interventions for childhood diarrhoea and pneumonia, out-of-facility HIV and reproductive health services for youth, and decentralised HIV care. Information and communication technology: mobile phone messaging for patients with long-term illnesses, mobile phone messaging reminders for attendance at healthcare appointments, mobile phone messaging to promote adherence to antiretroviral therapy, women carrying their own case notes in pregnancy, interventions to improve childhood vaccination. Quality and safety systems: decision support with clinical information systems for people living with HIV/AIDS. Complex interventions (cutting across delivery categories and other health system arrangements): emergency obstetric referral interventions.
AUTHORS' CONCLUSIONS: A wide range of strategies have been evaluated for improving delivery arrangements in low-income countries, using sound systematic review methods in both Cochrane and non-Cochrane reviews. These reviews have assessed a range of outcomes. Most of the available evidence focuses on who provides care, where care is provided and coordination of care. For all the main categories of delivery arrangements, we identified gaps in primary research related to uncertainty about the applicability of the evidence to low-income countries, low- or very low-certainty evidence or a lack of studies.
BACKGROUND: Diabetes is a common chronic disease that places an unprecedented strain on health care systems worldwide. Mobile health technologies such as smartphones, mobile applications, and wearable devices, known as mHealth, offer significant and innovative opportunities for improving patient to provider communication and self-management of diabetes.
OBJECTIVE: The purpose of this overview is to critically appraise and consolidate evidence from multiple systematic reviews on the effectiveness of mHealth interventions for patients with diabetes to inform policy makers, practitioners, and researchers.
METHODS: A comprehensive search on multiple databases was performed to identify relevant systematic reviews published between January 1996 and December 2015. Two authors independently selected reviews, extracted data, and assessed the methodological quality of included reviews using AMSTAR.
RESULTS: Fifteen systematic reviews published between 2008 and 2014 were eligible for inclusion. The quality of the reviews varied considerably and most of them had important methodological limitations. Focusing on systematic reviews that offered the most direct evidence, this overview demonstrates that on average, mHealth interventions improve glycemic control (HbA1c) compared to standard care or other non-mHealth approaches by as much as 0.8% for patients with type 2 diabetes and 0.3% for patients with type 1 diabetes, at least in the short-term (≤12 months). However, limitations in the overall quality of evidence suggest that further research will likely have an important impact in these estimates of effect.
CONCLUSIONS: Findings are consistent with clinically relevant improvements, particularly with respect to patients with type 2 diabetes. Similar to home telemonitoring, mHealth interventions represent a promising approach for self-management of diabetes.
OBJECTIVE: To assess the available scientific evidence regarding the efficacy of interventions aimed to enhance medication adherence in patients with multiple chronic conditions (PMCC).
DESIGN: Overview of systematic reviews.
DATA SOURCES: The following databases were consulted (September 2013): Pubmed, EMBASE, the Cochrane Library, CRD and WoS to identify interventions aimed to enhance medication adherence in PMCC, or otherwise, patients with chronic diseases common in the PMCC, or polypharmacy.
STUDY SELECTION: Systematic reviews of clinical trials focused on PMCC or similar were included. They should compare the efficacy of any intervention aimed to improve compliance to prescribed and self-administered medications with clinical practice or other interventions.
DATA EXTRACTION: Information about the study population, nature of intervention and efficacy in terms of improved adherence was extracted.
RESULTS: 566 articles were retrieved of which 9 systematic reviews were included. None was specifically focused on PMCC but considered patients with chronic diseases common in the PMCC, patients with more than one chronic disease and polypharmacy. The overall effectiveness of interventions was modest without relevant differences between behavioural, educational and combined interventions. Some components of these interventions including patient counselling and regimen simplification appear to be effective tools in improving adherence in this population group.
CONCLUSION: There is a large heterogeneity of interventions aimed to improve adherence with modest efficacy, none in PMCC.
Short messaging service (SMS) messages may present a convenient and cost-effective method to support healthcare interventions. This work assesses the effects of short messaging service on various healthcare interventions found in systematic reviews. The search strategy was based on two key concepts: short messaging service and healthcare delivery. The initial search was conducted in December 2012 and was updated in June 2013. Of the 550 identified references, 13 systematic reviews met the inclusion criteria, of which 8 were published in peer-reviewed journals and 5 were retrieved from the Cochrane library. Data analysis shows that low to moderate research evidence exists on the benefits of short messaging service interventions for appointment reminders, promoting health in developing countries and preventive healthcare. In many interventions, however, there were a few studies that were of high quality, and most of the studies were rated from low to moderate quality or had no rating at all. Healthcare organizations, policy makers, or clinicians using short messaging service messages to support healthcare interventions should (1) implement interventions that have been found to work in healthcare settings, (2) continue evaluating short messaging service interventions that have not been adequately assessed, and (3) improve collaboration between various healthcare entities to develop studies targeted at specific populations to evaluate the long-term impact of short messaging service on healthcare outcomes.
The aim of this systematic review of reviews is to identify mobile text-messaging interventions designed for health improvement and behavior change and to derive recommendations for practice. We have compiled and reviewed existing systematic research reviews and meta-analyses to organize and summarize the text-messaging intervention evidence base, identify best-practice recommendations based on findings from multiple reviews, and explore implications for future research. Our review found that the majority of published text-messaging interventions were effective when addressing diabetes self-management, weight loss, physical activity, smoking cessation, and medication adherence for antiretroviral therapy. However, we found limited evidence across the population of studies and reviews to inform recommended intervention characteristics. Although strong evidence supports the value of integrating text-messaging interventions into public health practice, additional research is needed to establish longer-term intervention effects, identify recommended intervention characteristics, and explore issues of cost-effectiveness.
BackgroundStrong international commitment and the widespread use of antiretroviral therapy have led to higher longevity for people living with human immune deficiency virus (HIV). Text messaging interventions have been shown to improve health outcomes in people living with HIV. The objectives of this overview were to: map the state of the evidence of text messaging interventions, identify knowledge gaps, and develop a framework for the transfer of evidence to other chronic diseases.MethodsWe conducted a systematic review of systematic reviews on text messaging interventions to improve health or health related outcomes. We conducted a comprehensive search of PubMed, EMBASE (Exerpta Medica Database), CINAHL (Cumulative Index to Nursing and Allied Health Literature), PsycINFO, Web of Science (WoS) and the Cochrane Library on the 17th April 2014. Screening, data extraction and assessment of methodological quality were done in duplicate. Our findings were used to develop a conceptual framework for transfer.ResultsOur search identified 135 potential systematic reviews of which nine were included, reporting on 37 source studies, conducted in 19 different countries. Seven of nine (77.7%) of these reviews were high quality. There was some evidence for text messaging as a tool to improve adherence to antiretroviral therapy. Text messages also improved attendance at appointments and behaviour change outcomes. The findings were inconclusive for self-management of illness, treatment of tuberculosis and communicating results of medical investigations. The geographical distribution of text messaging research was limited to specific regions of the world. Prominent knowledge gaps included the absence of data on long term outcomes, patient satisfaction, and economic evaluations. The included reviews also identified methodological limitations in many of the primary studies.ConclusionsGlobal evidence supports the use of text messaging as a tool to improve adherence to medication and attendance at scheduled appointments. Given the similarities between HIV and other chronic diseases (long-term medications, life-long care, strong link to behaviour and the need for home-based support) evidence from HIV may be transferred to these diseases using our proposed framework by integration of HIV and chronic disease services or direct transfer.
ABSTRACT Background The worldwide burden of chronic disease is widespread and growing. This shift from acute to chronic care requires rethinking how resources are invested in managing these conditions. One response has been to create programs and interventions that have the goal of helping patients better manage their own conditions. Over time, these self-management interventions and strategies have increasingly relied on various technologies for their implementation, with the newest technology being mobile phones and short message service (SMS). Aim The objective of this meta-review was to evaluate the current evidence on the use of mobile phones and SMS to deliver self-management interventions for chronic conditions. Methods A meta-review was conducted of the 11 systematic reviews (SRs) that were identified and retrieved after an extensive search of electronic databases covering the years 2000-2012. Relevant information was abstracted from each systematic review and a quality assessment carried out using the AMSTAR ('A Measurement Tool to Assess Systematic Reviews') criteria. Findings The number and types of included studies and total number of subjects varied significantly across the systematic reviews. Mobile phone text messaging was reported to significantly improve adherence to appointments and antiretroviral therapy, short-term smoking quit rates, and selected clinical and behavioral outcomes. AMSTAR scores ranged from 11 to 3, reflecting substantial variation in SR quality. Linking Evidence to Action Mobile phones and SMS show promise as a technology to deliver self-management interventions to improve outcomes of chronic care management. However, the quality of future studies and systematic reviews needs to be improved. There also are unresolved issues about the technology itself.
BACKGROUND: eHealth potentially enhances quality of care and may reduce health care costs. However, a review of systematic reviews published in 2010 concluded that high-quality evidence on the benefits of eHealth interventions was still lacking.
OBJECTIVE: We conducted a systematic review of systematic reviews and meta-analyses on the effectiveness/cost-effectiveness of eHealth interventions in patients with somatic diseases to analyze whether, and to what possible extent, the outcome of recent research supports or differs from previous conclusions.
METHODS: Literature searches were performed in PubMed, EMBASE, The Cochrane Library, and Scopus for systematic reviews and meta-analyses on eHealth interventions published between August 2009 and December 2012. Articles were screened for relevance based on preset inclusion and exclusion criteria. Citations of residual articles were screened for additional literature. Included papers were critically appraised using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement before data were extracted. Based on conclusions drawn by the authors of the included articles, reviews and meta-analyses were divided into 1 of 3 groups: suitable, promising, or limited evidence on effectiveness/cost-effectiveness. Cases of uncertainty were resolved by consensus discussion. Effect sizes were extracted from papers that included a meta-analysis. To compare our results with previous findings, a trend analysis was performed.
RESULTS: Our literature searches yielded 31 eligible reviews, of which 20 (65%) reported on costs. Seven papers (23%) concluded that eHealth is effective/cost-effective, 13 (42%) underlined that evidence is promising, and others found limited or inconsistent proof. Methodological quality of the included reviews and meta-analyses was generally considered high. Trend analysis showed a considerable accumulation of literature on eHealth. However, a similar percentage of papers concluded that eHealth is effective/cost-effective or evidence is at least promising (65% vs 62%). Reviews focusing primarily on children or family caregivers still remained scarce. Although a pooled (subgroup) analysis of aggregate data from randomized studies was performed in a higher percentage of more recently published reviews (45% vs 27%), data on economic outcome measures were less frequently reported (65% vs 85%).
CONCLUSIONS: The number of reviews and meta-analyses on eHealth interventions in patients with somatic diseases has increased considerably in recent years. Most articles show eHealth is effective/cost-effective or at least suggest evidence is promising, which is consistent with previous findings. Although many researchers advocate larger, well-designed, controlled studies, we believe attention should be given to the development and evaluation of strategies to implement effective/cost-effective eHealth initiatives in daily practice, rather than to further strengthen current evidence.
Search filters are standardised sets of search terms, with validated performance, that are designed to retrieve studies with specific characteristics. A cost-utility analysis (CUA) is the preferred type of economic evaluation to underpin decision-making at the National Institute for Health and Care Excellence (NICE). Until now, when searching for economic evidence for NICE guidelines, we have used a broad set of health economic-related search terms, even when the reviewer's interest is confined to CUAs alone.
METHODS:
We developed search filters to retrieve CUAs from MEDLINE and Embase. Our aim was to achieve recall of 90% or better across both databases while reducing the overall yield compared with our existing broad economic filter. We used the relative recall method along with topic expert input to derive and validate 3 pairs of filters, assessed by their ability to identify a gold-standard set of CUAs that had been used in published NICE guidelines. We developed and validated MEDLINE and Embase filters in pairs (testing whether, when used together, they find target studies in at least 1 database), as this is how they are used in practice. We examined the proxy-precision of our new filters by comparing their overall yield with our previous approach using publications indexed in a randomly selected year (2010).
RESULTS:
All 3 filter-pairs exceeded our target recall and led to substantial improvements in search proxy-precision. Our paired 'sensitive' filters achieved 100% recall (95% CI 99.0 to 100%) in the validation set. Our paired 'precise' filters also had very good recall (97.6% [95%CI: 95.4 to 98.9%]). We estimate that, compared with our previous search strategy, using the paired 'sensitive' filters would reduce reviewer screening burden by a factor of 5 and the 'precise' versions would do so by a factor of more than 20.
CONCLUSIONS:
Each of the 3 paired cost-utility filters enable the identification of almost all CUAs from MEDLINE and Embase from the validation set, with substantial savings in screening workload compared to our previous search practice. We would encourage other researchers who regularly use multiple databases to consider validating search filters in combination as this will better reflect how they use databases in their everyday work.
