Systematic Review Article Critique Essay

Special Feature: Methods SeriesSystematic Review and Meta-analysis: When One Study Is Just not Enough Amit X. Garg,* Dan Hackam,† and Marcello Tonelli‡ *Division of Nephrology and Department of Epidemiology and Biostatistics, University of Western Ontario, London, and †Division of Clinical Pharmacology and Toxicology, University of Toronto, and Cardiac Rehabilitation and Secondary Prevention Program, Toronto Rehabilitation Institute, Toronto, Ontario, and ‡Division of Nephrology and Department of Public Health Sciences, University of Alberta, and Institute of Health Economics, Edmonton, Alberta, Canada Clin J Am Soc Nephrol 3: 253–260, 2008. doi: 10.2215/CJN.01430307 W e live in the information age, and the practice of medicine is becoming increasingly specialized. In the biomedical literature, the number of published studies has dramatically increased: There are now more than 15 million citations in MEDLINE, with 10,000 to 20,000 new citations added each week (1). Multiple relevant studies usually guide most clinical decisions. These studies often vary in their design; methodologic quality; population studied; and the intervention, test, or condition considered. Because even highly cited trials may be challenged or refuted over time (2), clinical decision-making requires ongoing reconciliation of studies that provide different answers to the same question. Both clinicians and researchers can also benefit from a summary of where uncertainty remains. Because it is often impractical for readers to track down and review all of the primary studies (3), review articles are an important source of summarized evidence on a particular topic (4). Narrative Review, Systematic Review, and Meta-analysis Review articles have traditionally taken the form of a narrative review, whereby a content expert writes about a particular field, condition, or treatment (5–7). Narrative reviews have many benefits, including a broad overview of relevant information tempered by years of practical knowledge from an experienced author. Indeed, this article itself is in a narrative format, from authors who have published a number of metaanalyses in previous years. In some circumstances, a reader wants to become very knowledgeable about specific details of a topic and wants some assurance that the information presented is both comprehensive and unbiased. A narrative review typically uses an implicit process to compile evidence to support the statements being made. The reader often cannot tell which recommendations Published online ahead of print. Publication date available at www.cjasn.org. Correspondence: Dr. Amit Garg, London Kidney Clinical Research Unit, Room ELL-101, Westminster, London Health Sciences Centre, 800 Commissioners Road East, London, Ontario N6A 4G5, Canada. Phone: 519-685-8502; Fax: 519-685-8072; E-mail: amit.garg@lhsc.on.ca Copyright © 2008 by the American Society of Nephrology were based on the author’s clinical experience, the breadth to which available literature was identified and compiled, and the reasons that some studies were given more emphasis than others. It is sometimes uncertain whether the author of a narrative review selectively cited reports that reinforced his or her preconceived ideas or promoted specific views of a topic. Also, a quantitative summary of the literature is often absent in a narrative review. A systematic review uses a process to identify comprehensively all studies for a specific focused question (drawn from research and other sources), appraise the methods of the studies, summarize the results, present key findings, identify reasons for different results across studies, and cite limitations of current knowledge (8,9). In a systematic review, all decisions used to compile information are meant to be explicit, allowing the reader to gauge for him- or herself the quality of the review process and the potential for bias. In this way, systematic reviews tend to be more transparent than their narrative cousins, although they too can be biased if the selection or emphasis of certain primary studies is influenced by the preconceived notions of the authors or funding sources (10). Depending on the nature of the data, the results of a systematic review can be summarized in text or graphic form. In graphic form, it is common for different trials to be depicted in a plot where the point estimate and 95% confidence interval for each study are presented on an individual line (11). When results are mathematically combined (a process sometimes referred to as pooling), this is referred to as meta-analysis. Graphically, the pooled result is often presented as a diamond at the bottom of the plot. When performing a meta-analysis, a review team usually combines aggregate-level data reported in each primary study (point and variance estimate of the summary measure). On occasion, a review team will obtain all of the individual patient data from each of the primary studies (12,13). Although challenging to conduct (14), individual patient meta-analyses may have certain advantages over aggregate-level analyses. As highlighted in a review of angiotensin-converting enzyme (ACE) inhibitors for nondiabetic kidney disease, this includes the use of common definitions, coding and cutoff points beISSN: 1555-9041/301–0253 254 Clinical Journal of the American Society of Nephrology tween studies, addressing questions not examined in the original publication, and a better sense of the impact of individual patient (versus study level) characteristics (12,15). As first highlighted a decade ago (16), the number of systematic reviews in nephrology and other fields has increased dramatically with time, paralleling the rapid growth of biomedical literature during the past half century. Initiatives such as the Cochrane Collaboration have further increased the profile and rigor of the systematic review process (details of the structured process of Cochrane systematic reviews are available through their Web site) (17,18). From 1990 to 2005, there were more than 400 systematic reviews and meta-analyses published in the discipline of nephrology (Figure 1). Of these reviews, 40% pertained to chronic kidney disease or glomerulonephritis and 20, 16, 15, and 7% pertained to kidney transplantation, dialysis, acute kidney injury, and pediatric nephrology, respectively. As a publication type, however, systematic reviews have not been without controversy: Some authors consider a meta-analysis the best possible use of all available data, whereas others question whether they add anything meaningful to scientific knowledge (19). The strengths and weaknesses of this publication type are described next. Strengths of Systematic Review and Meta-analysis Physicians make better clinical decisions when they understand the circumstances and preferences of their patients and combine their personal experience with clinical evidence underlying the available options (20). The public also expects that their physicians will integrate research findings into practice in a timely way (21). Thus, sound clinical or health policy decisions are facilitated by reviewing the available evidence (and its limitations), understanding reasons why some studies differ in their results (a finding sometimes referred to as heterogeneity among the primary studies), coming up with an assessment of the expected effect of an intervention or exposure (for questions of therapy or etiology), and then integrating the new informa- Clin J Am Soc Nephrol 3: 253–260, 2008 tion with other relevant treatment, patient, and health care system factors. In this respect, reading a properly conducted systematic review is an efficient way to become familiar with the best available research evidence for a focused clinical question. The review team may also have obtained information from the primary authors which was not available in the original reports. The presented summary allows the reader to take into account a whole range of relevant findings from research on a particular topic. The process can also establish whether the scientific findings are consistent and generalizable across populations, settings, and treatment variations and whether findings vary significantly by particular subgroups. Again, the potential strength of a systematic review lies in the transparency of each phase of the synthesis process, allowing the reader to focus on the merits of each decision made in compiling the information, rather than a simple contrast of one study to another as sometimes occurs in other types of reviews. For example, studies demonstrating a significant effect of treatment are more likely to be published than studies with negative findings, are more likely to be published in English, and more likely to be cited by others (22–27). A well-conducted systematic review attempts to reduce the possibility of bias in the method of identifying and selecting studies for review, by using a comprehensive search strategy and specifying inclusion criteria that ideally have not been influenced by a priori knowledge of the primary studies. Mathematically combining data from a series of well-conducted primary studies may provide a more precise estimate of the underlying “true effect” than any individual study (28). In other words, by combining the samples of the individual studies, the size of the “overall sample” is increased, enhancing the statistical power of the analysis and reducing the size of the confidence interval for the point estimate of the effect. It is also more efficient to communicate a pooled summary than to describe the results for each of the individual studies. Sometimes, if the treatment effect in small trials shows a nonsignificant trend toward efficacy, then pooling the results may establish 100 90 80 Number 70 60 50 40 30 20 10 0 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 Year Figure 1. There have been more than 400 systematic reviews and meta-analyses published in the discipline of nephrology since 1990, with the annual number increasing with time. Frequencies were estimated from a MEDLINE and EMBASE search performed by an experienced renal librarian in December 2006. Citations were reviewed by a nephrologist for relevance. Duplicate publications from the same group of authors were counted only once. Clin J Am Soc Nephrol 3: 253–260, 2008 the benefits of therapy (16). For example, 10 trials examined whether ACE inhibitors were more effective than other antihypertensive agents for the prevention of nondiabetic kidney failure (29). Many of the 95% confidence intervals for the estimate provided by each study overlapped with a finding of no effect; however, the overall pooled estimate established a benefit of ACE inhibitors. For these reasons, a meta-analysis of similar, well-conducted, randomized, controlled trials has been considered one of the highest levels of evidence (30 –32). It is important to stress that the primary trials all have to be conducted with high methodologic rigor for the meta-analysis to be definitive. Alternatively, when the existing studies have important scientific and methodologic limitations, including smaller sized samples (which is more often the case), the systematic review may identify where gaps exist in the available literature. In this case, an exploratory meta-analysis can provide a plausible estimate of effect that can be tested in subsequent studies (33,34). Limitations of Systematic Review and Meta-analysis This type of publication type has many potential limitations that should be appreciated by all readers. First, the summary provided in a systematic review and meta-analysis of the literature is only as reliable as the methods used to estimate the effect in each of the primary studies. In other words, conducting a meta-analysis does not overcome problems that were inherent in the design and execution of the primary studies. It also does not correct biases as a result of selective publication, whereby studies that report dramatic effects are more likely to be identified, summarized, and subsequently pooled in metaanalysis than studies that report smaller effect sizes (an issue referred to as publication bias). Because more than three quarters of meta-analyses did not report any empirical assessment of publication bias (35), the true frequency of this form of bias is unknown. Controversies also arise around the interpretation of summarized results, particularly when the results of discordant studies are pooled in meta-analysis (36). The review process inevitably identifies studies that are diverse in their design, methodologic quality, specific interventions used, and types of patients studied. There is often some subjectivity when deciding how similar studies must be before pooling is appropriate. Combining studies of poor quality with those that were more rigorously conducted may not be useful and can lead to worse estimates of the underlying truth or a false sense of precision around the truth (36). A false sense of precision may also arise when various subgroups of patients defined by characteristics such as their age or gender differ in their observed response. In such cases, reporting an aggregate pooled effect might be misleading if there are important reasons to explain variable treatment effects across different types of patients (36 – 40). Finally, simply labeling a manuscript as a “systematic review” or “meta-analysis” does not guarantee that the review was conducted or reported with due rigor (41). To reduce the chance of arriving at misleading conclusions, guidelines on the conduct and reporting of systematic reviews were recently Systematic Review and Meta-analysis 255 published (42,43); however, important methodologic flaws of systematic reviews published in peer-reviewed journals have been well described (44 –54). For example, of the 86 renal systematic reviews published in 2005, the majority (58%) had important methodologic flaws (Mrkobrada M, ThiessenPhilbrook H, Haynes RB, Iansavichus AV, Rehman F, and Garg AX, submitted). The most common flaws among these renal reviews were failure to assess the methodologic quality of included primary studies and failure to avoid bias in study inclusion (Mrkobrada M, Thiessen-Philbrook H, Haynes RB, Iansavichus AV, Rehman F, and Garg AX, submitted). In some cases, industry-supported reviews of drugs have had fewer reservations about methodologic limitations of the included trials than rigorously conducted Cochrane reviews on the same topic (10); however, the hypothesis that less rigorous reviews more often report positive conclusions than good-quality reviews of the same topic has not been borne out in empirical assessment (48,53,55). Nonetheless, like all good consumers, users of systematic reviews should carefully consider the quality of the product and adhere to the dictum “caveat emptor”: Let the buyer beware. The limitations described in this section may explain differences in the results of meta-analyses as compared with subsequent large, randomized, controlled trials, which have occurred in approximately one third of cases (56). How to Appraise Critically a Systematic Review and Meta-analysis Users of systematic reviews need to assure themselves that the underlying methods used to assemble relevant information were sound. Before considering the results or how the information could be appropriately applied in patient care (9), there are a few questions that the reader can ask him- or herself when Table 1. Questions to ask when assessing the quality of a systematic reviewa 1. Was the review conducted according to a prespecified protocol? 2. Was the question focused and well formulated? 3. Were the right types of studies eligible for the review? 4. Was the method of identifying all relevant information comprehensive? a. Is it likely that relevant studies were missed? b. Was publication bias considered? 5. Was the data abstraction from each study appropriate? a. Were the methods used in each primary study appraised? 6. Was the information synthesized and summarized appropriately? a. If the results were mathematically combined in meta-analysis, then were the methods described in sufficient detail, and was it reasonable to do so? a Adapted from reference (9). 256 Clinical Journal of the American Society of Nephrology assessing the methodologic quality of a systematic review (Table 1). Was the Review Conducted According to a Prespecified Protocol? It is reassuring if a review was guided by a written protocol (prepared in advance) that describes the research question(s), hypotheses, review method, and plan for how the data will be extracted and compiled. Such an approach minimizes the likelihood that the results or the expectations of the reviewing team influenced study inclusion or synthesis. Although most systematic reviews are conducted in a retrospective manner, reviews and meta-analyses can in theory be defined at the time several similar trials are being planned or under way. This allows a set of specific hypotheses, data collection procedures, and analytic strategies to be specified in advance before any of the results from the primary studies are known. Such a prospective effort may provide more reliable answers to medically relevant questions than the traditional retrospective approach (41). Was the Question Focused? Clinical questions often deal with issues of treatment, etiology, prognosis, and diagnosis. A well-formulated question usually specifies the patient’s problem or diagnosis, the intervention or exposure of interest, any comparison group (if relevant), and the primary and secondary outcomes of interest (57). Were the “Right” Types of Studies Eligible for the Review? Different study designs can be used to answer different clinical questions. Randomized, controlled trials; observational studies; and cross-sectional diagnostic studies may each be appropriate depending on the primary question posed in the review. When examining the eligible criteria for study inclusion, the reader should feel confident that a potential bias in the selection of studies was avoided. Specifically, the reader should ask her- or himself whether the eligibility criteria for study inclusion were appropriate for the question asked. Whether the right types of studies were selected for the review also depends on the depth and breadth of the underlying literature search. For example, some review teams will consider only studies that were published in English. There is evidence that journals from certain countries publish a higher proportion of positive trials than others (58). Excluding non-English studies seemed to change the results of some reviews (59,60) but not others (61,62). Some review teams use broad criteria for their inclusion of primary studies (e.g., effects of agents that block the reninangiotensin system on renal outcomes [63]), whereas other teams use more narrow inclusion criteria (e.g., restricting the analysis only to patients who have diabetes without evidence of nephropathy [64]). There is often no single correct approach; however, the conclusions of any meta-analysis that is highly sensitive to altering the entry criteria of included studies should be interpreted with some caution (25). For example, two different review teams considered whether synthetic dialysis membranes resulted in better clinical outcomes compared with cellulose-based membranes in patients with acute renal failure. In Clin J Am Soc Nephrol 3: 253–260, 2008 one meta-analysis (65) but not the other (66), synthetic membranes reduced the chance for death. The discordant results were due to the inclusion of a study that did not meet eligibility for the second review (67). Was the Method of Identifying All Relevant Information Comprehensive? Identifying relevant studies for a given clinical question among the many potential sources of information is usually a laborious process (68). Biomedical journals are the most common source of information, and bibliographic databases are frequently used to search for relevant articles. MEDLINE currently indexes approximately 4800 medical journals and contains 13 million citations (69). Similarly, EMBASE indexes approximately 5000 medical journals and contains more than 11 million records. There are some key differences between EMBASE and MEDLINE, and the review team should have searched both databases (70 –72). For example, EMBASE provides the best coverage of European research as well as pharmaceutical research including renal adverse events (73). Positive studies may be more often published in journals that are indexed in MEDLINE, compared with nonindexed journals (25). Depending on the question posed, other databases may also have been searched. For example, if a team is summarizing the effects of exercise training in patients who receive maintenance hemodialysis, then searching the Cumulative Index to Nursing and Allied Health Literature (CINAHL) database would be appropriate (74). Alternatively, the ECONOLIT database may be useful for identifying information on the out-of-pocket expenses incurred by living kidney donors (75). As a supplementary method of identifying information, searching databases such as the Science Citation Index (which identifies all articles that cite a relevant article), as well as newer Internet search engines such as Google Scholar and Elsevier’s Scirus, can be useful for identifying articles that are not indexed well in traditional bibliographic databases (76). Searching bibliographies of retrieved articles can also identify relevant articles that were missed. Whatever bibliographic database was used, the review team should have used a search strategy that maximized the identification of relevant articles (77,78). Because there is some subjectivity in screening databases, citations should be reviewed independently and in duplicate by two members of the reviewing team, with the full-text article retrieved for any citation deemed relevant by any of the reviewers. There is also some subjectivity in assessing the eligibility of each full-text article, and the risk for incorrectly discarding relevant reports is reduced when two reviewers independently perform each assessment in a reliable manner (79). Important sources of information other than journal articles should not be overlooked. Conference proceedings, abstracts, books, and manufacturers all can be sources of potentially valuable information. Inquiries to experts, including those listed in trial registries, may have also proved useful (28). A comprehensive search of available literature reduces the possibility of publication bias, which occurs when studies with statistically significant results are more likely to be published Clin J Am Soc Nephrol 3: 253–260, 2008 and cited (80,81). It is interesting that some recent reviews of acetylcysteine for the prevention of contrast nephropathy analyzed as few as five studies, despite being submitted for publication almost 1 yr after publication of a review of 12 studies (82). Although there are many potential reasons for this, one cannot exclude the possibility that some search strategies missed eligible trials. In addition to a comprehensive search method, which makes it unlikely that relevant studies were missed, it is often reassuring if the review team used graphic and statistical methods to confirm that there was little chance that publication bias influenced the results (83). Was the Data Abstraction from Each Study Appropriate? In compiling relevant information, the review team should have used a rigorous and reproducible method of abstracting all relevant data from the primary studies. Often two reviewers abstract key information from each primary study, including study and patient characteristics, setting, and details about the intervention, exposure, or diagnostic test as is appropriate. Language translators may be needed. Teams who conduct their review with due rigor will indicate that they contacted the primary authors from each of the primary studies to confirm the accuracy of abstracted data as well as to provide additional relevant information that was not provided in the primary report. Some authors will go through the additional effort of blinding or masking the results from other study characteristics so that data abstraction is as objective as possible (84,85). One element that should have been abstracted is the methodologic quality of each primary study (recognizing this is not always as straightforward as it may first seem) (86 –91). The question to be posed by the reader is whether the reviewing team considered if each of the primary studies was designed, conducted, and analyzed in a way to minimize or avoid biases in the results (92). For randomized, controlled trials, lack of concealment of allocation, inadequate generation of the allocation sequence, and lack of double blinding can exaggerate estimates of the treatment effect (54,90,93). The value of abstracting such data is that it may help to explain important differences in the results among the primary studies (90). For example, long-term risk estimates can become unreliable when participants are lost to study follow-up; those who participate in follow-up often systematically differ from nonparticipants. For this reason, prognosis studies are vulnerable to bias, unless the loss to follow-up is less than 20% (94). In a systematic review of 49 studies on the renal prognosis of diarrhea associated hemolytic uremic syndrome, on average, 21% of patients were lost to follow-up (range 0 to 59% across studies) (95). It was hypothesized that patients who were lost to follow-up would contribute to worse estimates of long-term prognosis because they are typically healthier than those who continue to be followed by their nephrologists. Indeed, studies with a higher proportion of patients lost to follow-up demonstrated a higher proportion of patients with long-term renal sequelae, explaining 28% of the between-study variability. Systematic Review and Meta-analysis 257 How Was the Information Synthesized and Summarized? In cases in which the primary studies differ in the design, populations studied, interventions and comparisons used, or outcomes measured, it may have been appropriate for the review team simply to report the results descriptively using text and tables. When the primary studies are similar in these characteristics and the studies provide a similar estimate of a true effect, then meta-analysis may have been used to derive a more precise estimate of this effect (96). In meta-analysis, data from the individual studies are not simply combined as though they were from a single study; rather, greater weights are given to the results from studies that provide more information, because they are likely to be closer to true effect being estimated. Mathematically combining the results from the individual studies can be accomplished under the assumption of “fixed” effects or “random” effects model. Although a thorough description and merits of each approach is described elsewhere (97), it is fair to say that a random-effects model is more conservative than the fixed-effects approach, and a finding that is statistically significant with the latter but not the former should be viewed with skepticism. Whenever individual studies are pooled in meta-analysis, it is important for the reader to determine whether it was reasonable to do so. One way to assess the similarity of various studies is to inspect the graphic display of the results, looking for similarities in the direction of the estimated effect. Even without considering any combined meta-analytic result, a reader becomes much more confident when a similar effect is being observed across many studies (i.e., the results have replicated across many studies). Some review teams may report a statistical test to determine how different the studies are from one another (as described previously, this is often termed heterogeneity of the study results [98]). This can help to prove or disprove that differences in the results that were observed between the primary studies is no different from what would be expected by chance. The most common statistical test to quantify heterogeneity is something called the Q statistic, which is similar in concept to a ␹2 test. Although a nonsignificant result (by convention P ⬎ 0.1) is often taken to indicate that there are no substantial differences between the studies, it is important to consider that this test is underpowered, especially when the number of studies being pooled is small. A new statistic that is frequently being reported in meta-analysis these days is something called the I2 statistic. This statistic describes the percentage variability between the studies that is present beyond what would be expected by chance. When interpreting an I2 statistic, values of 0 to 30, 31 to 50, and ⬎50% represent mild, moderate, and marked differences between the studies, respectively (99). Whenever a review team identifies significant differences between the primary studies, they should try to explain possible reasons for these differences. This can be done in an informal way by analyzing certain types of studies separately or by selectively combining studies to determine which are particularly different from the remaining studies. Alternatively, a statistical approach can be taken to explore differences across studies, using a technique similar to linear or logistic regression 258 Clinical Journal of the American Society of Nephrology (which at the study level is something called meta-regression) (100). Either way, a careful exploration of why study results differ can yield important information about potential determinants of the effect being observed. Conclusions Like all types of research, systematic reviews and meta-analyses have both potential strengths and weaknesses. With the growth of renal clinical studies, an increasing number of these types of summary publications will certainly become available to nephrologists, researchers, administrators, and policy makers who seek to keep abreast of recent developments. To maximize their advantages, it is essential that future reviews be conducted and reported properly, with judicious interpretation by the discriminating reader. Acknowledgments A.X.G. was supported by a Clinician Scientist Award from the Canadian Institutes of Health Research (CIHR). D.H. was supported by a CIHR Fellowship Award, the Chisholm Memorial Fellowship, and the Clinician-Scientist Training Program of the University of Toronto. M.T. was supported by a Population Health Investigator Award from the Alberta Heritage Foundation for Medical Research and a New Investigator Award from the CIHR. We thank Drs. Chi Hsu and Harvey Feldman for help and advice. We thank Arthur Iansavichus, MLIS, who helped compile systematic reviews published in the discipline of nephrology. Disclosures None. References 1. National Library of Medicine: Fact Sheet Medline. http:// www.nlm.nih.gov/pubs/factsheets/medline.html. Accessed November 15, 2007 2. Ioannidis JP: Contradicted and initially stronger effects in highly cited clinical research. JAMA 294: 218 –228, 2005 3. Garg AX, Iansavichus AV, Kastner M, Walters LA, Wilczynski N, McKibbon KA, Yang RC, Rehman F, Haynes RB: Lost in publication: Half of all renal practice evidence is published in non-renal journals. Kidney Int 70: 1995–2005, 2006 4. Haynes RB, Cotoi C, Holland J, Walters L, Wilczynski N, Jedraszewski D, McKinlay J, Parrish R, McKibbon KA: Second-order peer review of the medical literature for clinical practitioners. JAMA 295: 1801–1808, 2006 5. Barrett BJ, Parfrey PS: Clinical practice: Preventing nephropathy induced by contrast medium. N Engl J Med 354: 379 –386, 2006 6. Halloran PF: Immunosuppressive drugs for kidney transplantation. N Engl J Med 351: 2715–2729, 2004 7. Schrier RW, Wang W: Acute renal failure and sepsis. N Engl J Med 351: 159 –169, 2004 8. Cook DJ, Mulrow CD, Haynes RB: Systematic reviews: Synthesis of best evidence for clinical decisions. Ann Intern Med 126: 376 –380, 1997 9. Oxman AD, Cook DJ, Guyatt GH: Users’ guides to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA 272: 1367–1371, 1994 10. Jorgensen AW, Hilden J, Gotzsche PC: Cochrane reviews Clin J Am Soc Nephrol 3: 253–260, 2008 compared with industry supported meta-analyses and other meta-analyses of the same drugs: Systematic review. BMJ 333: 782, 2006 11. Lewis S, Clarke M: Forest plots: Trying to see the wood and the trees. BMJ 322: 1479 –1480, 2001 12. Lyman GH, Kuderer NM: The strengths and limitations of meta-analyses based on aggregate data. BMC Med Res Methodol 5: 14, 2005 13. Simmonds MC, Higgins JP, Stewart LA, Tierney JF, Clarke MJ, Thompson SG: Meta-analysis of individual patient data from randomized trials: A review of methods used in practice. Clin Trials 2: 209 –217, 2005 14. Schmid CH, Landa M, Jafar TH, Giatras I, Karim T, Reddy M, Stark PC, Levey AS: Constructing a database of individual clinical trials for longitudinal analysis. Control Clin Trials 24: 324 –340, 2003 15. Schmid CH, Stark PC, Berlin JA, Landais P, Lau J: Metaregression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. J Clin Epidemiol 57: 683– 697, 2004 16. Fouque D, Laville M, Haugh M, Boissel JP: Systematic reviews and their roles in promoting evidence-based medicine in renal disease. Nephrol Dial Transplant 11: 2398 –2401, 1996 17. Campbell MK, Daly C, Wallace SA, Cody DJ, Donaldson C, Grant AM, Khan IH, Lawrence P, Vale L, MacLeod AM: Evidence-based medicine in nephrology: Identifying and critically appraising the literature. Nephrol Dial Transplant 15: 1950 –1955, 2000 18. The Cochrane Collaboration. Available at: http://www.cochrane.org/index.htm. Accessed March 3, 2007 19. Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C: Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 28: 1–9, 1999 20. Haynes RB, Devereaux PJ, Guyatt GH: Physicians’ and patients’ choices in evidence based practice. BMJ 324: 1350, 2002 21. Fones CS, Kua EH, Goh LG: ‘What makes a good doctor?’ Views of the medical profession and the public in setting priorities for medical education. Singapore Med J 39: 537–542, 1998 22. Sterne JA, Egger M, Smith GD: Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ 323: 101–105, 2001 23. Simes RJ: Confronting publication bias: A cohort design for meta-analysis. Stat Med 6: 11–29, 1987 24. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR: Publication bias in clinical research. Lancet 337: 867– 872, 1991 25. Egger M, Smith GD: Bias in location and selection of studies. BMJ 316: 61– 66, 1998 26. Dickersin K, Min YI, Meinert CL: Factors influencing publication of research results: Follow-up of applications submitted to two institutional review boards. JAMA 267: 374 –378, 1992 27. Stern JM, Simes RJ: Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. BMJ 315: 640 – 645, 1997 28. Pogue J, Yusuf S: Overcoming the limitations of current metaanalysis of randomised controlled trials. Lancet 351: 47–52, 1998 29. Giatras I, Lau J, Levey AS: Effect of angiotensin-converting enzyme inhibitors on the progression of nondiabetic renal disease: A meta-analysis of randomized trials. Angiotensin- Clin J Am Soc Nephrol 3: 253–260, 2008 Converting-Enzyme Inhibition and Progressive Renal Disease Study Group. Ann Intern Med 127: 337–345, 1997 30. Guyatt G, Gutterman D, Baumann MH, Addrizzo-Harris D, Hylek EM, Phillips B, Raskob G, Lewis SZ, Schunemann H: Grading strength of recommendations and quality of evidence in clinical guidelines: Report from an American College of Chest Physicians task force. Chest 129: 174 –181, 2006 31. Hadorn DC, Baker D, Hodges JS, Hicks N: Rating the quality of evidence for clinical practice guidelines. J Clin Epidemiol 49: 749 –754, 1996 32. Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L, Naylor CD, Wilson MC, Richardson WS: Users’ guides to the medical literature: XXV. Evidence-based medicine: principles for applying the Users’ Guides to patient care. EvidenceBased Medicine Working Group. JAMA 284: 1290 –1296, 2000 33. Anello C, Fleiss JL: Exploratory or analytic meta-analysis: Should we distinguish between them? J Clin Epidemiol 48: 109 –116, 1995 34. Boudville N, Prasad GV, Knoll G, Muirhead N, ThiessenPhilbrook H, Yang RC, Rosas-Arellano MP, Housawi A, Garg AX: Meta-analysis: Risk for hypertension in living kidney donors. Ann Intern Med 145: 185–196, 2006 35. Palma S, Delgado-Rodriguez M: Assessment of publication bias in meta-analyses of cardiovascular diseases. J Epidemiol Community Health 59: 864 – 869, 2005 36. Lau J, Ioannidis JP, Schmid CH: Summing up evidence: One answer is not always enough. Lancet 351: 123–127, 1998 37. Thompson SG: Why sources of heterogeneity in meta-analysis should be investigated. BMJ 309: 1351–1355, 1994 38. Berlin JA: Invited commentary: Benefits of heterogeneity in meta-analysis of data from epidemiologic studies. Am J Epidemiol 142: 383–387, 1995 39. Davey SG, Egger M, Phillips AN: Meta-analysis: Beyond the grand mean? BMJ 315: 1610 –1614, 1997 40. Thompson SG, Higgins JP: Treating individuals 4: Can metaanalysis help target interventions at individuals most likely to benefit? Lancet 365: 341–346, 2005 41. Yusuf S: Meta-analysis of randomized trials: Looking back and looking ahead. Control Clin Trials 18: 594 – 601, 1997 42. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF: Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 354: 1896 –1900, 1999 43. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB: Metaanalysis of observational studies in epidemiology: A proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group. JAMA 283: 2008 –2012, 2000 44. Choi PT, Halpern SH, Malik N, Jadad AR, Tramer MR, Walder B: Examining the evidence in anesthesia literature: A critical appraisal of systematic reviews. Anesth Analg 92: 700 – 709, 2001 45. Dixon E, Hameed M, Sutherland F, Cook DJ, Doig C: Evaluating meta-analyses in the general surgical literature: A critical appraisal. Ann Surg 241: 450 – 459, 2005 46. Kelly KD, Travers A, Dorgan M, Slater L, Rowe BH: Evaluating the quality of systematic reviews in the emergency medicine literature. Ann Emerg Med 38: 518 –526, 2001 47. Sacks HS, Reitman D, Pagano D, Kupelnick B: Meta-analysis: An update. Mt Sinai J Med 63: 216 –224, 1996 48. Assendelft WJ, Koes BW, Knipschild PG, Bouter LM: The relationship between methodological quality and conclusions Systematic Review and Meta-analysis 259 in reviews of spinal manipulation. JAMA 274: 1942–1948, 1995 49. Jadad AR, McQuay HJ: Meta-analyses to evaluate analgesic interventions: A systematic qualitative review of their methodology. J Clin Epidemiol 49: 235–243, 1996 50. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, Moher D: Methodology and reports of systematic reviews and meta-analyses: A comparison of Cochrane reviews with articles published in paper-based journals. JAMA 280: 278 – 280, 1998 51. Bero LA, Rennie D: Influences on the quality of published drug studies. Int J Technol Assess Health Care 12: 209 –237, 1996 52. Barnes DE, Bero LA: Why review articles on the health effects of passive smoking reach different conclusions. JAMA 279: 1566 –1570, 1998 53. Jadad AR, Moher M, Browman GP, Booker L, Sigouin C, Fuentes M, Stevens R: Systematic reviews and meta-analyses on treatment of asthma: Critical evaluation. BMJ 320: 537–540, 2000 54. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP: Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352: 609 – 613, 1998 55. Katerndahl DA, Lawler WR: Variability in meta-analytic results concerning the value of cholesterol reduction in coronary heart disease: A meta-meta-analysis. Am J Epidemiol 149: 429 – 441, 1999 56. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F: Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 337: 536 –542, 1997 57. Counsell C: Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 127: 380 –387, 1997 58. Vickers A, Goyal N, Harland R, Rees R: Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 19: 159 –166, 1998 59. Gregoire G, Derderian F, Le Lorier J: Selecting the language of the publications included in a meta-analysis: Is there a Tower of Babel bias? J Clin Epidemiol 48: 159 –163, 1995 60. Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G: Language bias in randomised controlled trials published in English and German. Lancet 350: 326 –329, 1997 61. Moher D, Pham B, Klassen TP, Schulz KF, Berlin JA, Jadad AR, Liberati A: What contributions do languages other than English make on the results of meta-analyses? J Clin Epidemiol 53: 964 –972, 2000 62. Juni P, Holenstein F, Sterne J, Bartlett C, Egger M: Direction and impact of language bias in meta-analyses of controlled trials: Empirical study. Int J Epidemiol 31: 115–123, 2002 63. Casas JP, Chua W, Loukogeorgakis S, Vallance P, Smeeth L, Hingorani AD, MacAllister RJ: Effect of inhibitors of the renin-angiotensin system and other antihypertensive drugs on renal outcomes: Systematic review and meta-analysis. Lancet 366: 2026 –2033, 2005 64. Strippoli GF, Craig MC, Schena FP, Craig JC: Role of blood pressure targets and specific antihypertensive agents used to prevent diabetic nephropathy and delay its progression. J Am Soc Nephrol 17: S153–S155, 2006 65. Subramanian S, Venkataraman R, Kellum JA: Influence of dialysis membranes on outcomes in acute renal failure: a meta-analysis. Kidney Int 62: 1819 –1823, 2002 260 Clinical Journal of the American Society of Nephrology 66. Jaber BL, Lau J, Schmid CH, Karsou SA, Levey AS, Pereira BJ: Effect of biocompatibility of hemodialysis membranes on mortality in acute renal failure: A meta-analysis. Clin Nephrol 57: 274 –282, 2002 67. Teehan GS, Liangos O, Lau J, Levey AS, Pereira BJ, Jaber BL: Dialysis membrane and modality in acute renal failure: Understanding discordant meta-analyses. Semin Dial 16: 356 – 360, 2003 68. Dickersin K, Scherer R, Lefebvre C: Identifying relevant studies for systematic reviews. BMJ 309: 1286 –1291, 1994 69. US National Library of Medicine: MEDLINE Fact Sheet. Available at: http://www.nlm.nih.gov/pubs/factsheets/ medline.html. Accessed March 3, 2007 70. Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, RamosRemus C: Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials 21: 476 – 487, 2000 71. Topfer LA, Parada A, Menon D, Noorani H, Perras C, SerraPrat M: Comparison of literature searches on quality and costs for health technology assessment using the MEDLINE and EMBASE databases. Int J Technol Assess Health Care 15: 297–303, 1999 72. Minozzi S, Pistotti V, Forni M: Searching for rehabilitation articles on MEDLINE and EMBASE: An example with crossover design. Arch Phys Med Rehabil 81: 720 –722, 2000 73. EMBASE DataStar Datasheets. Available at: http://ds.datastarweb.com/ds/products/datastar/sheets/emed.htm. Accessed March 3, 2007 74. Cheema BS, Singh MA: Exercise training in patients receiving maintenance hemodialysis: A systematic review of clinical trials. Am J Nephrol 25: 352–364, 2005 75. Clarke KS, Klarenbach S, Vlaicu S, Yang RC, Garg AX: The direct and indirect economic costs incurred by living kidney donors-a systematic review. Nephrol Dial Transplant 21: 1952– 1960, 2006 76. Steinbrook R: Searching for the right search: Reaching the medical literature. N Engl J Med 354: 4 –7, 2006 77. Wilczynski NL, Haynes RB: Robustness of empirical search strategies for clinical content in MEDLINE. Proc AMIA Symp 904 –908, 2002 78. Wilczynski NL, Walker CJ, McKibbon KA, Haynes RB: Reasons for the loss of sensitivity and specificity of methodologic MeSH terms and textwords in MEDLINE. Proc Annu Symp Comput Appl Med Care 436 – 440, 1995 79. Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R: Identification of randomized controlled trials in systematic reviews: Accuracy and reliability of screening records. Stat Med 21: 1635–1640, 2002 80. Davidson RA: Source of funding and outcome of clinical trials. J Gen Intern Med 1: 155–158, 1986 81. Rochon PA, Gurwitz JH, Simms RW, Fortin PR, Felson DT, Minaker KL, Chalmers TC: A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med 154: 157–163, 1994 82. Biondi-Zoccai GG, Lotrionte M, Abbate A, Testa L, Remigi E, Burzotta F, Valgimigli M, Romagnoli E, Crea F, Agostoni P: Compliance with QUOROM and quality of reporting of overlapping meta-analyses on the role of acetylcysteine in the prevention of contrast associated nephropathy: Case study. BMJ 332: 202–209, 2006 83. Egger M, Davey SG, Schneider M, Minder C: Bias in meta- Clin J Am Soc Nephrol 3: 253–260, 2008 analysis detected by a simple, graphical test. BMJ 315: 629 – 634, 1997 84. Berlin JA: Does blinding of readers affect the results of metaanalyses? University of Pennsylvania Meta-analysis Blinding Study Group. Lancet 350: 185–186, 1997 85. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ: Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 17: 1–12, 1996 86. Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, Lau J: Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 287: 2973–2982, 2002 87. Balk EM, Lau J, Bonis PA: Reading and critically appraising systematic reviews and meta-analyses: A short primer with a focus on hepatology. J Hepatol 43: 729 –736, 2005 88. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, Pham B, Klassen TP: Assessing the quality of reports of randomised trials: Implications for the conduct of meta-analyses. Health Technol Assess 3: i–98, 1999 89. Verhagen AP, de Vet HC, de Bie RA, Boers M, van den Brandt PA: The art of quality assessment of RCTs included in systematic reviews. J Clin Epidemiol 54: 651– 654, 2001 90. Juni P, Altman DG, Egger M: Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ 323: 42– 46, 2001 91. Devereaux PJ, Choi PT, El Dika S, Bhandari M, Montori VM, Schunemann HJ, Garg AX, Busse JW, Heels-Ansdell D, Ghali WA, Manns BJ, Guyatt GH: An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol 57: 1232–1236, 2004 92. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S: Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Control Clin Trials 16: 62–73, 1995 93. Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias: Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273: 408 – 412, 1995 94. Laupacis A, Wells G, Richardson WS, Tugwell P: Users’ guides to the medical literature. V. How to use an article about prognosis. Evidence-Based Medicine Working Group. JAMA 272: 234 –237, 1994 95. Garg AX, Suri RS, Barrowman N, Rehman F, Matsell D, Rosas-Arellano MP, Salvadori M, Haynes RB, Clark WF: Long-term renal prognosis of diarrhea-associated hemolytic uremic syndrome: A systematic review, meta-analysis, and meta-regression. JAMA 290: 1360 –1370, 2003 96. Deeks JJ: Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med 21: 1575–1600, 2002 97. DerSimonian R, Laird N: Meta-analysis in clinical trials. Control Clin Trials 7: 177–188, 1986 98. Hardy RJ, Thompson SG: Detecting and describing heterogeneity in meta-analysis. Stat Med 17: 841– 856, 1998 99. Higgins JP, Thompson SG: Quantifying heterogeneity in a meta-analysis. Stat Med 21: 1539 –1558, 2002 100. Thompson SG, Higgins JP: How should meta-regression analyses be undertaken and interpreted? Stat Med 21: 1559 – 1573, 2002 Systematic Review MATERNAL Buprenorphine-naloxone use in pregnancy: a systematic review and metaanalysis Heather M. Link, MD, MPH; Hendree Jones, PhD; Lauren Miller, MD; Karol Kaltenbach, PhD; Neil Seligman, MD Introduction The United States is in the midst of an “opioid crisis” that has recently claimed the lives of more than 90 Americans per day.1 In 2017, a nationwide public health emergency was declared, which led to an increased focus on strengthening treatment and recovery services, data collection, and research to combat this epidemic.2 Recognizing the increasing rates of opioid use disorder (OUD) among pregnant women, the American College of Obstetricians and Gynecologists recommends that healthcare providers engage in universal screening and the provision of comprehensive services for OUD in pregnancy.3e5 These recommendations leverage the window of opportunity that pregnancy provides to affect health behavior changes and acknowledge the risks of untreated OUD in pregnancy. Medication-assisted treatment (MAT) is the mainstay of treatment for OUD during and outside of pregnancy.6e8 However, fetal exposure to opioid agonist medications such as methadone or buprenorphine, like fetal exposure to OBJECTIVE: The goal of this systematic review and metaanalysis is to compare preg- nancy outcomes between pregnant women undergoing treatment for opioid use disorder with buprenorphine-naloxone and those undergoing treatment for opioid use disorder with other forms of medication-assisted treatment. STUDY DESIGN: PubMed, Embase, PsycINFO, Cochrane Clinical Trials, and Web of Science were searched to identify studies assessing the relationship between maternal buprenorphine-naloxone use and pregnancy outcomes. Outcomes assessed included neonatal abstinence syndrome diagnosis and treatment, neonatal intensive care unit admission, length of neonatal hospital stay, delivery complications, mode of delivery, labor analgesia, illicit drug use, medication-assisted treatment dosage, gestational age at delivery, breastfeeding status, miscarriage, congenital anomalies, intrauterine fetal demise, birthweight, head circumference, length, and Apgar scores. RESULTS: Overall, 5 studies comprising 6 study groups met the inclusion criteria. Of the 1875 mother-baby dyads available for analysis, medications prescribed as part of the medication-assisted treatment included buprenorphine-naloxone, buprenorphine alone, methadone, or long-acting opioids. There were no serious adverse maternal or neonatal outcomes associated with maternal buprenorphine-naloxone use reported among any of the studies. Women prescribed with buprenorphine-naloxone for delivered neonates who were less likely to require treatment for neonatal abstinence syndrome were compared with pregnant women prescribed with other opioid agonist medications. Of the remaining outcomes assessed, metaanalysis did not detect any statistically significant differences when comparing the groups of women using buprenorphine-naloxone with the groups of women prescribed with other medications as part of the medication-assisted treatment. CONCLUSION: Pregnant women undergoing treatment for opioid use disorder with buprenorphine-naloxone do not experience significantly different pregnancy outcomes than women undergoing treatment with other forms of opioid agonist medicationassisted therapy. Key words: buprenorphine, buprenorphine-naloxone, medication-assisted treatment, methadone, neonatal abstinence syndrome, opioid use disorder, pregnancy From the Kaleida Health Oishei Children’s Hospital, Buffalo, NY (Dr Link); Department of Obstetrics & Gynecology, University of North Carolina, Raleigh, NC (Dr Jones); St. Luke’s Clinic, Boise, ID (Dr Miller); Department of Pediatrics, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, PA (Dr Kaltenbach); and Division of Maternal-Fetal Medicine, Department of Obstetrics & Gynecology, University of Rochester Medical Center, Rochester, NY (Dr Seligman). Received June 28, 2020; accepted July 1, 2020. The authors report no conflict of interest. This research did not receive financial support. Corresponding author: Heather M. Link, MD, MPH. hmlink@gmail.com 2589-9333/$36.00 ª 2020 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.ajogmf.2020.100179 all opioids, carries a risk of neonatal abstinence syndrome (NAS). NAS results from the abrupt discontinuation of in utero opioid exposure at birth and manifests clinically as a constellation of signs and symptoms that reflect dysregulation of the gastrointestinal, respiratory, central, and autonomic nervous systems.9 Methadone has been the cornerstone of MAT in pregnancy since the late 1960s.10 Buprenorphine was approved in 2002 for treatment of OUD in the United States11 and is, generally, a more favorable treatment option because of its superior safety profile and easier access (the medication is available from waivered providers and can be prescribed in a medical office).7 A growing body of evidence has documented the use and relative safety of buprenorphine during pregnancy.4,12,13 Treatment with buprenorphine in pregnancy is associated with a lower incidence of NAS and shorter duration of NAS treatment than MAT with methadone.13 The initial studies examining the relative safety and maternal and neonatal treatment outcomes related to buprenorphine pharmacotherapy in pregnancy used the AUGUST 2020 AJOG MFM 1 Systematic Review Maternal AJOG MFM at a Glance Why was this study conducted? Information regarding the use of buprenorphine-naloxone during pregnancy for medication-assisted treatment (MAT) of opioid use disorder (OUD) is limited. This systematic review and metaanalysis was conducted to review the available evidence and synthesize evidence-based recommendations for women using buprenorphine-naloxone in pregnancy. Key findings Pregnant women receiving MAT for OUD with buprenorphine-naloxone have similar pregnancy outcomes when compared with women undergoing treatment with other forms of MAT. What does this add to what is known? Healthcare providers can use these results to both reassure patients regarding pregnancy outcomes and be thoughtful about any consideration for adjustment of MAT. For women restricted in their access to MAT, these results can be utilized to further support the use of buprenorphine-naloxone. buprenorphine monoproduct as it was the only product available at the time the studies were initiated.13e16 Since then, buprenorphine was also approved as a combination product with naloxone to decrease the risk of diversion and misuse.17 Although there is no strong reason to believe that buprenorphinenaloxone will result in different clinical outcomes, the current robust and highquality evidence regarding the safety and efficacy of buprenorphine in pregnancy exists for buprenorphine alone. Information regarding the use of buprenorphine-naloxone during pregnancy is limited to smaller cohorts and case series.18e24 Although clinical guidance has historically recommended switching women from buprenorphinenaloxone to buprenorphine alone because of the lack of relative safety data on the combination produced, switching can result in unintended problems for women (eg, lack of buprenorphine availability, vulnerability of relapse because of switching medications, etc.).4,25,26 Currently, a systematic review and metaanalysis of the collected evidence regarding the relative safety and efficacy of OUD treatment with buprenorphine-naloxone during pregnancy is needed to provide an evidence base for recommendations regarding the use of this medication and whether or not women should be urged to switch to 2 AJOG MFM AUGUST 2020 buprenorphine alone if they are newly pregnant or considering pregnancy. Objective The goal of this systematic review and metaanalysis is to compare pregnancy outcomes between pregnant women undergoing treatment for OUD with buprenorphine-naloxone and those undergoing treatment for OUD with other forms of MAT. Methods Search strategy and eligibility criteria This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (2009) framework guidelines. We conducted systematic manual searches on PubMed, Embase, PsycINFO, Cochrane Clinical Trials, and Web of Science through October 2019 to identify all published observational and retrospective cohort studies and randomized controlled trials assessing the relationship between maternal buprenorphine-naloxone use and pregnancy outcomes. The following key words, in different combinations and Medical Subject Headings, were used to identify relevant studies: “pregnant,” “opioids,” “neonatal abstinence syndrome,” “buprenorphine,” “naloxone,” and “methadone.” Additional search terms included generic names, brand names, and synonyms for all the listed pharmaceuticals. The search was restricted to full-text English-language references. Conference abstracts were excluded as it was not always possible to determine from the abstract whether buprenorphine or buprenorphinenaloxone were included in the data presented. We excluded cross-sectional studies, guidelines, expert opinion, editorials, letters to the editors, and comments (Supplemental Table). Study selection Data were screened and extracted by a single investigator (H.L.). Outcomes that were assessed included NAS diagnosis and treatment, neonatal intensive care unit (NICU) admission, length of neonatal hospital stay, delivery complications, mode of delivery, labor analgesia, illicit drug use, MAT dosage, gestational age at delivery, breastfeeding status, miscarriage, congenital anomalies, intrauterine fetal demise, birthweight, head circumference, length, and Apgar scores. Data synthesis Assessment of bias was performed using an approach similar to that described by the Cochrane nonrandomized study group.27 With this approach, differences in baseline characteristics are compared to evaluate for selection bias because the factors determining to which group a woman is allocated are often unknown. The study design and similarity of treatment and control groups for the following 8 characteristics were evaluated to estimate the risk of bias: preterm birth, breastfeeding, active psychiatric disease, use of psychiatric medications (benzodiazepines and/or selective serotonin reactive inhibitors), smoking, delivery dose of MAT, multimodal treatment for NAS, and ongoing illicit drug use. These characteristics were identified by the authors as potential sources for confounding when evaluating neonatal outcomes after exposure to MAT. If a study reported a statistically significant (P