Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Addiction. Author manuscript; available in PMC 2011 January 27.
Published in final edited form as:
PMCID: PMC2888616

How confident should we be that smoking cessation treatments work?



To determine a) the concordance among recent meta-analyses about which treatments for smoking cessation are vs are not efficacious, b) the similarity of odds ratios (ORs) across meta-analyses and c) among the validated treatments, the proportion of studies that found higher quit rates


Computerized literature search for meta-analyses in the last 5 yrs in PubMed and PsychInfo. Data were extracted from summary tables of overall effect of validated treatments


The 14 meta-analyses agreed 100% on the presence/absence of efficacy of 17 proven treatments. The ORs differed by < 0.5 in 72/76 of the comparisons of meta-analyses. Among the 37 meta-analyses that reported individual study results, in 33 meta-analyses, > 85% of the individual studies reported numerical superiority for the active treatment.


The efficacy of treatments for smoking cessation are extremely reliable. This argues for inclusion of treatment as an essential feature of tobacco control and clinical practice and argues for reimbursement of smoking cessation treatments on par with other medical and behavioral disorders.

Keywords: meta-analysis, smoking, smoking cessation, substance-related disorders, rehabilitation, therapy


As many as 70% of smokers do not believe smoking cessation treatments are effective [1]. In addition, many tobacco control advocates and health care funders appear to undervalue smoking cessation [2]. For example, a recent article on “Global regulatory strategies for tobacco control” in JAMA omitted smoking cessation products and services [3]. One possible reason for this undervaluing of treatment is that tobacco advocates and health care funders are unaware of how reliable the data are that smoking cessation treatments work. Recent releases of the two most-cited meta-analyses; i.e. those by the USPHS Treating Tobacco Use and Dependence Clinical Practice Guideline (www.surgeongeneral/tobacco) and the Cochrane Database of Systematic Reviews (, provide an opportunity to better illustrate this reliability.


To examine the reliability of treatment effects for smoking cessation, I first located the individual meta-analyses tables in the 2008 USPHS Guideline [4] (Table 1). Next, I searched the appropriate Cochrane Reviews for the most similar meta-analyses [5-15]. For brevity, I compared only meta-analyses of generic use of treatments in the general population of smokers trying to quit; i.e., I did not include sub-analyses of different subsamples, treatment procedures, etc. Given the focus of this paper, I only examined meta-analysis of treatments that either the USPHS or Cochrane meta-analyses concluded were validated treatments. The investigators who conducted the USPHS and Cochrane meta-analyses (see acknowledgements).then reviewed these results for accuracy. This search resulted in 17 treatments to compare between the USPHS and Cochrane guidelines (the specific Cochrane meta-analyses and the specific comparisons can be obtained from the author). The median (and range) of number of studies/meta-analysis was 11 (2-93) among the USPHS meta-analyses and 14.5 (4-53) among the Cochrane meta-analyses. Only the Cochrane provided total sample sizes and the median was 6385 participants/meta-analysis (417-22,240). When all NRTs were grouped in the Cochrane the number of studies was 111 and the number of participants 43,040.

Table 1
Odds ratios (ORs) (and proportion of comparisons with numerical superiority) for treatment in meta-analyses of recommended smoking cessation treatments

Next, I searched Pub Med ( and PsychInfo ( using the keywords “smok* OR tobacco OR nicotin* OR cigar*) crossed with (meta-analy* OR “systematic review”) and restricted the search to the last 5 yrs. This resulted in 46 meta-analyses. From these I excluded those that evaluated a) a treatment for which neither the USPHS nor Cochrane did a meta-analysis (e.g. cytisine), b) a specific subgroup of smokers (e.g. among pregnant smokers), c) a specific setting (e.g. over-the-counter), or d) specific procedures within a given treatment (e.g. pre-treatment with NRT). This resulted in 12 other meta-analyses of generic treatment effects, in the general population of smokers trying to quit [16-27].

To examine “statistical reliability” across meta-analyses, I compared whether the meta-analyses were concordant on whether treatment effects did or did not meet the meta-analytic statistical criterion and also compared the numerical value of the reported Odds Ratios (OR). The more recent Cochrane reviews report Risk Ratios rather than OR; however the Cochrane group recalculated these as ORs for this paper so that a more fair comparison could be made. To examine “experimental reliability,” I examined, within each Cochrane meta-analysis, what proportion of all the within-study comparisons showed a numerical superiority for the treatment. This information was not available for the USPHS meta-analysis..


Before discussing the outcomes, readers should be aware the USPHS, Cochrane and other meta-analyses differed in many ways; e.g., the exact comparison being made, study inclusion criteria, use of usual care/placebo/no-treatment control groups, length of follow-up, definition of abstinence, use of biochemical verification, publication year, amount of adjunctive treatment, statistical tests, setting, and use of unpublished studies. Prior papers have discussed how these could influence meta-analytic outcomes [28].

Despite these differences, the USPHS, Cochrane and other reviews agreed on statistical significance (yes/no) on 17 of 17 treatments and neither recommended a treatment that the other did not. (The reviews also had a high concordance on which treatments were not efficacious -these data are available from the author). In addition, the ORs were strikingly similar (Table 1). When I used the USPHS or Cochrane as the gold standard, and compared the USPHS vs Cochrane ORs plus compared the other 14 meta-analyses vs both the USPHS and Cochrane meta-analyses, this resulted in 76 comparisons of ORs. The OR from one meta-analysis differed by < 0.5 from the OR in the other meta-analysis in 72 of the 76 (95%) comparisons. This concordance is especially striking given that many meta-analyses of the same treatment differed dramatically in methods. For example, the USPHS meta-analysis of nicotine gum included 15 comparisons of gum vs a control but the Cochrane included 53 comparisons. Even so the OR for the USPHS was 1.5 and for the Cochrane was 1.6.

The experimental reliability of effects was also striking (Table 1). Among the 37 meta-analyses that reported individual study results, in 33 meta-analyses, > 85% of the studies found a numerical superiority for active treatment over placebo. For example, among the 111 comparisons of NRT in the Cochrane meta-analyses, 102 (92%) reported numerical superiority for NRT. Similarly, among the 31 comparisons of bupropion in the Cochrane review, 31 (100%) reported a numerical superiority


The concordance of OR across the meta-analyses (despite multiple methodological differences and very different sample sizes) plus the high rates of numerical superiority for treatments, suggest the efficacy of smoking cessation treatments is extremely reliable. Such high reliability of treatment effects is consistent with other indirect evidence; i.e., cessation treatments effects have been found to persist across funders, settings, samples, outcome definitions, and amount of adjunctive treatment [14,16,29]. For example, nicotine replacement therapies (NRTs) have been effective in those in an intensive counseling session and those in an over-the-counter setting [30], in smokers differing in motivation [31], and in smokers of different races [32]. Although the high rates of numerical superiority could be due to publication bias [33], almost all meta-analyses that tested for publication bias in smoking cessation meta-analyses did not find it [21,22,26,27,29]. The high proportion of comparisons that report numerical superiority for treatment of smoking appears to be greater than that for many other treatments of behavioral disorders; e.g. several marketed antidepressants and antipsychotics are effective in only half to two-thirds of the RCTs conducted [34]. On the other hand, the proportion of numerically successful trials of acamprosate and naltrexone treatments for alcohol problems is quite high [35].

This strong statistical and experimental reliability is as great as, if not greater than, that for tobacco control policies [2]. For example, the Cochrane meta-analyses for sales restrictions [36], media [37,38], prevention [39,40], community interventions [41,42] and public restrictions [43] have fewer numbers of studies, often use non-randomized design, and report smaller proportions of positive studies or smaller ORs or effect sizes.

The strong reliability of treatment effects is important evidence to argue that provision of smoking cessation treatment should be an essential part of both tobacco control efforts and clinical practice and [44]. It is also a strong argument that treatments for smoking cessation should be reimbursed on par with other behavioral and medication treatments.


Supported by Senior Scientist Award DA000490 from the US National Institute on Drug Abuse. I thank Bruce Christiansen, Mike Fiore, Lindsay Stead, and Wendy Theobald for providing results and verifying accuracy of Cochrane (LS) and USPHS (BC, MF, WT) results in Table 1.


Potential Conflicts of Interest : In the last 3 yrs, Dr Hughes has received research grants from the US National Institutes of Health and Pfizer Pharmaceuticals. He has received consulting and speaking fees from multiple governmental, nonprofit and for-profit organizations or companies that develop, sell or promote smoking cessation products and services, including those discussed in this article.


(1) Shiffman S, Ferguson SG, Rohay J, Gitchell JG. Perceived safety and efficacy of nicotine replacement therapies among US smokers and ex-smokers: relationship with use and compliance. Addiction. 2008;103:1371–1378. [PubMed]
(2) Hughes JR. Tobacco control findings vs scientific evidence. American Journal of Preventive Medicine. 2007;32:449. [PMC free article] [PubMed]
(3) Gostin LO. Global regulatory strategies for tobacco control. JAMA. 2007;287(17):2057–2059. [PubMed]
(4) Fiore M, Jaen C, Baker T, Bailey W, Benowitz N, Curry S, et al. US Public Health and Human Services; Washington, DC: 2008. Treating Tobacco Use and Dependence: Clinical Practice Guideline.
(5) Stead LF, Lancaster T. Group behaviour therapy programmes for smoking cessation. The Cochrane Library; 2005. [PubMed]
(6) Lancaster T, Stead LF. Individual behavioural counselling for smoking cessation. The Cochrane Library; 2005. [PubMed]
(7) Stead LF, Bergson G, Lancaster T. Physician advice for smoking cessation. The Cochrane Library; 2008. [PubMed]
(8) Rice VH, Stead LF. Nursing interventions for smoking cessation. The Cochrane Library; 2008. [PubMed]
(9) Stead LF, Perera R, Lancaster T. Telephone counseling for smoking cessation. The Cochrane Library; 2006. [PubMed]
(10) Hajek P, Stead LF. Aversive smoking for smoking cessation. The Cochrane Library; 2001.
(11) Lancaster T, Stead LF. Self-help interventions for smoking cessation. The Cochrane Library; 2005. [PubMed]
(12) Hughes, Stead LF, Lancaster T. Antidepressants for smoking cessation. The Cochrane Library; 2007. [PubMed]
(13) Gourlay SG, Stead LF, Benowitz NL. Clonidine for smoking cessation (Review) The Cochrane Library; 2004. [PubMed]
(14) Stead L, Perera R, Bullen C, Mant D, Lancaster T. Nicotine replacement therapy for smoking cessation. The Cochrane Library; 2008. [PubMed]
(15) Cahill K, Stead LF, Lancaster T. Nicotine receptor partial agonists for smoking cessation. The Cochrane Library; 2008. [PubMed]
(16) Fagerstrom K-O. Clinical treatment of tobacco dependence: The endurance of pharmacologic efficacy. Journal of Clinical Psychiatry. 2003;18:35–40.
(17) Bala MM, Lesniak W. Efficacy of non-pharmacological methods used for treating tobacco dependence: Meta-analysis. Polskie Archiwum Medycyny Wewnetrnej. 2007;117:504–511. [PubMed]
(18) Bala MM, Lesniak W, Strzeszynski L. Efficacy of pharmacological methods used for treating tobacco dependence: Meta-analysis. Polskie Archiwum Medycyny Wewnetrnej. 2008;118:20–28. [PubMed]
(19) Eissenberg MJ, Filion KB, Yavin D, Belisle P, Mottillo S, Joseph L, et al. Pharmacotherapies for smoking cessation : A meta-analysis of randomized controlled trials. Canadian Medical Association Journal. 2008;179:135–144. [PMC free article] [PubMed]
(20) Etter JF, Stapleton JA. Nicotine replacement therapy for long-term smoking cessation: A meta-analysis. Tobacco Control. 2006;15:280–285. [PMC free article] [PubMed]
(21) Myung SK, Yoo KY, Oh SW, Park SH, Seo HG, Hwang SS, et al. Meta-analysis of studies investigating one-year effectiveness of transdermal nicotine patches for smoking cessation. American Journal of Health-System Pharmacy. 2007;64:2471–2476. [PubMed]
(22) Pan W. Proactive telephone counseling as an adjunct to minimal intervention for smoking cessation: A meta-analysis. Health Education Research. 2006;21:416–427. [PubMed]
(23) Rice VH. Nursing intervention and smoking cessation: Meta-analysis update. Heart and Lung. 2006;35:147–163. [PubMed]
(24) Wu P, Wilson K, Dimoulas P, Mills EJ. Effectiveness of smoking cessation therapies: A systematic review and meta-analysis. BioMed Central Public Health; 2006. 6. [PMC free article] [PubMed]
(25) Wagena EJ, Knipschild P, Zeegers MPA. Should nortriptyline be used as a first-line aid to help smokers quit. Results from a systematic review and meta-analysis. Addiction. 2005;100:317–326. [PubMed]
(26) Gorin SS, Heck JE. Meta-analysis of the efficacy of tobacco counseling by health care providers. Cancer Epidemiology, Biomarkers & Prevention. 2004;13:2012–2022. [PubMed]
(27) Mojica WA, Suttorp MJ, Sherman SE, Morton SC, Roth EA, Maglione MA, et al. Smoking-cessation interventions by type of provider: A meta-analysis. American Journal of Preventive Medicine. 2004;26:391–401. [PubMed]
(28) Jadad AR, Cook DJ, Browman GP. A guide to interpreting discordant systematic reviews. Canadian Medical Assocation Journal. 1997;156:1411–1416. [PMC free article] [PubMed]
(29) Etter JF, Burri M, Stapleton J. The impact of pharmaceutical company funding on results of randomized trials of nicotine replacement therapy for smoking cessation: A meta-analysis. Addiction. 2007;102:815–822. [PubMed]
(30) Hughes JR, Shiffman S, Callas P, Zhang J. A meta-analysis of the efficacy of over-the-counter nicotine replacement. Tobacco Control. 2002;12:21–27. [PMC free article] [PubMed]
(31) Hughes JR, Carpenter MJ. Does smoking reduction increase future cessation and decrease disease risk? A qualitative review. Nicotine and Tobacco Research. 2006;8:739–749. [PubMed]
(32) Robles GI, Singh-Franco D, Ghin HL. A review of the efficacy of smoking-cessation pharmacotherapies in nonwhite populations. Clinical Therapy. 2008;30:800–812. [PubMed]
(33) Dubben HH, Beck-Bornholdt HP. Systematic review of publication bias in studies on publication bias. British Medical Journal. 2005;331:433–434. [PMC free article] [PubMed]
(34) Temple R, Ellenberg SS. Placebo-controlled trials and active-control trials in the evaluation of new treatments. Part 1. Ethical and scientific issues. Annals of Internal Medicine. 2000;133:455–463. [PubMed]
(35) Snyder J, Bowers T. The efficacy of Acamprosate and Naltrexone in the treatment of alcohol dependence: A relative benefits analysis of randomized controlled trials. The American Journal of Drug and Alcohol Abuse. 2008;34:449–461. [PubMed]
(36) Stead LF, Lancaster T. Interventions for preventing tobacco sales to minors. The Cochrane Library; 2005. [PubMed]
(37) Bala M, Strzeszynski L, Cahill K. Mass media interventions for smoking cessation in adults. The Cochrane Library; 2008. [PubMed]
(38) Sowden AJ, Arblaster L. Mass media interventions for preventing smoking in young people. The Cochrane Library; 1999. [PubMed]
(39) Thomas RE, Baker P, Lorenzetti D. Family-based programmes for preventing smoking by children and adolescents. The Cochrane Library; 2007. [PubMed]
(40) Thomas R, Perera R. School-based programmes for preventing smoking. Issue 3. The Cochrane Library; 2006. . 2006. Cochrane Database of Systematic Reviews.
(41) Secker-Walker RH, Gnich W, Platt S, Lancaster T. Community interventions for reducing smoking among adults. The Cochrane Library; 2002. [PubMed]
(42) Sowden A, Arblaster L, Stead L. Community interventions for preventing smoking in young people. Cochrane review. The Cochrane Library; 2002. [PubMed]
(43) Serra C, Cabezas C, Bonfill X, Pladevall-Vila M. Interventions for preventing tobacco smoking in public places. The Cochrane Library; 2006. [PubMed]
(44) Fiore MC, Croyle RT, Curry SJ, Cutler CM, Davis RM, Gordon C, et al. Preventing 3 million premature deaths and helping 5 million smokers quit: A National Action Plan for Tobacco Cessation. American Journal of Public Health. 2004;94:205–210. [PubMed]