PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nictobLink to Publisher's site
 
Nicotine Tob Res. 2011 July; 13(7): 512–522.
Published online 2011 April 6. doi:  10.1093/ntr/ntr055
PMCID: PMC3129240

Effectiveness of Over-the-Counter Nicotine Replacement Therapy: A Qualitative Review of Nonrandomized Trials

John R. Hughes, M.D.,corresponding author1,2,3 Erica N. Peters, Ph.D.,4 and Shelly Naud, Ph.D.5

Abstract

Aim:

Randomized trials conducted in over-the-counter (OTC) settings have shown that nicotine replacement therapy (NRT) is effective. This paper reviews nonrandomized tests of the effectiveness of OTC NRT.

Methods:

Literature search via computer and other methods located (a) retrospective cohort studies of users versus nonusers of OTC NRT and (b) studies of quit rates before versus after NRT went OTC or before versus after NRT was given free to quitline callers. The methods were too heterogeneous to allow meta-analysis.

Results:

The results were similar for cohort and pre- versus post-studies. Most of the studies found numerically greater quitting among NRT users than nonusers. Often when NRT was not found effective, other assumed effective treatments (e.g., phone counseling) were also not found effective, suggesting biased or insensitive study methods. Only about half of the studies found statistically greater quitting among NRT users, and the most rigorous studies did not find greater quitting among users. Many studies found selection bias, for example, NRT users are more dependent smokers.

Conclusions:

Some lines of evidence appear to confirm the effectiveness of OTC NRT, but others do not. We believe further secondary analyses using nonrandomized comparisons are unlikely to resolve this issue due to sensitivity, specificity, and selection bias problems.

Introduction

The purpose of this review is to investigate whether over-the-counter (OTC) nicotine replacement therapy (NRT) is “effective.” Effectiveness is usually defined as showing a treatment effect in a study with high external validity, that is, a study that uses a relatively unselected sample and employs treatment under the conditions in which a treatment is intended to be used (Gartlehner, Hansen, Nissman, Lohr, & Carey, 2006; Nash, McCrory, Nicholson, & Andrasik, 2005; Prochaska, Evers, Prochaska, Van Marter, & Johnson, 2007). In contrast, efficacy is usually defined as showing a treatment effect in a study with high internal validity, that is, a study that uses highly motivated participants, standardized treatment protocols, and under an ideal highly controlled research environment (Gartlehner et al., 2006; Nash et al., 2005; Prochaska et al., 2007).

Based on over 100 randomized controlled trials (RCTs) of over 40,000 participants, all meta-analyses in the last five years have concluded that all NRTs are efficacious; typically, odds ratios (ORs) for NRT in these meta-analyses are 1.5–2.0 (Hughes, 2009). Most of the NRTs (nicotine gum, inhaler, nasal spray, lozenge, microtab, and patch) were initially marketed as prescription (Rx) medications; however, when it became clear that having to see a physician was a barrier to access to NRT (Shiffman & Sweeney, 2007), almost all were approved for OTC sale. Currently, OTC NRT is, by far, the most widely used treatment for smoking cessation. In the United States, one third of those who try to stop smoking use OTC NRT (Shiffman, Brockwell, Pillitteri, & Gitchell, 2008b).

Several medications that were efficacious in RCTs appear to not be effective when used in real-world settings (Walsh, 2008). With NRT, the absence of professional advice, the inclusion of less-motivated smokers or poor compliance, might undermine NRT effectiveness (Walsh, 2008). The optimal way to test effectiveness is via prospective controlled trials in effectiveness settings. Several controlled trials examined NRT in OTC-like settings (e.g., store-front settings with no advice given). Our meta-analysis of these trials concluded that OTC NRT was effective (Hughes, Shiffman, Callas, & Zhang, 2003); however, the number of OTC trials in our analysis was small (n = 7) and some used nonrandomized designs.

Although RCTs in real-world situations are the most valid measure of effectiveness, volunteer bias may still occur (Amori & Lenox, 1989) plus the monitoring and structure of a RCT could influence results. Several nonrandomized studies have been reported, and their results could provide a different test of the effectiveness of OTC NRT. The two designs used have been retrospective cohort, and pre- versus post-studies. Retrospective cohort studies have compared the abstinence rates of those who chose to use OTC NRT on a quit attempt with those who chose not to do so with the null hypothesis of no difference in successful cessation in the two groups. These studies have been done with convenience and population-based samples. In this review, we have also included retrospective cohort studies that used samples from quitlines and in-person treatments that provided free OTC NRT. These are less-valid tests of effectiveness because, although not documented, it is likely the treatments gave advice about use of NRT and thus have some Rx features to them. Nevertheless, for completeness, we include their results.

One asset of retrospective cohort studies is that their samples usually are more externally valid than those of the RCTs, that is, most retrospective cohort samples have few inclusion criteria and most are of smokers not enrolled in a formal treatment program. The major liability to retrospective cohort studies is that smokers self-select into these groups. Several lines of evidence indicate smokers who choose to use NRT are different than those who choose not to use NRT (Shiffman, Brockwell, Pillitteri, & Gitchell, 2008a). It is an almost universal finding that those with more severe illnesses are more likely to seek treatment; these phenomena have been labeled “indication bias” (Shiffman et al., 2008a). In fact, NRT users are heavier and more dependent smokers and have had more difficulty quitting in the past (Shiffman, Di Marino, & Sweeney, 2005). Retrospective cohort studies attempt to correct for such “confounds” by using post-hoc covariates, but most of these studies come from surveys in which there is limited information on the relevant confounds. Another problem is that most retrospective cohort studies use retrospective recall to assess quit attempts, which can be biased. For example, smokers forget many quit attempts (Berg et al., 2010; Gilpin & Pierce, 1994), and they may be more likely to recall treatments in which NRT was used than in those in which it was not used.

Other studies have compared abstinence rates between Rx NRT and OTC NRT periods, which we will label “pre- versus post-studies” (Campbell & Stanley, 1966). These studies are typically population surveys that test whether quit rates were similar in the OTC and Rx periods. Like the retrospective cohort studies, the pre- versus post-studies should have more externally valid samples than efficacy trials. Their major liabilities are the self-selection bias described above plus historical confounds (Shiffman et al., 2008a). For example, if the population of smokers is “hardening” over time, that is, as prevalence of smoking falls, remaining smokers are those who are more dependent, have more problems of living, etc (Warner & Burns, 2003), this could falsely lower OTC quit rates compared with Rx NRT quit rates.

We have included studies of quitlines in pre- versus post-studies. Even though these studies did not directly test OTC NRT, they do report quit rates when the quitlines did not provide free NRT and then after they did so (the latter always occurred during the OTC period). Thus, if OTC NRT is not effective, then the quit rates before and after free NRT should be identical. This is probably the weakest test of OTC NRT for two reasons. First, as mentioned above, quitlines may have given advice on use of NRT and thus did not differ substantially from prescription (Rx) NRT. Second, it is likely highly dependent smokers were not willing to call a quitline when no NRT was offered but decided to attend when they heard free NRT was available.

Importantly, a few of the studies also reported not only just on the effectiveness of OTC NRT but also on the effectiveness of Rx medications and counseling (Fiore et al., 2000) and, thus, can be used to assess the specificity of any negative results. For example, assume OTC NRT users have quit rates similar to that in nonusers. If, in that same study, counseling was found effective, this would suggest a true negative result for OTC NRT. If, on the other hand, counseling was also not found effective, then (if one believes quitlines are truly effective treatments) this result would suggest the study methods were insensitive to detect changes in quit rates due to use of treatment.

Methods

To locate studies, the first author searched PubMed and PsycInfo using the terms “(replacement OR transdermal OR patch OR gum OR inhaler OR tablet OR lozenge) AND (quit* OR stop* OR cessation OR treat*) AND (tobacco OR smok* OR cigar*)” and from 1990 to 2009 (only Switzerland had OTC NRT prior to this date). He also searched the Cochrane Database of Randomized Trials (http://www.cochrane.org) and his own set of articles. The first author located 769 articles whose titles suggested that they were applicable and read their abstracts. This led to 71 articles that appeared to be applicable. Next, the first two authors independently screened these 71 for the following inclusion criteria: (a) examined abstinence rates among those trying to quit (one included study reported survival analyses, not abstinence; Gilpin, Messer, & Pierce, 2006) and (b) not an efficacy trial (see definition of efficacy above). Since we were interested in success from a given quit attempt, another inclusion criterion was available data on success among those who attempted to quit. Thus, our use of the term “quit rate” refers not to abstinence in the whole sample but to abstinence only among those who attempted to quit. In addition, the study had to either compare abstinence between those using versus not using NRT when they tried to quit (retrospective cohort studies) or examined quit rates before versus after NRT was available to those trying to quit (pre- vs. post-studies). A list of excluded studies and the reasons for exclusion can be found at http://www.uvm.edu/~hbpl/pdfs/Effectiveness.excluded.studies.pdf.

The first two authors independently coded each study for how the participants were recruited, intent-to-treat (ITT) sample sizes, demographics and smoking history of the sample, whether other active treatments (e.g., phone counseling) occurred, exact NRT used, duration of follow-up, incidence of missing data at follow-ups, procedures for missing data, number abstinent, use of covariates in analyses, and reported adjusted odds ratio (AOR). Although we did not record the rate of disagreements by the two authors, it was not insubstantial, often because the paper failed to report methodological details. Disagreements were resolved by discussion. We sent a draft of the paper to all the corresponding authors listed in the Tables and asked for any corrections or comments. Few replied. The results of our coding are available at http://www.uvm.edu/~hbpl/?Page=codingmanual.xls.

Results

Retrospective Cohort Studies

We located 11 retrospective cohort studies that provided 14 comparisons of the quit rates for OTC NRT users versus nonusers (Table 1). The most common reasons for exclusion were that the study (a) did not compare NRT users versus nonusers; (b) compared NRT users and nonusers on outcomes other than abstinence (e.g., dependence); (c) examined NRT use and abstinence among all smokers, not just those who had tried to quit; (d) examined lifetime use of NRT and lifetime abstinence, rather than success in a recent quit attempt; and (e) reported data in a form such that we could not verify calculation of fraction abstinent in users versus nonusers. The two exceptions were that we included in the results section a widely cited population-based study on NRT effectiveness (Pierce & Gilpin, 2002), even though we could not obtain actual numerators and denominators from the report.

Table 1.
Methods of Retrospective Cohort Studiesa

The 11 studies varied widely in sampling frame/setting and can be divided into three groups: population-based samples (n = 4 studies per 4 comparisons), convenience samples (n = 3 studies per 6 comparisons), and treatment samples (n = 4 studies per 4 comparisons). The studies also varied in control groups, time of follow-up, definition of abstinence, and amount of missing data. Based on these methodological differences, we believed that the studies were too methodologically heterogeneous to conduct a meta-analysis (Slavin, 1995). In fact, the results were extremely heterogeneous (I2 = 96% heterogeneity, Q (10) = 271, p < .0001; Higgins & Thompson, 2002). As an alternative, we present a qualitative review of their outcomes. Before examining results, we briefly review the studies’ methods and their conclusions (Table 1).

Retrospective Cohort Population-Based Samples

Gilpin et al. (2006) compared NRT users versus nonusers among respondents to the 1999 and 2002 CA, USA, Tobacco Surveys. The results are presented only for those who smoked ≥ 15 cigarettes (cigs)/day a year earlier. The study did not report on abstinence at a follow-up but did report that NRT use was associated with less rapid relapse in a survival curve analysis. This study found a similar result for bupropion users. We used the last timepoint in the survival curve as the quit rate.

Pierce and Gilpin (2002) used three CA Tobacco Surveys to compare quit rates among moderate to heavy smokers who were NRT users versus nonusers, before and after NRT went OTC and, thus, can be used in both the retrospective cohort and pre- versus post-OTC NRT analyses. This study did not provide data such that we could calculate actual numerators and denominators for quit rates. Nevertheless, the study was included because it has high internal and external validity and has been highly cited. We estimated quit rates from survival curves in the article using software (Digimatic). The study found that during the OTC period, NRT users had higher quit rates in the short term but not in the long term.

Shiffman et al. (2008b) examined respondents to the 2003 Tobacco Use Cessation Supplement to the Current Population Survey of the United States. Given this is a large national sample and used adjusted analyses and compared several treatments, it is probably the most rigorous of the retrospective cohort studies. In both adjusted and unadjusted analyses, use of OTC NRT was associated with less abstinence. Importantly, a similar finding occurred with bupropion and counseling.

Thorndike (Thorndike, Biener, & Rigotti, 2002) used the 1993–1999 MA, USA, Tobacco Surveys to calculate quit rates among NRT users versus nonusers before and after NRT went OTC and, thus, can be used in both the retrospective cohort and pre- versus post-analyses. Our calculation of its results indicates that the quit rates among NRT users during the OTC period showed a nonsignificant trend to be greater than that among NRT nonusers during the OTC period.

Retrospective Cohort Convenience Samples

Gomez-Zamudio et al. (2004) examined randomly selected participants in the 2002 Quebec, Canada, Quit-and-Win contest. Free pamphlets and use of phone counseling were available but smokers had to pay for medications. Separate results were presented for users of nicotine gum and patch as well as other therapies. The rate of missing data was high (52%); whether the incidence of missing data varied between NRT users and nonusers was not stated; and no adjustments for selection bias were included in the analyses. The study found no benefit associated with use of nicotine gum but a benefit for use of patch. It also found a benefit for use of bupropion but not for counseling.

Solberg et al. (2001) examined the use of several treatments among smokers in a health maintenance organization in MN, USA, who had tried to quit in the last six months. The outcome was not abstinence at long-term follow-up but rather seven or more days of abstinence at some point during the prior six months. The incidence of missing data was 13%. Some adjustments for possible selection bias were used. Use of a nicotine patch was associated with ever achieving seven or more days of abstinence but use of gum was not. Use of bupropion was associated with cessation but use of quit smoking classes or phone counseling was not.

West and Zhou (2007) examined data from surveys conducted in 2003 and 2004, in Canada, France, United Kingdom, and United States, all of which had OTC NRT available. The survey was limited to Internet users between 35 and 65 years who smoked >5 cigs/day. This prospective study followed both those who had used NRT without behavioral support in the last three months and those who had not, for another six months. The outcome was six months of prolonged abstinence. The incidence of missing data was 20%–25%; however, the rate of missing was similar in the NRT and non-NRT groups. In both the unadjusted and adjusted analyses and in both surveys, those who used NRT were twice as likely to stop smoking. This study used an unusual extra comparison group of those using non-validated treatments (e.g., acupuncture) and found their use was not associated with increased quitting.

Retrospective Cohort Treatment Studies

Treatment samples probably are the weakest test of OTC NRT because in some studies, smokers probably received encouragement to use NRT and advice on how to use NRT during treatment, which probably rarely occurs with true OTC use. Hawk et al. (2006) compared upstate New York, USA, smokers who chose a Quit-and-Win contest that did not offer free NRT and who also declined free phone counseling versus those who enrolled in phone quitline counseling that offered free NRT; thus, the comparison of NRT versus no NRT in this study is confounded by choice of no-counseling versus phone counseling. They conducted a follow-up of a random subset of enrollees a mean of 5.5 months later. This study found no benefit for NRT plus counseling in either unadjusted or adjusted analyses.

Miller et al. (2005) compared New York City (NYC), USA, smokers who called a quitline for free nicotine patches and received a mailed patch versus those who called but their patches were returned as undeliverable. The response rate for follow-up in the control group was only 31%, and the prevalence of abstinence in the non-NRT users was unusually low for a quitline (6%). Results were adjusted for several confounders. This study found a much higher association of NRT with cessation than the other retrospective cohort studies.

Swartz, Cowan, Klayman, Welton, and Leonard (2005) examined outcomes in callers to the ME, USA, quitline who did versus did not use free NRT (mostly patch) offered via the quitline. In unadjusted analyses, users of NRT were more likely to be abstinent than those who did not use NRT. In contrast, users of counseling were not more likely to quit.

Tinkelman, Wilson, Willett, and Sweeney (2007) examined quit rates among those in an Ohio quitline who used NRT during the free NRT program versus those who declined to do so in both Rx and OTC periods; thus, its results can be used for both the retrospective cohort and pre- versus post-OTC analyses. After some adjustments, those who chose to use NRT during the free OTC NRT period had a higher quit rate than those who chose not to use NRT.

Collating Results Across Retrospective Cohort Studies

All of the studies had adequate sample sizes (n > 100; Table 1). The demographics of the samples of these studies were fairly similar to each other and appeared to be similar to the average characteristics of U.S. smokers (Hughes & Callas, 2010) except that most studies had few minorities. We report four methods that can be used to draw conclusions. The first is experimental replication across studies, that is, the proportion of studies in which the quit rate for the NRT user group was greater than that in the nonuser group. For this analysis, whenever possible, we recalculated the percent abstinent for NRT users and nonusers, and the unadjusted OR for NRT users versus nonusers, using ITT denominators (Table 2). AORs were taken directly from the reports. If OTC NRT is effective, then the OR for quit rates in OTC NRT users versus nonusers should be greater than 1.0 (to avoid labeling minor increases important, we required > 1.1 for our analyses). In the unadjusted analyses, the OR was numerically greater than 1.1 in 7/11. Given the known selection biases, more weight should be placed on the adjusted analyses, even though their adjustments were incomplete. In the adjusted analyses, OTC NRT was numerically greater than 1.1 in 6/9 comparisons. If data from treatment studies are ignored for the reasons listed above, then the unadjusted OR is numerically greater than 1.1 in 4/8 of the unadjusted comparisons and in 4/6 of the adjusted comparisons. The second method is the fraction of studies that found statistically significant greater quitting among NRT users. About half (5/11) of the unadjusted comparisons show significantly greater quitting with NRT users and all (6/6) of the adjusted analyses did so. Among non-treatment samples, corresponding fractions were 2/6 and 3/8. The third method is examination of the studies, which appear to have the greatest internal and external validity. As described above, the Shiffman et al. study is probably the most rigorous. It did not find NRT to be effective.

Table 2.
Outcomes of Retrospective Cohort Studies

The fourth method is based on specificity, that is, instances in which OTC NRT users did not show increased rates, but users of other drugs or counseling assumed to be effective (bupropion and phone counseling) did show increased rates, that is, inclusion of other drugs or counseling as a “positive control.” This criterion is important to rule out the possibility that instances of failure to show efficacy of NRT are due to biased or insensitive methods. Bupropion was associated with success in 3/4 comparisons, but receipt of counseling was associated with success in 0/4 comparisons (Gilpin et al., 2006; Shiffman et al., 2008b; Solberg et al., 2001). There was no study in which bupropion or counseling was found effective and NRT was not.

Examination of Table 2 also suggests that results supporting the effectiveness of OTC NRT were somewhat less robust for population-based studies than convenience samples which, in turn, were less robust that treatment samples. Adjusted and unadjusted results appeared similar. Finally, the major reason to doubt the efficacy of OTC NRT is hypothesized low compliance (Hughes, 2001; Walsh, 2008). Because compliance with gum is lower than with patch, we examined whether the effectiveness of gum versus patch differed. The two studies that included both NRTs, each found effectiveness for patch but not for gum.

Pre- and Post-Studies

We located seven pre- and post-studies, three of which were also used in the retrospective cohort analyses (Table 3). Three studies used population-based samples and four used treatment samples. The three population-based studies examined abstinence rates among smokers who tried to quit before versus after NRT went OTC. The four treatment studies all examined abstinence rates before versus after the treatment provided free nicotine patches. One study provided separate comparisons of gum, of patch, and of gum or patch; thus, we had a total of nine comparisons. Before describing the results, we describe the studies.

Table 3.
Methods of Pre- and Post-Studiesa

Pre- Versus Post-Population–Based Samples

Hyland, Rezaishiraz, Giovino, Bauer, and Cummings (2005) examined data from the Community Intervention Trial for Smoking Cessation (COMMIT). Although this was not a true population-based sample, the sample was quite large, from several U.S. cities and approximated a random sample of smokers. The study tested policy interventions, which had only small effects. In 2001, the study asked participants about use of NRT over a long recall period, that is, the Rx period (1993–1996) and the OTC period (1997–2001). It also reported on gum and patch users versus nonusers. This analysis was probably the most rigorous among the pre- versus post-studies. The analysis was unusual, in that it did not compare abstinence rates among all who attempted to quit pre- versus post-OTC change, but rather compared abstinence rates only among NRT users during pre-OTC conditions versus among NRT users during OTC conditions. The OTC patch users were not less likely to abstain than the Rx patch users; OTC gum users were actually more, not less, likely to abstain than Rx gum users (see Table 4). When gum and patch use was combined, no difference in rates pre- versus post-OTC switch was found.

Table 4.
Outcomes of Pre- and Post-Studies

The Pierce & Gilpin (2002) study was described among the retrospective cohort studies. When we estimated quit rates from its survival curve data, quit rates decreased in the post-OTC era.

The Thorndike et al. (2002) study was also described among the cohort studies. When we calculated ORs using the data presented in these figures and text, quit rates increased with introduction of OTC NRT.

Pre- Versus Posttreatment Samples

Pre- versus post-studies using treatment samples are probably the weakest test of OTC NRT effectiveness because they do not directly test OTC versus prescription use of NRT. Rather they test the general principle that if one makes NRT more available without physician advice, this will increase quitting. In addition, as stated above, some of these studies likely gave some advice about use of NRT, which would not occur in a true OTC NRT setting.

Alberg et al. (2004) examined the effect of pre- versus post-free NRT in a state health department group treatment program. Although at early follow-ups those offered free NRT had higher quit rates (66% vs. 38%), by the eighteen-month follow-up, this advantage had disappeared and, in fact, those offered NRT appeared to have worse outcomes (7% vs. 14%).

Bush et al. (2008) examined pre- versus post-free NRT in a state quitline that offered a single session of counseling. Those who enrolled when NRT was available appeared to be the more dependent smokers. Nevertheless, in both the unadjusted and adjusted analyses, the six-month abstinence rate in the free NRT condition was twice that when no NRT was available.

Cummings et al. (2006) compared quit rates among NYC smokers who called a quitline prior to and after a free-patch program. In unadjusted analyses, the twelve month quit rate was greater after the patch offer. However, during this same time, a smoking ban occurred in NYC that may have contributed to the increased abstinence rate.

The Tinkelman et al. (2007) study is described among the cohort studies. It also included a pre- versus post-free NRT comparison similar to the above studies. This study found a substantial increase in quit rates with OTC NRT.

Collating Results Across Pre- and Post-Studies

All of the pre- versus post-studies had substantial sample sizes (n > 200). Participant characteristics were similar across studies and similar to those of the average U.S. smoker (Hughes and Callas, 2010) with the exception of the underrepresentation of minorities in all of the studies. The studies varied substantially in the incidence of missing data and how missing data were handled, definitions of abstinence and time of follow-up; plus there were substantial differences in the sampling frame of the quitline and population-based studies, and only a few studies were available within each sampling frame. Again, we believe the methods were too heterogeneous to conduct a meta-analysis, and again, the results were, in fact, heterogeneous (I2 = 86% heterogeneity, Q(7) = 51, p < .0001; Higgins, Thompson, Deeks, & Altman, 2003); thus, we again describe results qualitatively and report the same four criteria to draw conclusions.

If OTC NRT is effective, then in these studies, the OR for post- versus pre-quit rates should be >1.0. The unadjusted ORs were numerically ≥1.1 in 5/9 comparisons and the AORs were ≥1.1 in 3/4 comparisons. If the treatment samples are ignored, then in the unadjusted comparisons, 1/4 showed OTC NRT had greater outcomes and the single adjusted comparison did so as well. In terms of prevalence of statistical significance, 2/9 of the unadjusted and 1/1 of the adjusted comparisons were significant. The Hyland study was thought the most rigorous. It found no effect for OTC NRT overall. Specificity of results could not be tested because no study reported on non-NRT treatments.

As with the cohort studies, the effectiveness of OTC NRT appeared to be somewhat less in the population-based studies than the treatment samples, but adjusted and unadjusted results appeared similar. Finally, only the Hyland et al. study reported separately on gum and patch. It found the opposite of expected, that is, the effectiveness of gum was greater than that for patch.

Discussion

This review examined two types of nonrandomized studies of the effectiveness of OTC NRT: (a) comparisons of smokers who tried to quit and used OTC NRT versus smokers who tried to quit and did not use NRT during the same time period (retrospective cohort studies) and (b) comparisons of quit rates before and after NRT became more widely available either via OTC status or via free offers during treatment (pre- vs. post-studies). These studies were conducted using population-based samples, convenience samples, or treatment samples. Some adjusted for confounders; others did not. Only a few reported on non-NRT treatment to allow tests of specificity.

We reported on four criteria to make decisions: (a) experimental replicability, (b) incidence of statistical significance, (c) results of the most rigorous study, and (d) specificity of effects. The results were similar for cohort and pre- versus post-studies. Experimental replicability was good; most of the studies found numerically greater quitting among NRT users than nonusers. Also, no instances of specificity were found, that is, no instances in which NRT was found ineffective but bupropion and phone counseling were found effective. On the other hand, only about half of the studies found statistically greater quitting among NRT users, and the most rigorous cohort study and pre- versus post-study did not find greater quitting among users. Also, results appeared somewhat less robust in the population-based samples than in the convenience or treatment samples. In summary, the results varied by decision criteria and no firm conclusions can be reached.

We also examined compliance because this is one of the reasons that OTC NRT has been hypothesized to not be effective (Walsh, 2008). Unfortunately, the measures of compliance in the nonrandomized studies varied so widely that we could not test if compliance was less than in treatment settings. Because compliance with patch is typically much better than with gum, one would expect OTC gum to have poorer outcomes than OTC patch. Two of our studies found this (Gomez-Zamudio et al., 2004; Solberg et al., 2001), but the third did not (Hyland et al., 2005). Another possible reason for failure to show NRT use is associated with effectiveness is that a substantial minority of NRT use could be for noncessation reasons (Hammond et al., 2008; Hughes, 2008). None of the included studies measured this possibility.

But the major problem with demonstrating effectiveness in nonrandomized studies is selection bias. Six studies have shown that those who choose to use NRT or enroll in programs offering free NRT are those who would be expected to have worse outcomes, that is, those who are more dependent or have had more trouble quitting in the past (Bansal, Cummings, Hyland, & Giovino, 2004; Cokkinides, Ward, Jemel, & Thun, 2005; Cummings, Hyland, Ockene, Hymowitz, & Manley, 1997; Klesges et al., 2007; Shiffman et al., 2005, 2008a). Most of the studies in the current review attempted to correct for this bias by adjusting for such differences. Importantly, the effectiveness of OTC NRT remained at a similar level after these adjustments. However, because most of the analyses were secondary analyses of datasets not set up to test for effectiveness, none of the studies included an adequate set of possible confounders.

All four of the studies that also examined counseling found it was not associated with success; yet current guidelines and funding agencies believe phone counseling is effective in real-world settings (Fiore et al., 2000). This finding could be seen as a strong argument against the validity of retrospective cohort analyses (unless one also believes phone counseling is ineffective). Given this result and the strong selection bias discussed above, we believe that it is unlikely that further secondary analyses of trials not designed to examine the effectiveness of OTC NRT are likely to yield more definitive results. We think either a nonrandomized prospective study that collects detailed data on possible confounders or a RCT in an OTC setting is much more likely to lead to more definitive conclusions.

In terms of RCTs of OTC NRT, our 2003 meta-analysis located four placebo-controlled RCTs of NRT conducted in OTC settings (i.e., no counseling was provided) (Hughes et al., 2003). We calculated an OR of 2.5 (95% CI = 1.8–3.6; Hughes et al., 2003) favoring active NRT. In addition, that meta-analysis also located four other trials comparing OTC and Rx NRT (two of which were nonrandomized trials) and did not find that OTC NRT produced lower quit rates. Criticisms of the external validity of these trials have focused on the fact that NRT was free and monitoring of smoking status occurred and could have boosted quit rates (Walsh, 2008).

Although no new RCTs of active NRT versus placebo NRT in effectiveness settings have been published since that analysis, two studies comparing NRT in an OTC (i.e., with no counseling) setting versus in a counseling setting have occurred. One study found that quit rates with NRT in an OTC setting were inferior to those in a setting in which in-person counseling was available (Leischow, Ranger-Moore, Muramoto, & Matthews, 2004); however, the in-person counseling was by trained experts, not typical clinicians. The other study compared OTC NRT plus a smoking cessation manual versus a manual alone among smokers in a Veteran's Affairs system. This study found worse quit rates in the former (Velicer et al., 2006).

In summary, the evidence from existing nonrandomized effectiveness studies is mixed. We believe that, given the problems of secondary analyses of nonrandomized studies delineated above (i.e., questions about sensitivity, specificity, and selection bias), further secondary analyses are unlikely to lead to more definitive conclusions. Instead, we believe that either a prospective cohort study with good measures of possible confounders or further placebo-controlled RCTs in effectiveness settings are needed to provide an adequate test of “real-world” OTC NRT.

Funding

This analysis was supported by Senior Scientist Award DA-00490 (J.R.H.) and Institutional Training grant DA-07242 (E.N.P.) from the U.S. National Institute of Health.

Declaration of Interests

J.R.H. is currently employed by The University of Vermont and Fletcher Allen Health Care. Since January 1, 2008, he has received research grants from the National Institute on Health and Pfizer; the later develops and sells smoking cessation medications. During this time, he has accepted honoraria or consulting fees from several non-profit and for-profit organizations and companies that develop, sell, or promote smoking cessation products or services or educate/advocate about smoking cessation: Abbot Pharmaceuticals; Aradigm; American Academy of Addiction Psychiatry; American Psychiatric Association; American Psychiatric Institute for Research and Education; Cambridge Hospital; Dean Foundation; Dartmouth-Hitchcock; DLA Piper; EPI-Q; European Respiratory Society; Evotec; Free and Clear; Glaxo-Smith Kline; Golin Harris; Healthwise; Integrated Communication; Invivodata; Maine Health; McGill University Medical School, McNeil Pharmaceuticals; Novartis Pharmaceuticals; Oglivy Health PR; Ottawa Heart Institute; Pfizer Pharmaceuticals; Pinney Associates; Propagate Pharmaceuticals; Reckner Associates; Scientia; University of Arkansas for Medical Sciences; University of California-San Francisco; U.S. National Institutes on Health; Wolters Publishing. Neither of the other authors have competing interests.

Acknowledgments

We thank the authors of the papers who responded to our request for clarifications and comments. We also thank Saul Shiffman for his comments on the paper.

References

  • Alberg A, Margalit R, Burke A, Rasch K, Stewart N, Kline J, et al. The influence of offering free transdermal nicotine patches on quit rates in a local health department's smoking cessation program. Addictive Behaviors. 2004;29:1763–1778. doi:10.1016/j.addbeh.2004.04.007. [PubMed]
  • Amori G, Lenox RH. Do volunteer subjects bias clinical trials? Journal of Clinical Psychopharmacology. 1989;9:321–327. [PubMed]
  • Bansal MA, Cummings KM, Hyland A, Giovino GA. Stop-smoking medications: Who uses them, who misuses them, and who is misinformed about them? Nicotine & Tobacco Research. 2004;6:S303–S310. [PubMed]
  • Berg C, An L, Kirch M, Guo H, Thomas J, Patten C, et al. Failure to report attempts to quit smoking. Addictive Behaviors. 2010;35:900–904. doi:10.1016/j.addbeh.2010.06.009. [PubMed]
  • Bush T, McAfee T, Deprey M, Mahoney L, Fellows J, McClure J, et al. The impact of a free nicotine patch starter kit on quit rates in a state quit line. Nicotine & Tobacco Research. 2008;10:1516. doi:10.1080/14622200802323167. [PubMed]
  • Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research. Chicago, IL: Rand McNally College Publishing Company; 1966.
  • Cokkinides VE, Ward E, Jemel A, Thun MJ. Under-use of smoking-cessation treatments: Results from the National Health Interview Survey, 2000. American Journal of Preventive Medicine. 2005;28:119–122. doi:10.1016/j.amepre.2004.09.007. [PubMed]
  • Cummings K, Hyland A, Fix B, Bauer U, Celestino P, Carling-Menter S, et al. Free nicotine patch giveaway program: 12-month follow-up of participants. American Journal of Preventive Medicine. 2006;31:181–184. doi:10.1016/j.amepre.2006.03.027. [PubMed]
  • Cummings KM, Hyland A, Ockene JK, Hymowitz N, Manley M. Use of the nicotine skin patch by smokers in 20 communities in the United States, 1992–1993. Tobacco Control. 1997;6:S63–S70. [PMC free article] [PubMed]
  • Fiore MC, Bailey WC, Cohen SJ, Dorfman SF, Goldstein MG, Gritz ER, et al. Treating tobacco use and dependence. Clinical practice guideline. Rockville, MD: U.S. Public Health Service; 2000.
  • Gartlehner G, Hansen R, Nissman D, Lohr K, Carey T. A simple and valid tool distinguished efficacy from effectiveness studies. Journal of Clinical Epidemiology. 2006;59:1040–1048. doi:10.1016/j.jclinepi.2006.01.011. [PubMed]
  • Gilpin E, Messer K, Pierce J. Population effectiveness of pharmaceutical aids for smoking cessation: What is associated with increased success? Nicotine & Tobacco Research. 2006;8:661–669. doi:10.1080/14622200600910801. [PubMed]
  • Gilpin E, Pierce JP. Measuring smoking cessation: Problems with recall in the 1990 California Tobacco Survey. Cancer Epidemiology, Biomarkers & Prevention. 1994;3:613–617. [PubMed]
  • Gomez-Zamudio M, Renaud L, Labrie L, Masse R, Pineau G, Gagnon L. Role of pharmacological aids and social supports in smoking cessation associated with Quebec's 2000 Quit and Win campaign. Preventive Medicine. 2004;38:662–667. doi:10.1016/j.vpmed.2003.12.015. [PubMed]
  • Hammond D, Reid J, Driezen P, Cummings K, Borland R, Fong G, et al. Smokers’ use of nicotine replacement therapy for reasons other than stopping smoking: Findings from the ITC Four Country Survey. Addiction. 2008;103:1696–1703. doi:10.1111k/1360-0043.2008.02320.x. [PubMed]
  • Hawk L, Jr., Higbee C, Hyland A, Alford T, O’Connor R, Cummings K. Concurrent quit & win and nicotine replacement therapy voucher giveaway programs: Participant characteristics and predictors of smoking abstinence. Journal of Public Health Management Practice. 2006;12:52–59. [PubMed]
  • Higgins JPT, Thompson S. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 2002;21:1539–1558. doi:10.1002/sim.1186. [PubMed]
  • Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analysis. British Medical Journal. 2003;327:557–560. doi:10.1136.bmj.327.7414.557. [PMC free article] [PubMed]
  • Hughes JR. The effectiveness of over-the-counter nicotine replacement: A rebuttal. Drug and Alcohol Review. 2001;20:319–324. doi:10.1080/09595230120079639.
  • Hughes JR. Significance of off-label use of NRT. Addiction. 2008;103:1704–1705. doi:10.1111/j.1360-0443.2008.02357.x. [PMC free article] [PubMed]
  • Hughes JR. How confident should we be that smoking cessation treatments work? Addiction. 2009;104:1637–1640. doi:10.1111/j.1360-0443.2009.-2645.x. [PMC free article] [PubMed]
  • Hughes JR, Callas P. Data to assess the generalizability of samples from studies of adult smokers. Nicotine & Tobacco Research. 2010;12:73–76. doi:10.1093/ntr.ntp16. [PMC free article] [PubMed]
  • Hughes JR, Shiffman S, Callas P, Zhang J. A meta-analysis of the efficacy of over-the-counter nicotine replacement. Tobacco Control. 2003;12:21–27. doi:10.1136.5c.12.1.21. [PMC free article] [PubMed]
  • Hyland A, Rezaishiraz H, Giovino G, Bauer JE, Cummings KM. Over-the-counter availability of nicotine replacement therapy and smoking cessation. Nicotine & Tobacco Research. 2005;7:547–555. doi:10.1080/14622200500185975. [PubMed]
  • Klesges R, Klesges L, Vander Weg M, DeBon M, Poston W, Ebbert J, et al. Characteristics of Air Force personnel who choose pharmacological aids for smoking cessation following an involuntary tobacco ban and tobacco control program. Health Psychology. 2007;26:588–597. doi:10.1037/0278-6133.26.5.588. [PubMed]
  • Leischow S, Ranger-Moore J, Muramoto M, Matthews E. Effectiveness of the nicotine inhaler for smoking cessation in an OTC setting. American Journal of Health Behavior. 2004;28:291–301. [PubMed]
  • Miller N, Frieden TR, Liu SY, Matte TD, Mostashari F, Deitcher DR, et al. Effectiveness of a large-scale distribution programme of free nicotine patches: A prospective evaluation. Lancet. 2005;365:1849–1854. doi:10.1016/S0140-6736(05)66615-9. [PubMed]
  • Nash J, McCrory D, Nicholson R, Andrasik F. Efficacy and effectiveness approaches in behavioral treatment trials. Headache. 2005;45:507–512. [PubMed]
  • Pierce JP, Gilpin EA. Impact of over-the-counter sales on effectiveness of pharmaceutical aids for smoking cessation. Journal of the American Medical Association. 2002;288:1260–1264. doi:10.1001/jama.288.10.1260. [PubMed]
  • Prochaska J, Evers K, Prochaska J, Van Marter D, Johnson J. Efficacy and effectiveness trial: Examples from smoking cessation and bullying prevention. Journal of Health Psychology. 2007;12:170–178. doi:10.1177/1359105307071751. [PubMed]
  • Shiffman S, Brockwell S, Pillitteri J, Gitchell J. Individual differences in adoption of treatment for smoking cessation: Demographic and smoking history characteristics. Drug and Alcohol Dependence. 2008a;93:121–131. doi:10.1016/j.drugalcdep.2007.09.005. [PubMed]
  • Shiffman S, Brockwell SE, Pillitteri JL, Gitchell JG. Use of smoking-cessation treatments in the United States. American Journal of Preventive Medicine. 2008b;34:102–111. doi:10.1016/j.amepre.2007.09.033. [PubMed]
  • Shiffman S, Di Marino ME, Sweeney CT. Characteristics of selectors of nicotine replacement therapy. Tobacco Control. 2005;14:346–355. doi:10.1136/5c.2004.009183. [PMC free article] [PubMed]
  • Shiffman S, Sweeney CT. Ten years after the Rx-to OTC switch of nicotine replacement therapy: What have we learned about the benefits and risks of non-prescription availability? Health Policy. 2007;86:17–26. doi:10.1016/j.healthpol.2007.08.006. [PubMed]
  • Slavin RE. Best evidence synthesis: An intelligent alternative to meta-analysis. Journal of Clinical Epidemiology. 1995;48:9–18. doi:10.1016/0895-4356(94)00097-A. [PubMed]
  • Solberg LI, Boyle RG, Davidson G, Magnan S, Carlsson CL, Alesci NL. Aids to quitting tobacco use: How important are they outside controlled trials? Preventive Medicine. 2001;33:53–58. doi:10.1006/pmed.2001.0853. [PubMed]
  • Swartz S, Cowan T, Klayman J, Welton M, Leonard B. Use and effectiveness of tobacco telephone counseling and nicotine therapy in Maine. American Journal of Preventive Medicine. 2005;29:288–294. doi:10.1016/j.amepre.2005.06.015. [PubMed]
  • Thorndike AN, Biener L, Rigotti NA. Effect on smoking cessation of switching nicotine replacement therapy to over-the-counter status. American Journal of Public Health. 2002;92:437–442. [PubMed]
  • Tinkelman D, Wilson S, Willett J, Sweeney C. Offering free NRT through a tobacco quitline: Impact on utilisation and quit rates. Tobacco Control. 2007;16:i42–i46. doi:10.1136/5c.2007.019919. [PMC free article] [PubMed]
  • Velicer W, Fava J, Keller S, Ramelson H, Friedman R, Gulliver S, et al. Evaluating nicotine replacement therapy and stage-based therapies in a population-based effectiveness trial. Journal of Consulting and Clinical Psychology. 2006;74:1162–1172. doi:10.1037/0022=006X.74.6.1162. [PubMed]
  • Walsh R. Over-the-counter nicotine replacement therapy: A methodological review of the evidence supporting its effectiveness. Drug and Alcohol Review. 2008;27:529–547. doi:10.1080/09595230802245527. [PubMed]
  • Warner KE, Burns DM. Hardening and the hard-core smoker: Concepts, evidence, and implications. Nicotine & Tobacco Research. 2003;5:37–48. doi:10.1080/1462220021000060428. [PubMed]
  • West R, Zhou X. Is nicotine replacement therapy for smoking cessation effective in the “real world”? Findings from a prospective multinational cohort study. Thorax. 2007;62:998–1002. doi:10.1136/thx.2007.078758. [PMC free article] [PubMed]

Articles from Nicotine & Tobacco Research are provided here courtesy of Oxford University Press