PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Cancer Epidemiol Biomarkers Prev. Author manuscript; available in PMC 2010 December 1.
Published in final edited form as:
PMCID: PMC2790162
NIHMSID: NIHMS153957

Assessing Consumer Responses to PREPs: A Review of Tobacco Industry and Independent Research Methods

Abstract

Objective

Internal tobacco industry documents and the mainstream literature are reviewed to identify methods and measures for evaluating tobacco consumer response. The review aims to outline areas in which established methods exist, identify gaps in current methods for assessing CR, and consider how these methods might be applied to evaluate PREPs and new products.

Methods

Internal industry research reviewed included published papers, manuscript drafts, presentations, protocols, and instruments relating to consumer response measures were identified and analyzed. Peer-reviewed research was identified using PubMed and Scopus.

Results

Industry research on consumer response focuses on product development and marketing. To develop and refine new products, the tobacco industry has developed notable strategies for assessing consumers' sensory and subjective responses to product design characteristics. Independent research is often conducted to gauge the likelihood of future product adoption by measuring consumers' risk perceptions, responses to product, and product acceptability.

Conclusions

A model which conceptualizes consumer response as comprising the separate, but interacting domains of product perceptions and response to product is outlined. Industry and independent research supports the dual domain model, and provides a wide range of methods for assessment of the construct components of consumer response. Further research is needed to validate consumer response constructs, determine the relationship between consumer response and tobacco user behavior, and improve reliability of consumer response measures. Scientifically rigorous consumer response assessment methods will provide a needed empirical basis for future regulation of PREPs and new products, to counteract tobacco industry influence on consumers, and enhance the public health.

Keywords: Tobacco, Harm Reduction, Reduced Exposure Product, Consumer Perceptions, Subjective Response

Introduction

The commercial success of tobacco products depends largely on cigarette manufacturers designing and promoting products to ensure that they retain their existing customers, attract the customers of their competitors, and replenish the market with new smokers who choose those brands. An extensive body of evidence derived from analysis of internal tobacco industry documents has demonstrated the use of sophisticated and subtle design variations by the tobacco industry to promote product acceptance among tobacco consumers. The design features of tobacco products may prompt sensory and other subjective responses which are important in the determination of taste and smoking satisfaction (1,2), psychological reward (3), and craving diminution (4,5): all of which may contribute to consumer acceptance. Tobacco companies have also strived to communicate the advantages to their consumers of new design features through new packaging and branding, sophisticated marketing messages, and even the form of the tobacco product itself (6).

Advertising and other messaging strategies are used by the tobacco industry to convey product information, and so positively influence consumer perceptions. Advertising and marketing of lower tar cigarette brands over the past 50 years has emphasized “satisfaction” (taste and sensory effects) and “pleasure” (nicotine effects), while simultaneously communicating reduced cigarette strength and, by implication, lower product risk (1). The misguided belief that a low tar cigarette is less risky is undoubtedly a deterrent to smoking cessation, since health concerns are an important factor in motivating smoking cessation (7). A case in point is the Marlboro Lights brand. Marlboro Lights was one of first brands to use the “Lights” descriptor to communicate its lower tar (and, by implication, lowered harm) feature. The Marlboro Lights pack, achieves a lighter look with a white pack and a gold chevron instead of the distinctive red chevron found on Marlboro Red. Consumer research conducted by Philip Morris has shown that smokers perceived different tastes when they smoked identical cigarettes in red vs. gold packs (8). Marlboro Lights advertising also used more white space, less action, and muted colors: all intended to reinforce the lower tar delivery message of the brand. The cigarette itself also has been designed to reinforce the perception of a safer product, with the use of white tipping paper on the filter tip, in contrast to the cork colored paper used on most full-flavored cigarettes (8). The use of chemical additives and milder tobacco blends in low tar cigarettes also helps reinforce the consumer's perception that a “light” cigarette is safer, when in fact it has been designed in a way that simply enhances the smoker's ability to puff more freely on the cigarette (9,10). Despite the intensive efforts taken by the industry to implicitly communicate lowered harm, internal industry studies show that lower machine yield cigarettes have greater toxicity compared to higher yield (full-flavor) cigarettes (8).

The proliferation of novel tobacco products that purport to be safer than conventional products has created significant challenges for the public health community. So-called PREPs (Potential Reduced Exposure tobacco Products) use novel design features or ingredients that are intended to reduce exposure to tobacco-related toxicants, and thus lower disease risk for the consumer (11). The Institute of Medicine has stated the importance of understanding how consumers perceive and respond to PREPs to avoid misperceptions among consumers, such as those that resulted from the misleading marketing of so-called low tar/light cigarettes (11). There are well-founded concerns that consumers could be misled by implicit or explicit marketing messages that communicate health benefits of PREPs, since official standards for evaluating claims for tobacco products have not yet been developed. Explicit marketing messages and claims made by tobacco manufacturers for reduced exposure tobacco products include messaging and claims about reduced exposure to harmful constituents (12). Not all reduced risk claims are explicit: implicit claims can be communicated through product descriptors, including the colors and graphics used in product packaging, together with qualifier terms such as “ultra” and “lights.”

Most PREPs introduced in the last decade are no longer in the market, in part because products designed to reduce exposure tend to have low acceptability among consumers. Poor sensory and subjective characteristics of PREPs may have contributed to their low consumer acceptance, despite a substantial research and development program by major tobacco companies to address this (i.e., Marlboro UltraSmooth) (12). The modification of tobacco products to reduce harmful toxicants can affect design features that facilitate positive consumer responses. For example, activated carbon is sometimes added to cigarette filters to reduce harmful gas phase constituents, but this also reduces or alters flavor characteristics (13). Similarly, products designed to heat tobacco electrically may be too dissimilar to a cigarette to promote broad consumer interest. However, new or future PREPs may appeal to consumers and could receive broad acceptance and corresponding market share. In the new tobacco regulatory environment provided under the Family Smoking Prevention and Tobacco Control Act of 2009 (14), the Food and Drug Administration (FDA) will have the capacity to restrict or ban product design features, including flavorants and additives designed to enhance consumer acceptability, and product messaging that promotes perceptions of lowered risk. Thus, there is a great need for scientifically validated methods to allow rapid and early assessment of consumer responses to PREPs and other new tobacco products, and to provide an empirical basis for this new regulatory framework.

Consumer response (CR) to tobacco products is defined as a set of subjective and behavioral responses which convey information, affect behavior and likelihood of long term product use by the consumer and his or her future intentions for product adoption (see Figure 1). The Theory of Reasoned Action (TRA) (15,16) provides a basis for understanding the link between attitudes and behavior, and may provide a heuristic for conceptualizing of consumer responses to tobacco products. TRA posits that the likelihood of a behavioral outcome is influenced by an individual's attitude combined with subjective norms about that behavior. Consistent with this theory, CR might be seen as a complex set of attitudes and norms that influence the likelihood of product trial (the behavioral response). In the model proposed here, consumer responses are represented by two broad categories: perception of the product and response to the product. Product perceptions encompass subjective responses to product information, and include individual perceptions of risk, attitudes and beliefs about the product, social acceptability, and outcome expectancies such as satisfactory nicotine and sensory effects. Responses to product encompass acute nicotine and sensory effects (including drug reward; relief of craving and withdrawal; taste, harshness/smoothness, and other sensory characteristics), conditioned motivational responses to sensory cues, affective responses, and product personal acceptability. Also represented in Figure 1 are two general domains within which tobacco manufacturers seek to positively influence consumer response: product advertising and marketing (“messaging”); and product design. In general, tobacco industry messaging is aimed at directly influencing product perceptions, while design manipulations are used to influence product responses. A hypothetical link exists between product perceptions and response, such that each exist in a mutual feedback loop. The combined influence of product perceptions and product response determine the likelihood of initial and subsequent use. Also assumed in this model are pre-existing personal factors, such as degree of tobacco dependence, and social and personal norms, in the context of which industry messaging and product design must operate to be effective. The tobacco industry has designed and marketed products to appeal to target groups with specific characteristics such as age, gender, ethnicity/race and degree of tobacco dependence (2). This model thus allows a basis not only for predicting future use of the product, but for understanding relationships between the range of factors that contribute to positive consumer responses.

Figure 1
Hypothetical Model of Tobacco Consumer Response

This paper reviews both internal tobacco industry documents and the mainstream literature to identify methods and measures for evaluating tobacco CR. The review aims to outline areas in which established methods exist, identify gaps in current methods for assessing CR, and consider how these methods might be applied to evaluate new products, including PREPs. Although CR research has been conducted with conventional products, the present review focuses on products designed to reduce exposure, owing to the special challenges associated with assessing their potential for consumer acceptance and appropriately communicating their risks. Strategies used to evaluate PREPs have largely evolved from those used to evaluate conventional products, thus making a broader review largely redundant. For reasons of economy, the current review therefore will focus on PREPs.

Methods

Review of Internal Industry Methods

Internal tobacco industry documents were identified using databases at Tobacco Documents Online (http://www.tobaccodocuments.org), the Legacy Tobacco Documents Library (http://legacy.library.ucsf.edu), and the British American Tobacco Documents Archive (http://bat.library.ucsf.edu). A snowball sampling design was used for text-based and index searches, with an initial set of keywords and phrases (i.e., consumer perception, reduced harm, test market) which resulted in the development of hundreds of further search terms. Search terms may have been combined for more targeted searches, initially yielding large numbers of documents (see examples in Table 1). These results were visually skimmed, and duplicates and obviously irrelevant documents (i.e., deposition transcripts, popular newspaper and magazine articles, smoke chemistry data) were discarded. The remaining documents were carefully reviewed for relevant methods and measures of consumer response.

Table 1
Example Terms Used in Searches of Tobacco Industry Document Archives

Relevant documents included (a) product development efforts to address issues of consumer perception of PREPs; and (b) market research and surveillance conducted following the launch of novel tobacco products. Of the approximately 8,000,000 documents available in the archives, the final sample was narrowed to include approximately 970 relevant documents dating from 1980 to present. A total of 60 documents contained sufficiently detailed descriptions of industry methods, instruments and measures to be used as citations in this paper.

Review of Independent Scientific Literature

A previous literature review conducted by Pederson and Nelson (2007) summarized the findings of studies on consumer perceptions of PREPs (17). In contrast, this paper focused on cataloguing the methods used by independent researchers to measure CR, including responses to product, and compared them with research strategies used by the tobacco industry. The published literature within PubMed, ranging from January 1995 to August 2008 was searched using keywords such as reduced exposure, reduced harm, novel tobacco products, sensory, smoker perception, and subjective response.

Selected manuscripts examined smokers' knowledge, attitudes, beliefs and intentions about, and subjective responses to, potentially reduced-exposure tobacco products (PREPs). The criteria used by Pederson and Nelson (17) were used to define PREPs; specifically, tobacco products that resemble conventional cigarettes but do not necessarily require both tobacco and nicotine and are heated or burned, or is some other oral nicotine-containing product made by a tobacco company (i.e., electrically-heated cigarettes, snus, de-nicotinized cigarettes, charcoal filtered cigarettes).

Results

I. Methods to Assess Product Perceptions

Manufacturers of PREPs are concerned with consumer adoption of their proposed product, and the first phases of product development or marketing efforts include measuring perceptions of the product “concept” using a variety of methods. The product concept can be assessed independently of actual product use, or combined in a product-concept test, where subjects view and rate the concept, and sample and rate the product (18). Methods used by the tobacco industry for PREP concept and message testing include focus groups, public intercept surveys, and random digit dial surveys. Independent researchers have applied all of these methods for post market PREP assessment, with the inclusion of web surveys (Table 2). The following describes methods used by the tobacco industry, at pre- and post-market stages, to assess consumers' perceptions of product and product concepts; and methods used by independent researchers, primarily at the post-market stage, to assess perceptions of product.

Table 2
Strategies to Assess PREP Consumer Response Domains

i) Focus groups

Focus groups are commonly used in market research to assess consumer reactions to new ideas or products, and social scientists have increasingly applied the method to gain insight into specific research questions and frame formative issues for further study (19). Typically, focus groups consist of groups of 6-10 respondents screened to represent the target market of interest, gathered in the same room and guided by a moderator through a discussion on topics of interest (20,21).

Focus group research conducted by the industry has included both product concept naïve participants as well as participants who have previously used a PREP and can offer feedback about their experience. RJ Reynolds conducted focus groups among current users of Premier in 1990, to determine which features of the PREP they found most appealing and to catalogue reasons for continued use (22). Similar sessions were held by the company in test markets for Eclipse in 1996, where perceived benefits and drawbacks of the product concept and actual product were discussed among smokers (23).

Among independent researchers, focus groups have been used to assess motivations and use patterns of PREPs, and consumer's perceptions of PREP messaging. Carabello et al (24) asked smokers aged 30-50 questions about how they learned about PREPs, their reasons for trying in the first place, which products they had tried, their perceptions of the product at first trial, their reasons for continuing or discontinuing use, and whether they would recommend PREPs to others. Other studies have used focus groups to assess consumer's perceptions of PREP messaging by showing advertisements for products currently on the market to participants. O'Hegarty et al (30) used recruitment criteria described by Carabello et al (24) in cities where Eclipse was in test market. Following a discussion about PREP use, participants were shown a series of print advertisements and promotional materials for specific products. A discussion about the participants' reactions followed, including their impressions of the intended message of the advertisement and its influence on their interest in PREPs.

Both tobacco industry and independent researchers recruit and/or classify focus group subjects by demographic or social background, smoking history, or an individual's knowledge, attitudes, and beliefs regarding smoking. Some of these characteristics may be used as inclusion criteria, but the industry has used them also to group subjects in the analysis of data collected in consumer perception studies. Tobacco companies maintain extensive databases of current and former customers, to expedite recruitment of participants in PREP consumer perception studies. In studies of Eclipse, RJ Reynolds contacted smokers by telephone or mail to invite them to participate in focus groups and surveys; samples of the test product were sometimes sent beforehand for the consumer to try (23). In the independent literature, Hughes et al (25) utilized a similar resource in their investigation of Eclipse; the researchers contacted RJ Reynolds for access to the database of smokers who had ever purchased Eclipse, as well as those who had received Eclipse promotional materials. The company provided the names and addresses to a third party mailing service, which sent study recruitment materials to the smokers, identifying the study investigators as independent of the manufacturer.

ii) Public intercept surveys

Public intercept surveys use a convenience sampling strategy to administer a structured questionnaire to participants who are recruited at a public place (i.e., a shopping mall). Participants may also be shown examples of products or concepts to evaluate (26-30). Questionnaires are written according to the objectives of the study, are administered in a structured interview format, and may also function as a screening tool to recruit participants for follow-up surveys or tracking studies on tobacco use by the company (31-33). Public intercept surveys for tobacco products may face unique limitations that compromise the representativeness of the final sample; for example, in 1997 RJ Reynolds discontinued its use of public intercept research for product Eclipse because of a high rate of refusal among mall patrons (34). It is unclear whether the high rate of refusal was inherent to the method, or whether it was due to the fact that consumers were resistant to participating in a tobacco industry study.

Independent researchers have used intercept surveys to investigate the nature and impact of implicit and explicit health messages contained in PREP promotional materials. Hamilton and colleagues (35) asked adult smokers to review a single advertisement for each type of product (one of two print advertisements for regular cigarettes, light cigarettes, and PREPs). Health messages, which promoted the health-related benefits of using PREPs instead of regular cigarettes, were distinguished from health risk warnings (35). In addition to Surgeon General's warnings (which were contained in advertisements for all types of tobacco products), PREPs contained specific warnings which were evaluated by respondents in the public intercept interview. Respondents were asked to assess the health risk (i.e., risk of lung or heart disease caused by the product), amount of “tar”, “level of things that might cause cancer” on a scale from 1 to 10. Participants were asked whether the ads contained any set of specified messages, including: whether the message was indicating that the product safer or healthier than other cigarettes, and whether it will help someone quit smoking. Questions intended to de-emphasize the focus on health issues were also included, which elicited information on the perceived potential benefits or expectations of product response (i.e., whether the product would taste better and whether it would be a good product to smoke with friends).

Shadel and colleagues (36) conducted an intercept survey in 14 states in which the test product (Quest) was not being marketed. A sample of 200 regular smokers, most of whom (92%) had never heard of Quest, were asked about their perception of Quest after viewing a single advertisement. Eight questions assessed beliefs on a 5-point Likert scale (1=definitely untrue, 5=definitely true). One question, whether Quest had lower nicotine than regular cigarettes, was based on information that was explicitly stated in the advertisement. The remaining questions asked about health-related product attributes that were not mentioned in the advertisement but may have been inferred (i.e., Quest cigarettes are less addictive than regular cigarette; Quest cigarettes make smoking safer). Additional measures were taken on respondents' need for cognition, using a nine-item scale to determine respondents' levels of need for understanding and preference for thinking about complex issues (37,38). Three items asked about smokers' perceived vulnerability to smoking-related health harms on a 5-point scale: (a) “How much do you think you can smoke without harming your health?”; (b) “To what extent do you feel that your overall health has been affected by smoking”; and (c) How much do you think that quitting smoking could help your health?”.

An alternative approach involves door-to-door surveys: one independent study exposed smokers to one of four experimental conditions, in which they were shown advertisements for PREPs with or without a box containing health information superimposed onto the ad (39). Respondents completed a 30 minute survey about the type of information they used to evaluate advertisements for tobacco products. Materials included print advertisements for two PREPs (Eclipse and Accord) with or without product health information, as well as a second page with a bar graph demonstrating levels of three carcinogens in PREPs compared to those found in a standard “light” cigarette.

iii) Population-based surveys within test markets

Tobacco companies commonly select cities or regions as “test markets” to launch new products prior to their national introduction. These test markets are used to validate results from consumer testing and confirm industry projections of consumer demand. They provide important data on consumer adoption processes and extended use behaviors (i.e., where consumers buy products, the level of promotion and marketing necessary to achieve a desired trial rate, and the level of product experience needed to “acclimate tryers and stimulate conversion”) (40,41).

Test markets for PREPs are carefully selected according to several criteria that potentially impact outcomes (40). The geographic distribution of an area (i.e., population density, demographics) is of interest to tobacco companies, particularly as it corresponds to target groups. These markets must be nationally representative in terms of low-tar smokers, as well as smokers of discount brands (to minimize bias resulting from extremely high or low coupon use). The tobacco company's existing share of the market is evaluated in order to assess whether “cannibalization” (switching of consumers from existing company brands to the new product, rather than attracting new consumers from other brands or among non-smokers) may occur.

The industry uses random-digit dial methods to conduct surveys, often in multiple waves, through market research firms and follow standard RDD methods to obtain a sample of that is representative of the test market or the national population. Quota controls may be implemented according to gender, age, ethnicity, geography, price (i.e., generic/full-price), and flavor (menthol/non-menthol) (42-44). Mail surveys are a common method used in test markets as well as national markets; household names may be purchased from a direct mail company. Mail surveys and telephone surveys may also be conducted among eligible households that have previously participated in market research conducted by the tobacco companies, or among customers in a pre-existing company database. Recruitment for studies of PREPs in test market have included offers in local newspapers to “try Eclipse for free” by calling a toll-free number; respondents receive mail surveys or telephone surveys 2 to 5 weeks after receiving the product (45,46).

In the independent literature, population-based surveys using random-digit dialing have asked respondents whether they are aware of specific PREPs, providing them with name of the product (i.e., Advance, Omni, Accord, or Eclipse) (47). Others have asked respondents to name any specific PREP that they were aware of, but did not provide brand names (48). This is an important consideration in study design, as awareness of the concept of reduced-exposure products does not necessarily equate awareness of specific brands: in the study by O'Connor et al (48), approximately 27% of the respondents who expressed a general knowledge of such products were able to identify a specific product. In addition to questions on brand awareness, questions about demographic information, history of product use, interest in trial, and attitudes about products and government regulation of tobacco products are posed to respondents (47,49). Sample sizes for RDD surveys of PREPs by independent researchers ranged from 1174 (all current smokers) to 9736 (inclusive of current smokers, former smokers, and non-smokers) (49).

Tobacco industry surveys of consumers' attitude and product awareness are conducted by telephone, mail, or email. Measures have included: current awareness of the test product at baseline and after product launch; feelings about smoking generally; opinions about PREP benefits or claims; product expectations; and influence of media reports of test product on consumer expectations and interest (44,45,50-52). In a 1997 telephone survey within test market, RJ Reynolds evaluated whether consumers responded with higher ratings of acceptance of the test PREP, Eclipse, when they received a promotional video prior to product trial versus receiving only the product by mail (53).

iv) Web surveys

Web surveys provide a means to access large numbers of survey participants in a rapid and cost-effect manner. This method has been used by independent researchers to assess consumers' perceptions to PREP messaging from specific populations, such as college students. O'Connor et al (6) asked a sample of 424 participants to rate advertisements in terms of positive and negative expectancies about the product. Expectancies were measured on a Likert-type scale ranging from -3 (completely unlikely) to +3 (completely likely). Participants were asked to rate their product expectations when presented with a range of adjectives and phrases (see Table 3). No evidence of the use of web surveys for assessment of PREP product perceptions was found in the industry documents archive, although Freeman and Chapman (54) recently conducted content analyses of tobacco companies' interactive marketing and consumer feedback campaigns conducted on their promotional websites.

Table 3
Subject Characteristics and Outcomes Variables Used for Consumer Perception of Product, by Method of Assessment

II. Methods for Assessing Response to Product

Field Testing

i) One-stick studies

While not found in the independent literature, it is not uncommon for tobacco companies to measure consumer perceptions after a participant has sampled a product for a single, rather than continual, use. “One stick” studies are conducted among targeted consumers prior to large-scale consumer testing (55). Industry validation of the one-stick method compared to longer trials (i.e., studies where respondents smoke 1-2 packs before being asked to evaluate the product) revealed that one-stick tests are valid methods for widespread, quick screening of products for further development work. However, these tests are less valid for certain attributes (i.e., harshness, blend characteristics), where sensory characteristics become perceptible only after several cigarettes (56). They are occasionally administered within focus groups, public intercept surveys, or other concept/product testing methods, and measure smokers' opinions and reactions to the product after smoking a single sample of the product (approximate N = 300) (57).

Prior to sampling the product, smokers may be asked a series of questions regarding their perceptions of the brand based on prior awareness and/or exposure to an advertisement. Measures of perceptions specific to PREPs include: likes, dislikes, taste rating, aroma rating, tar level, ease of lighting, feel in the hand, ease of draw, extinguishing cues (i.e., “how they know when it's out”), ease of disposal, reactions to accompanying instruction booklet, and reactions to packaging (58). In order for a prototype to be recommended for further acceptance testing, it must achieve significantly higher ratings on overall taste preference and on most monadic taste ratings (59).

ii) In-home product testing

Tobacco companies have provided pre-recruited respondents with product samples to use at home for a limited period of time. Some studies provide a single pack of the test product (32), while others entail longer-term use (i.e., for three days to one week) or more complex switching study designs. In a switching study, two products (the test product and a standard control) are given to participants to compare over the course of the study period (60,61), and usually the products are “blinded” whenever possible to avoid response bias (46). Prior to the in-home phase, subjects may complete a questionnaire on ideal product ratings to assess baseline sensory preferences (62). Participants provide ratings after using the product in an environment that is more naturalistic than the smoking panel study design.

Products may be mailed or personally distributed to subjects at a research site. Advantages of mailing products for the in-home testing method, compared to lab-based sensory panels, include the ability to test a larger number of subjects, potentially over a wider geographic area. However, this method has distinct disadvantages, including a longer turnaround time before receiving results, researchers' loss of control over a subject's smoking behavior or use of the product, a substantial commitment by the subject, and the possibility that test products may be shared with non-subjects (63). In-home product tests that require subjects to return to a laboratory or other research site for follow-up visits provide the opportunity for longer product use as with mailed in-home product tests, but ensure faster turnaround time, lower likelihood of attrition due to the fact that subjects return to the research facility for payment, and the opportunity for sensory or other lab-based procedures at follow-up (63).

Final sample sizes range from 100-350 complete responses per test product, with an expected response rate of about 50% (60, 63-67). Participants rate products for overall satisfaction, purchase interest, and on key sensory attributes, in a process known as monadic testing (68). Subjects are presented with one test product at a time and asked to rate them on selected attributes, rather than smoking a number of products at once and comparing them (69). These ratings are compared to subjects' initial ideal product ratings from the beginning of the study. While respondents may be instructed to use the test product in lieu of their regular cigarette brand, follow-up questionnaires do not necessarily measure smoking behaviors such as frequency and combined use (62).

Laboratory-Based Assessments

i) Sensory evaluation panels

Sensory evaluation panels are used to measure the influence of specific product design characteristics on sensory responses. These properties may include taste and aftertaste, odor, tactile properties of product use (e.g. mouthfeel; “kick” or “bite”), tactile properties of the product itself (e.g. ease of lighting, firmness of rod), and strength (amount of nicotine content or delivery). This testing is typically conducted by tobacco companies prior to broader -scale acceptance testing (70). The tobacco industry often recruits among a company's own employees for sensory evaluation panels, although these panelists are also recruited from the community (71-75). Panels typically consist of 15-50 participants who have been trained to use descriptive terms (i.e., taste and aroma) in speficied ways when evaluating cigarettes (70). A controlled “triangle” testing strategy has been used in industry research to control for the potential for a given product to “mask” the seinsory effects of the next product sampled. Under this regimen, three products are presented (two identical and one different) and participants are asked to identify and rate the one that differed. Order of presentation is varied in five combinations (i.e., AAB, ABA, BAA, BBA, BABA).

ii) Quantitative descriptive analysis

Quantitative descriptive analysis is conducted tobacco industry laboratories also using trained sensory evaluation panelists (76). Prospective panelists are recruited based on “verbal articulation, social interaction, and most importantly, sensory acuity.” Approximately 30-40% of respondents are invited to be trained as panelists. Ten, one-hour training sessions are held over a period of two weeks, where panelists are taught to identify sensory terminology. Development of a common set of terms describing taste and aroma characteristics, and associated definitions, are agreed to by the panel (77). Panelists rate the sensory attributes of each product on a structured questionnaire known as a “ballot.” Prior to each study, a warm-up session is conducted to expose the panelist to a subset of test products in order to calibrate the sensory scales and, if necessary, add additional terms to the ballot (77). Overall acceptance of the product can be measured using a structured scale (i.e., 0 = dislike a lot, 50 = indifferent, 100 = like a lot) often called a “thermometer rating” (Figure 2a) (78), and recording perceptions of a variety of sensory attributes on a computer-administered unstructured line scale (Figure 2b) (79). For each study, 2 to 6 products are assessed, with no more than 6 products evaluated within a one-week period. Three replicate evaluations of each product are made by the panelists, with a randomized complete block design being implemented to prevent order bias (77).

Figure 2Figure 2
Figure 2a. Thermometer rating scale for overall acceptance (78)

iii) Independent sensory and subjective response assessment

Studies by independent scientists also have incorporated sensory assessment into studies of smoking behavior, although the focus of this research has tended to be on measures of nicotine effect, smoking urge and withdrawal relief and abuse liability rather than consumer acceptability. In an earlier study of de-nicotinized cigarettes (80), subjects were asked to rate sensory and reinforcing properties of the products, with anchor points ranging from 0 to 100 reflecting extreme responses to each question (i.e., “no taste” = 0, “a lot of taste” = 100). A study examining differences in subjective effects between moist snuff, medicinal nicotine lozenge, and three smokeless tobacco lozenge PREPs, measured drug effects and liking through a 17-item scale ranging from 1 to 10 (“not at all” to “extremely”) (81). Items include good drug effect, harshness of cigarette, heaviness of cigarette, desire to smoke the cigarette just smoked, intensity of flavor, and freshness of cigarette (82). The same drug effect and liking measures were used to compare a carbon filtered PREP and conventional control cigarettes (83). The carbon filtered PREP was rated as having less drug effect despite similar smoke machine nicotine yield, and was less liked than the conventional product.

Independent scientists have also assessed the influence of the sensory attributes of PREPs on use behavior and product adoption. These studies have involved switching designs, in which subjects are randomly assigned to PREP and non-PREP comparison conditions. Comparison conditions have included nicotine replacement inhalers (84,85), nicotine lozenge (81), subjects' regular cigarette brand (4,80,86), a “light” cigarette brand (80), moist snuff (81), or cessation (86). Sample sizes in these studies have ranged from 10 (4,80,81) to 50 (38,84,85). Subjects may be enrolled based on their smoking status (i.e., regular brand is “light” or “ultralight”) (4,86) and/or level of nicotine dependence (4). Smokers may be asked to try a product in a single laboratory session (82), a series of laboratory sessions (2-5) (4,81), or completely have a forced switch over the course of days (3-5) or weeks (2-6) per product (80,84,86).

Studies have also measured the capacity for PREPs to relieve urge to smoke and symptoms of nicotine withdrawal, compared with conventional cigarettes. In laboratory-based smoking experiments, a three-item craving/satisfaction measure has been used after each puff, as well as a separate 4-item measure at 15-minute increments in the 90 minutes following smoking, and a widely-used 11-item tobacco withdrawal scale (87) may be administered after 90 minutes of deprivation (4,80,81,86). (see Table 4 for further information on measures). These scales may use a visual analog scale (VAS), with items consisting of a word or phrase above a horizontal line anchored by phrases such as “not at all” and “extremely”. The Questionnaire on Smoking Urges (QSU) measures intention to smoke and anticipation of relief from withdrawal (4,86,88). Questionnaires are administered immediately before of after the product is presented to subjects (81).

Table 4
Subject Characteristics and Outcomes Variables Used for Consumer Response to Product, by Method of Assessment

Discussion

Evidence from both industry and independent research has confirmed the utility of conceiving tobacco CR as comprised of two distinct but related domains: product perceptions and response to product. Product perceptions, which encompass consumers' awareness, attitudes and beliefs to product concept, marketing and/or messaging, are typically assessed via multiple measures that are contained in a structured custom-designed survey. In contrast, responses to the product – usually determined through actual use – involve measurement of sensory responses (including taste and aftertaste, mouth feel, and bite/kick), other subjective responses (including nicotine effect, urge and withdrawal relief), and ratings of product acceptability, using sensory panels or switching study designs. The highly varied nature of the components that comprise CR has produced a broad array of methods, which are chosen according to the specific questions being addressed. Methods used by the tobacco industry fall into three major areas: concept testing, messaging studies and sensory assessment. In contrast, independent research on PREPs is conducted post-market, after the product has been refined by industry CR research, and includes assessment of product messaging and sensory and subjective responses. The findings from independent studies are used to inform the scientific community on the public health implications of PREPs and new products. Further work should guide the development of public health interventions and regulation of PREPs and new products.

The range of research methods identified provides a potential basis for developing a systematic set of PREP CR assessment strategies. Comprehensive listings of available information on CR outcome measures used in research on PREPs are provided, and are organized according to the two broad domains of product perception (Table 3), response to product (Table 4).

Research on product perceptions generally makes use of custom-designed, large scale surveys. Telephone surveys are a standard methodology for tracking trends in risk factors and behaviors over time, evaluating the effects of interventions, and examining cross-sectional and longitudinal characteristics of target populations. The future of random-digit dial surveys is compromised by the decrease in landline telephone coverage due to the popularity of cell phones, as well as technology such as voice mail or call screening that reduces the likelihood of reaching participants directly. Mainstream scientists have developed strategies for addressing these methodological challenges, including: using multi-mode designs; incorporating cell phone users into sampling frames or applying weights based on the demographic characteristics of the cell phone users; utilizing text messaging and voice mail; and sending advance letters and scheduling calls (89). Web surveys have become an increasingly popular tool among mainstream researchers, because of their capacity to rapidly recruit and screen participants, present visual information, and collate survey data. Nevertheless, adapting telephone surveys to a web-based format requires consideration about issues about recruitment of random samples, question format, and visual design (90). The independent scientific community is continually testing web-based measurement issues and their effects on survey error, but the method may present considerable benefits to researchers. In addition to being cost-effective, researchers have found that in instances when respondents are asked to report sensitive information, computer-based surveys may yield more accurate data (90). Intercept surveys, particularly in malls and other public places, have been a popular survey method among commercial marketing researchers since the 1970s. Issues of validity and reliability arise in public intercept surveys; while more traditional door-to-door survey methodologies allow randomization and stratification, public intercept surveys are subject to response bias yielding a potentially non-representative sample (91). Intercept surveys nonetheless provide a low-cost option for researchers seeking to directly expose consumers to a product and obtain immediate feedback. However, specific locations may have a tendency to “burn out” after a period of time, if too many researchers repeatedly sample from the same pool of respondents (91).

Research aimed at assessing responses to product must provide smokers with an opportunity to sample the product, restricting research methods to laboratory or home based studies. Tobacco industry and independent scientists take slightly different approaches. The tobacco industry considers its users' characteristics according to target markets and/or product preferences, while mainstream scientists consider subjects according to other measures commonly found in the published literature (i.e., socioeconomic status, addiction and abuse liability, smoking behavior). While measures of sensory response are important in determining product acceptability, other measures are required. Scales used by the industry to measure overall acceptability and sensory response are useful for informing product developers (and those anticipating the likelihood of future adoption) because they help to determine why consumers may or may not find a product appealing (i.e., low overall liking scores due to a product being too light) (92). Such scales require further refinement by independent scientists to confirm the validity of key sensory constructs and their predictive relationship with product acceptability and future intentions for trial. In this context, a measure can be validated through demonstration of concordance with widely understood or accepted examples of the construct in question (“face validity”), or by demonstrating concordance with an established measure. In addition, however, validation of measures requires attention to the purpose of the assessment and the special circumstances under which it might be applied. Such circumstances include the type of product tested, the claims made or implied by the manufacturer or the product design, and special populations to whom assessment is directed. The reliability of sensory scales also requires empirical demonstration.

Strategies to assess CR should, of course, be guided by their appropriateness for providing answers to a specific research question. Sets of questions or response items generally have been developed in response the specific demands of a research program and are generally not standardized for general use.

Further research is needed to develop a more complete understanding of CR and methods for its assessment. In particular, research to better understand the relationship between positive CR components and use behavior, especially the progression from trial use to patterns of established use, is needed. Studies which assess the effect of manipulating product information on risk perceptions and use behavior are also required. Likewise, manipulation of product design, emissions on CR and use behavior outcomes would be valuable. The proposed inter-relationship between product perceptions and response to product has not been systematically investigated by industry or independent scientists. Further investigation of the relationships between product perceptions and response is needed, and future research on CR should, where possible, address these dual domains. This paper provided a descriptive overview of CR methods used in both independent and industry research; however, a comparison of the effectiveness of methods for demonstrating specific outcomes is still needed. Consumer response assessment methods should be examined and compared for validity, particularly according to type of PREP and the range of claims that could be presented. PREPs such as Omni have referred to reduced cancer risk in messaging materials, whereas Eclipse has also referred to reduced risk for respiratory illness. Other products, such as Camel Snus, present information about the convenience of the product and pleasure associated with its use. Further research is also required on consumer perceptions of implicit product messages, and the relationship between the consumer's understanding of actual message and the influence on risk-benefit ratio. Research has shown that implicit messages are perceived similarly to explicit messages, which further underscores the need for research in this area. Looking more broadly, the applicability of existing or future methods for special populations requires investigation. Currently, PREPs have been shown to have greater appeal to women, older smokers, ex-smokers, and health conscious smokers (35,93). Further research should address PREP interest and use among ethnic and racial minorities and youth. Finally, the changing nature of PREP and new product design, marketing strategies and consumer needs means that assessment methods must be able to respond to the “moving target” by adapting appropriately, but while maintaining empirical validity.

Strategies for comprehensive assessment of CR have previously been proposed by Stratton (2001) and Hatsukami et al (2005) (5). The present review provides a basis for establishing methods and measures for assessing CR to PREPs using a conceptual model. This model allows testing of predicted CR outcomes that may arise through future modification of product design or messaging. The tobacco industry has focused on CR assessment to ensure that its products meet with market success. Industry strategy now appears intent not only on successfully modifying product design and messaging, but tailoring those modifications to specific sub-groups of smokers, such as youth, women and racial/ethnic minorities (2). The proposed model of CR may provide a basis for anticipating CR among sub groups by accounting for social norms and personal factors such as tobacco use history and personal acceptability. The present model also provides a systematic strategy for those charged with regulation of tobacco product design and marketing, such as the Food and Drug Administration.

Conclusion

To ensure the commercial success of new PREPs, we can be sure that tobacco manufacturers will attempt to make those products acceptable alternatives for current smokers and may also enhance their appeal for new and ex-tobacco users. Tobacco products that purport to reduce risk, whether or not they succeed in reducing human exposure to harmful toxicants, may perpetuate tobacco-related harms if their acceptance and use at a population level is enhanced. Consumer acceptance is conceived as the related components of product perceptions and sensory and other responses to product. In response, there has been a substantial research effort by the public health community to understand consumer perceptions and responses to product to PREP messaging and product design. As methods and measures for PREP assessment are adopted and standardized, their policy and regulatory function must be heeded. The present model provides an empirically-based strategy for understanding and predicting consumer response, trial, and likelihood of future adoption of new PREPs. With further development of research methods, the model may provide a basis for understanding and predicting consumer responses of targeted smoker subgroups. This information is essential for effective regulation of new tobacco products by authorities such as FDA and states' attorneys general.

Acknowledgments

These activities were supported by National Cancer Institute contract N01-PC-64402 - Laboratory Assessment of Tobacco Use Behavior and Exposure to Toxins Among Users of New Tobacco Products, and NCI grant #1RO1-CA-125224.

References

1. Pollay R, Dewhirst T. The dark side of marketing seemingly “Light” cigarettes: successful images and failed fact. Tobacco Control. 2002;11(Suppl 1):i18–i31. [PMC free article] [PubMed]
2. Carpenter C, Wayne G, Connolly G. The role of sensory perception in the development and targeting of tobacco products. Addiction. 2007;102(1):136–147. [PubMed]
3. Wayne G, Connolly G, Henningfield J. Assessing internal tobacco industry knowledge of the neurobiology of tobacco dependence. Nicotine and Tobacco Research. 2004;6(6):927–940. [PubMed]
4. Buchhalter A, Eissenbert T. Preliminary evaluation of a novel smoking system: effects on subjective and physiological measures and on smoking behavior. Nicotine and Tobacco Research. 2000;2:39–43. [PubMed]
5. Hatsukami D, Giovino A, Eissenberg T, Clark P, L D, L S. Methods to assess potential reduced exposure products. Nicotine and Tobacco Research. 2005;7(6):827–844. [PubMed]
6. O'Connor R, Ashare R, Fix B, Hawk L, Cummings K, Schmidt W. College students' expectancies for light cigarettes and potential reduced exposure products. American Journal of Health Behavior. 2007;31(4):402–410. [PubMed]
7. Riedel B, Robinson L, Klesges R, McLain-Allen B. What motivates adolescent smokers to make a quit attempt? Drug and Alcohol Dependence. 2002;68(2):167–174. [PubMed]
8. Cummings K, Hyland A, Bansal M, Giovino G. What do Marlboro Lights smokers know about low-tar cigarettes? Nicotine and Tobacco Research. 2004;(6) 3:323–332. [PubMed]
9. Kozlowski L, O'Connor R. Cigarette filter ventilation is a defective design because of misleading taste, bigger puffs, and blocked vents. Tobacco Control. 2002;11(Suppl 1):I40–50. [PMC free article] [PubMed]
10. Shiffman S, Pillitteri J, Burton S, Rohay J, G J. Effect of health messages about “Light” and “Ultra Light” cigarettes on beliefs and quitting intent. Tobacco Control. 2001;10(Suppl 1):i24–32. [PMC free article] [PubMed]
11. Institute of Medicine. Clearing the smoke: Assessing the science base for tobacco harm reduction. Washington, DC: National Academy Press; 2001. [PubMed]
12. Rees V, Connolly G. Potentially reduced exposure tobacco products. Boston, MA: Harvard School of Public Health; 2008.
13. Rees V, Wayne G, Thomas B, Connolly G. Physical design analysis and mainstream smoke constituent yields of the new potential reduced exposure product, Marlboro UltraSmooth. Nicotine and Tobacco Research. 2007;9(11):1197–1206. [PubMed]
14. Rep Waxman HA. Family Smoking Prevention and Tobacco Control Act. H.R. 1256. 2009 June 22;
15. Fishbein M, Ajzen I. Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley; 1975.
16. Ajzen I, Fishbein M. Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall; 1980.
17. Pederson L, Nelson D. Literature review and summary of perceptions, attitudes, beliefs and marketing of potentially reduced exposure products: Communication implications. Nicotine and Tobacco Research. 2007;9(5):525–534. [PubMed]
18. Hawkins SC. RJ Reynolds. Perspective on concept-product and product-concept tests. 1988. Bates No. 507261406/1410.
19. Kreuger R, Casey M. Focus Groups: A practical guide for applied research. 3rd. Thousand Oaks, CA: Sage Publications, Inc; 2000.
20. Caples J. Tested advertising methods. 5th. Paramus, NJ: Prentice Hall; 1997.
21. Centers for Disease Control and Prevention. Designing and implementing an effective tobacco counter-marketing campaign. US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; Atlanta, GA: 2003.
22. RJ Reynolds. Survey proposal. 1990. Bates #510484034.
23. Brown & Williamson. Analytic Insight, Inc - Eclipse focus group. Chattanooga: 1996. Bates #575391163/575391164.
24. Carabello R, Pederson L, Gupta N. New tobacco products: do smokers like them? Tobacco Control. 2006;15:39–44. [PMC free article] [PubMed]
25. Hughes J, Keely J, Callas P. Ever users versus never users of a “less risky” cigarette. Psychology of Addictive Behaviors. 2005;19(4):439–442. [PubMed]
26. Pratkanis A. Report on research investigating the perception of health benefit claims in Eclipse marketing communications. University of California; Santa Cruz: 2006. Bates #552734992/552735140.
27. Hunter C, Weber J, Tinsley C. Marketing research proposal. RJ Reynolds. Fourth quarter 1994 (940000) product claims development and screening research - prisms I. 1994 BIAD #94-2353-19. Bates #510353297/510353300.
28. RJ Reynolds. Project EW communications test. Dennis & Co.; 1994. BIAD #94-11125. DAC #2204. Bates #513856979/513857147.
29. RJ Reynolds. EW ad screening mall intercept screening questionnaire. 1994. Bates #514080617/514080648.
30. Hunter C, Weber J, Willard R. Marketing research proposal. RJ Reynolds. Product benefit/claim development and screening research. 1993 BIAD #93- Bates #514283566A/514283580.
31. Brace I. Questionnaire design: How to plan, structure and write survey material for effective market research. Philadelphia, PA: Kogan Page US; 2004.
32. RJ Reynolds. Product positioning and development - status. 1986. Bates #505497712/505497721.
33. Tobacco Institute. TI-Mall intercept master questionnaire. Undated. Bates #TIMN0030936-0030941.
34. RJ Reynolds. Market research proposal. Atlanta consumer tracking - new methods. 1997 97-6303-18. Bates #520584398/520584399.
35. Hamilton W, diStefano Norton G, Ouellette T, Rhodes W, Kling R, Connolly G. Smokers' responses to advertisements for regular and light cigarettes and potential reduced-exposure tobacco products. Nicotine and Tobacco Research. 2004;6(Suppl 3):S353–S362. [PubMed]
36. Shadel W, Lerman C, Capella J, Strasser A, Pinto A, Hornik R. Evaluating smokers' reactions to advertising for new lower nicotine Quest cigarettes. Psychology of Addictive Behaviors. 2006;20(1):80–84. [PubMed]
37. Cacioppo J, Petty R, Quintanar L. Individual differences in relative hemispheric alpha abundance and cognitive responses to persuasive communications. J Pers Soc Psychol. 1982;43(3):623–636. [PubMed]
38. Strasser A, Lerman C, Capella J. Lower nicotine cigarettes may not lower harm. LDI Issue Brief. 2006;12(2) [PubMed]
39. Biener L, Bogen K, Connolly G. Impact of corrective health information on consumers' perceptions of “reduced exposure” tobacco products. Tobacco Control. 2007;16:306–311. [PMC free article] [PubMed]
40. Martin C. RJ Reynolds. Premier test market plan - II. 1989. Bates #514099137/514099139.
41. Shouse D. RJ Reynolds. Lead markets. 1987. Bates #506403062/506403070.
42. MARC; RJ Reynolds. Database and tracker analysis. 1995. Bates #514655933/514655957.
43. Still M, Etzel E, Whitlach W, et al. Marketing research proposal. RJ Reynolds. TF test market research - awareness, usage, and perceptions study. 1989 MRD #89-43502. Bates #507339627/507339629.
44. RJ Reynolds. Lincoln consumer tracking - wave I. 1997. Bates #519958365/519958372.
45. RJ Reynolds. Indianapolis ad study for Eclipse. 1997. Bates #519958357/519958364.
46. Benson S. MPID consumer research group overview. Lorillard; 1998. Bates #86289074/86289086.
47. Hund L, Farrelly M, Allen J, et al. Findings and implications from a national study on potential reduced exposure products (PREPs) Nicotine and Tobacco Research. 2006;8(6):791–797. [PubMed]
48. O'Connor R, Hyland A, Giovino A, Fong G, Cummings K. Smoker awareness of and beliefs about supposedly less-harmful tobacco products. American Journal of Preventive Medicine. 2005;29(2):85–90. [PubMed]
49. Parascadola M, Hurd A, Augustson E. Consumer awareness and attitudes related to new potential reduced-exposure tobacco products. American Journal of Health Behavior. 2008;32(4):431–437. [PubMed]
50. SCoR program: carbon filter technology. Philip Morris; 2005. Bates #3006670071/3006670162.
51. RJ Reynolds. MDD perspective on Project SPA test marketing. 1986. Bates #506036242/506036266.
52. RJ Reynolds. Project SPA research program. 1988. Bates #506432640/506432649.
53. Bellis J. Market research proposal (96-6303-36) RJ Reynolds. Direct marketing product test. 1997 Bates #520192383/520192386.
54. Freeman BC S. Open Source Marketing: Camel cigarette brand marketing in the Web 2.0 world. Tobacco Control. 2009 February 11; 2009 [Epub ahead of print.] [PubMed]
55. RJ Reynolds. Reduced CO product test. 1994. Bates #514304798/514304809.
56. RJ Reynolds. (890000) consumer research summaries annual report. 1989. 1989.Bates #:516411847/516412071.
57. Gurry N. Nicotine-extracted ad/one stick study. Philip Morris; Feb 8, 1989. Bates #2021503949/2021503951.
58. Areas of questioning for Premier tracking. Philip Morris; 1988. Bates #2040929857/2040929859.
59. Brown A. Consumer and sensory product research. RJ Reynolds. Non-menthol Eclipse alternate substrates: Acceptance panel evaluation. Consumer acceptance evaluation. Final report. 1997 Bates #522144121/522144133.
60. Phillips A, Dryden M, Leary E, Potter D, Weber J, Swaim M. RJ Reynolds. MM: Winston select product test. 1995. Proposal. Bates #513950744/513950746.
61. Day D. New flavor prototypes in-home product testing. Lorillard; 1999. Bates #82341351/82341353.
62. American Tobacco. American Tobacco, new Lucky Strike filters in-home product use test. 1981 Bates #990891741/990891744.
63. Gravely L, Wilhelm J. “Mail Panel” and “Take Home Testing” discussion at RD&E - 870403. Brown & Williamson; 1987. Bates #674116794/674116809.
64. RJ Reynolds. Proposal approval needed. EW full flavor consumer acceptance product test. 1994 Bates #512347867/512347869.
65. Dryden M, Hunter C, Potter D, Swaim M, Gordin H. Prism I CC concept product test - II. Product research proposal BAID#93-2352-16. RJ Reynolds. Prism-I-Project CC -- technology screenign concept/product test. 1993 Bates #508470018/508470020.
66. Scism A. Product research report. RJ Reynolds. Project XGT FF LT NM 85 prototype test final report. 1989 Bates #514343880/514343895.
67. Burke Marketing Services Inc. In-home cigarette testing - different products - for B&W, BMR #48-513-516. Brown & Williamson; 1983. Bates #620310135/620310263.
68. Gannon T, Smith A. Old sensory 101 presentation early 900000's. Philip Morris; 1993. Bates #2075663262/2075663316.
69. American Society for Testing and Materials. Standard guide for sensory claim substantiation. Philip Morris; 1998. Bates #2082155878/2082155898.
70. RJ Reynolds. Proposed Eclipse testing. 1998. Bates #530772466/530772467.
71. Smith K, Pence D. RJ Reynolds. SED panel. 1997. Bates #530977258.
72. RJ Reynolds. Business units & competitive claims. Brands R&D; 1995. Bates #518064256/518064273.
73. DiMarco G. RJ Reynolds. NPT sensory panel participation. 1987. Bates #505940104.
74. DiMarco G. RJ Reynolds. NPT sensory panel participation. 1987. Bates #505973439/505973440.
75. DiMarco G. RJ Reynolds. NPT panelist support. 1987. Bates #505944377.
76. Norman A. RJ Reynolds. HRRC approval for consumer and SED panel evaluations of XA prototypes. 1990. Bates #510744270/510744273.
77. RJ Reynolds. Sensory evaluation methodologies. 2004. Bates # 551074115/551074116.
78. RJ Reynolds. Place ““thermometer”” scale in front of respondent. 1988. Bates #506904751/506904752.
79. Ryan G. RJ Reynolds. QDA and extended use evaluation of XA fflt menthol 100 prototypes. 1993. SED92 1780. Bates #510697256/510697281.
80. Gross J, Lee J, Stitzer M. Nicotine-containing versus de-nicotinized cigarettes: Effects on craving and withdrawal. Pharmacology Biochemistry and Behavior. 1997;57(12):159–165. [PubMed]
81. Kotlyar M, Mendoza-Baumgart M, Li Z, et al. Nicotine pharmacokinetics and subjective effects of three potential reduced exposure products, moist snuff and nicotine lozenge. Tobacco Control. 2007;16:138–142. [PMC free article] [PubMed]
82. Schuh L, Stamat H, Downey K, Schuh K. Subjective and discriminative stimulus effects of two de-nicotinized cigaretes with different tar yields. Nicotine and Tobacco Research. 2001;3(1):77–83. [PubMed]
83. Rees V, Wayne G, Connolly G. Puffing style and human exposure minimally altered by switching to a carbon-flavored cigarette. Cancer Epidemiology Biomarkers and Prevention. 2008;17(11):2995–3003. [PubMed]
84. Fagerstrom K, Hughes J, Rasmussen T, Callas P. Randomised trial investigating the effect of a novel nicotine delivery device (Eclipse) and a nicotine oral inhaler on smoking behaviour, nicotine and carbon monoxide exposure, and motivation to quit. Tobacco Control. 2000;9:327–333. [PMC free article] [PubMed]
85. Fagerstrom K, Hughes J, Callas P. Long-term effects of the Eclipse cigarette substitute and the nicotine inhaler in smokers not interested in quitting. Nicotine and Tobacco Research. 2002;4(Suppl 2):S141–145. [PubMed]
86. Breland A, Acosta M, Eissenberg T. Tobacco specific nitrosamines and potential reduced exposure products for smokers: a preliminary evaluatoin of AdvanceTM. Tobacco Control. 2003;12:317–321. [PMC free article] [PubMed]
87. Hughes J, Hatsukami D. Signs and symptoms of tobacco withdrawal. Archives of General Psychiatry. 1986;43(3):289–294. [PubMed]
88. Tiffany S, Drobes D. The development and initial validation of a questionnaire on smoking urges. British Journal of Addiction. 1991;86(11):1467–1476. [PubMed]
89. AM K, PL R. New challenges for telephone survey research in the twenty-first century. Annual Review of Public Health. 2007;28:113–126. [PubMed]
90. Dillman DA, Smyth JD. Design effects in the transition to web-based surveys. American Journal of Preventive Medicine. 2007;32(5S):S90–S96. [PubMed]
91. Gates R, Solomon PJ. Research using mall intercept: state of the art. Journal of Advertising Research. 1982;22(4):43–49.
92. Reitter RN. Product testing in segmented markets. Journal of Marketing Research. 1969;6:179–184.
93. O'Connor R, McNeill A, Borland R, et al. Smokers' beliefs about the relative safety of other tobacco products: Findigns from the ITC Collaboration. Nicotine and Tobacco Research. 2007;9(10):1033–1042. [PubMed]
94. O'Hegarty M, Richter P, Pederson L. What do adult smokers think about ads and promotional materials for PREPs? American Journal of Health Behavior. 2007;31(5):526–534. [PubMed]
95. Premier focus groups. Philip Morris; 1988. Bates #2049258301/2049258312.
96. RJ Reynolds. Second quarter, product claims, development ND screening research prisms I. 1993. Bates #511214460/511214516.
97. British American Tobacco Company. Project Greendot - review of progress. Brown & Williamson; Sept 2, 1988. Bates #620215601/620215611.
98. British American Tobacco Company. Project Greendot - Attitudinal Monitor. Brown & Williamson; Bates #465663530/465663583.
99. Naasner W. RJ Reynolds. Concept/product test. 1992. Bates #515472766/515472801.
100. Bellis J. RJ Reynolds. Panel topline on hiq vs a product acceptance test and carton results on hiq vs 12mm. 1996. Bates #516759481/516759486.