|Home | About | Journals | Submit | Contact Us | Français|
Measures of neighborhood food environments have been linked to diet and obesity. However, the appropriate measurement methods and how people actually perceive their food environments are still unclear. In a cross-sectional study of 939 adults, the perceived presence of food outlets was compared to the geographic-based presence of outlets within a participant’s neighborhood, utilizing percent agreement and Kappa statistics. Perceived presence was based on survey-administered questions, and geographic-based presence was characterized using 1-, 2-, 3- and 5-mile (1-mile=1.6 km) Euclidean- and network-based buffers centered on each participant’s residence. Analyses were also stratified by urban and non-urban designations. Overall, an individual’s perceived neighborhood food environment was moderately correlated with the geographic-based presence of outlets. The performance of an individual’s perception was most optimal using a 2- or 3-mile geographic-based neighborhood boundary and/or when the participant lived in a non-urban neighborhood. This study has implications for how researchers measure the food environment.
It has been suggested that neighborhood food environment, measured either objectively or subjectively, is associated with dietary intake (Caspi et al. 2012a). To date, geographic information systems (GIS) have been the most-utilized objective method to characterize neighborhood food environments (McKinnon et al., 2009; Charreire et al., 2010; Van et al., 2011; Thornton et al., 2011). However, it is unknown whether GIS-based measures are the most appropriate means of defining an individual’s food environment (Caspi et al., 2012b, 2012a; Mujahid et al., 2007). There are currently no standardized methods for characterizing a food environment, and thus assessment and development of appropriate food environment measures is warranted (Caspi et al. 2012b; Lytle, 2009).
Perception measures based on surveys and self-report have been used increasingly to characterize food environments (Moore et al., 2008a, 2008b; Sharkey, 2009; McKinnon et al., 2009). Such measures have included an individual’s perception of the availability of healthy food items in his/her neighborhood (Moore et al., 2008a, 2008b; Freedman and Bell, 2009; Zenk et al., 2009; Gustafson et al., 2011; Williams et al., 2011; Moore et al., 2012), as well as information on the individual’s perceived presence of different retail food outlets (Zenk et al., 2009; Gustafson et al., 2011; Williams et al., 2011; Caspi et al., 2012b). In a recent review by Caspi and colleagues, studies using measures of perceived food environments have shown significant associations with dietary outcomes (Caspi et al. 2012a; Sharkey et al. 2010; Moore et al. 2009, 2008b; Inglis et al. 2008), whereas studies utilizing GIS-based measures of retail outlet presence or density have shown mixed and varying relationships (Caspi et al. 2012b).
Methodological decisions regarding how to define food environments, including geographic boundaries and contexts, could have a significant role in deciphering inconsistent findings among studies. Using GIS, food environments have typically been characterized by geographic “neighborhood” boundaries defined as census tracts, block groups and/or Euclidean or network buffers centered on some point of reference (e.g., population-weighted centroids or home addresses) (Charreire et al., 2010; Van Meter et al., 2011; Thornton et al., 2011). However, the use of such boundaries has notable limitations, including the modifiable areal unit problem, which can bias research findings based on the choice (i.e., number, size and shape) of boundaries for areal units (Openshaw, 1983; Christian, 2012). In addition, one cannot assume that all individuals conceptualize and/or interact within their environments similarly. For instance, in neighborhood perception studies utilizing mental maps, researchers found that an individual’s perceived neighborhood can cover many different spaces and produce different boundaries based on age, race, class, gender and various other factors (Coulton et al., 2001, 2012). In a recent activity-based food environment study, researchers found that individuals encountered very different food environments in their daily travels compared to those located within or near their residential-defined neighborhood (Christian, 2012). Thus, how a person may operationalize and perceive his/her neighborhood food environment could vary based on that person’s daily routine and sociodemographic factors.
Despite challenges in defining and characterizing a food environment, only a handful of studies have examined differences in perceived and objective environments (Caspi et. al., 2012a; Moore et al., 2008a, 2008b, 2012; Zenk et al., 2009; Gustafson et al., 2011; Williams et al., 2011; Giskes et al., 2007; Freedman and Bell, 2009). Of these studies, only two directly compared perceived and GIS-based presence (availability) of retail food outlets (Caspi et al., 2012a; Williams et al., 2011); the primary focus of these studies was the identification of traditional food outlets (i.e., supermarkets) within a pre-specified neighborhood boundary.
In this study, we sought to provide an in-depth comparison of GIS-based and perceived presence of retail food outlets in a sample of adults living in an eight-county region of South Carolina. In doing so, we aimed to determine what retail food outlets are available in an individual’s neighborhood, as defined by field-validated GIS (as the gold standard), and to what extent individuals are aware of the presence of these food outlets via survey. Aims of our analyses included: 1) to examine whether the objective presence of retail food outlets within a standard 1-mile (~1.6 km) buffer used to define an individual’s GIS-based neighborhood is accurately reflected in the perceived presence of retail food outlets within a 1-mile or a 20-minute walk from an individual’s home; 2) to conduct sensitivity analyses by varying defined GIS-based neighborhoods, utilizing 2-, 3- and 5-mile buffers to examine changes in agreement (i.e., percent agreement and Kappa statistics) while keeping the perception buffer the same (1 mile or a 20-minute walk); and 3) to examine the accuracy between perceived and GIS-based presence by a sociodemographic factor, specifically urban or non-urban neighborhood designation.
Findings from this study could contribute to exploring whether measures of perceived availability of food retail outlets are viable alternative measures to GIS-based measures in food environment studies. In addition, this work could contribute to refining methods that researchers and policymakers use to describe a person’s perception of his/her food environment and whether an individual’s perceptions are adequate to detect changes in the retail food environment resulting from food access interventions, policy initiatives and associations with diet and weight outcomes.
This was a cross-sectional, non-experimental research study utilizing survey responses from 939 primary household food shoppers conducted in the spring of 2010, along with corresponding GIS-based measures of the respondents’ food environments, within an eight-county region in South Carolina. This was a supplemental study related to a larger research effort focused on developing measures of the built nutritional environment (Liese et al., 2010, 2013a) and examining perceptions, shopping behaviors and diet among residents of the eight-county study region (Ma et al., 2013; Liese et al., 2013b). This study was approved by the University of South Carolina (USC) Institutional Review Board.
The study area consisted of a contiguous geographical region encompassing eight counties (seven non-urban and one urban) in South Carolina (Fig. 1). The one urban county, Richland, contains the state capital, Columbia. The seven non-urban counties (Calhoun, Chester, Clarendon, Fairfield, Kershaw, Lancaster and Orangeburg) comprised the remaining study area.
Data collection occurred between April and July of 2010. Recruitment of study participants was geographically based and was designed to achieve spatial coverage of the entire study area. Specifically, selection was done through a random selection of landline telephone numbers with listed addresses restricted to 64 eligible ZIP codes within the study area, with a goal of 15 respondents per ZIP code. Recruitment calls were made by the interviewing staff at USC’s Survey Research Laboratory. Respondents were screened for the eligibility criteria, including being 1) at least 18 years of age, 2) the primary food shopper of the household, 3) English speaking and 4) living in the eight-county study area. Of the 2,477 household telephone numbers screened, a total of 968 residents were eligible and completed the interview; there were 553 refusals, 377 ineligibles and 579 residents of non-contact, unknown or other status. Applying response rate formula 4 as outlined by the American Association for Public Opinion Research (American Association for Public Opinion Research (AAPOR), 2011), we estimated a response rate of 47%. The AAPOR is the leading professional organization of public opinion and survey research, providing standardized definitions and methodologies for the comparison of response rates for different surveys in the United States (AAPOR, 2011). Our survey response rate was very comparable to the 49% rate among landline households that was achieved in a recent evaluation of the Behavioral Risk Factor Surveillance System (BRFSS) landline response rates conducted in 18 US states (Hu et al., 2011). In our analyses, 11 participants were removed because of missing perception data, and 18 were removed because of errors in geocoding addresses. The final analytic sample included 939 participants.
Perceived presence of a food retail outlet was obtained using a set of newly developed, validated questions (Table 1) (Ma et al., 2013). Specifically, participants were asked “which of the following stores, if any, are located in your neighborhood, defined as a 1-mile buffer or 20-minute walk around your home?” Stores listed included six types of retail food outlets, including supermarkets, supercenters, convenience stores, drug and pharmacy stores, dollar and variety stores and fast food restaurants. Response options were dichotomous: perceived presence was reported as yes or no. The test-retest reliability of the questions determined by Phi coefficients ranged from 0.58 for convenience stores to 0.96 for supermarkets (Ma et al., 2013). A person’s neighborhood was defined as a 1-mile buffer or 20-minute walk around their home based on validated perception questions published previously (Moore et al., 2008a, 2008b; Mujahid et al., 2007). In analyses, supermarkets and supercenters (defined as any large retail store that sells both groceries and general merchandise, e.g., Wal-Mart or Target) were aggregated on the basis that supermarkets and supercenters typically represent food outlets providing access to nutritious food in greater variety and affordability and higher quality (Franco et al., 2008; Block and Kouba, 2006). This classification was used previously by the US Centers for Disease Control and Prevention in their 2013 State Indicator Report on Fruits and Vegetables (Centers for Disease Control and Prevention, 2013).
The GIS-based presence of retail food outlets was determined using each respondent’s home address as the point of reference, with varying (1-, 2-, 3- and 5-mile) Euclidean and street network buffers representing neighborhood boundaries (Fig. 2). The buffers were selected to provide a reasonable range of distances that could accommodate both sensitivity analyses and any urban vs. non-urban differences in the presence of retail food outlets. Overall, in our sample, the average distance to the nearest retail food outlet ranged from 3 to 8 miles. For urban participants, this average distance ranged from 1 to 2 miles, and for non-urban participants, it ranged from 3 to 9 miles. Thus, a 1-mile network buffer was selected as the standard comparison to perception measures, and incremental increases up to 5 miles were examined. For each outlet type, presence was determined by linking geospatial retail food environment data that were validated in 2009. Methods for creating the GIS-based retail food environment dataset have been published elsewhere (Liese et al., 2010, 2013a). Dichotomous variables representing presence (yes or no) for all food outlet types were created.
Residential addresses were geocoded using address-match geocoding in ArcGIS Network Analyst 10.0 (ESRI Redlands, CA 2010). Street network data were provided by StreetMap Premium within ArcGIS. Of the 968 addresses provided, 950 (98.1%) were matched to a corresponding street address on the street network in ArcGIS and assigned a location. In addition, a 15-meter side offset was selected to place addresses on the correct side of the road based on the methods of Cayo and Talbot (Cayo and Talbot, 2003). Eighteen (1.9%) participant addresses were unmatched. Of these, 5 were PO boxes and 13 had street names that did not exist in the street network data provided. The participants who provided these addresses were excluded from analyses.
Demographic characteristics were determined based on questions from BRFSS (Centers for Disease Control and Prevention BRFSS, 2011) and included age, sex, race/ethnicity, education, employment status, utilization of the Supplemental Nutrition Assistance Program (SNAP), marital/partner status and number of individuals living in the home. Each survey respondent was also classified individually with respect to urbanicity—urban or non-urban—using the 2010 US Census–defined urban classification via a point-in-polygon operation within ArcGIS (United States Census Bureau, 2010). Specifically, a resident’s home address was geocoded, and the US Census tract in which their home is located was determined. An urban area was determined as any census tract classified as either “an urbanized area of 50,000 or more people or an urban cluster consisting of at least 2,500 and less than 50,000 people.” All census tracts not designated as urban were considered to be non-urban.
All statistical analyses were conducted using SAS 9.3 (SAS Institute, Inc., Cary, NC, 2012). First, the percent agreement was calculated to represent a basic measure of the proportion of respondents that accurately perceived the presence or absence of a food outlet type in their corresponding GIS-based neighborhood food environment when there was an actual food outlet presence or absence, respectively. Kappa statistics were then calculated to examine whether the objective GIS-based presence of retail food outlets was reflected by an individual’s perceptions, after accounting for the effects of chance agreement alone (Cohen, 1960; Viera and Garrett, 2005).
For all statistics, GIS-based presence was considered the gold standard. 95% confidence intervals (CIs) were calculated for all statistics by approximating the binomial distribution with a normal distribution. For ease of interpretation of the Kappa statistic values, the following guidelines as outlined by Landis and Koch and other studies were used: 0.81–1.00, almost perfect; 0.61–0.80, substantial; 0.41–0.60, moderate; 0.21–0.40, fair; 0–0.20, slight; and <0, poor (Landis and Koch, 1977; Viera and Garrett, 2005).
When conducting sensitivity analyses, agreement statistics were constructed, varying by GIS-based neighborhood buffer sizes (i.e., 1-, 2-, 3-, and 5-mile buffers), while the measure of perceived presence remained at 1 mile (or a 20-minute walk). Differences between statistics by buffer sizes were then assessed using non-overlapping CIs. Thus, if the CIs for two statistics did not overlap, the values were considered significantly different. Finally, the agreement between perceived and GIS-based presence was compared for individuals living in urban and non-urban neighborhoods. Fisher’s exact tests were performed to determine differences between groups.
Descriptive statistics of the study sample are displayed in Table 2. Of the 939 participants, the majority were female (79%), non-Hispanic white (67%) and lived in non-urban neighborhoods (80%). The mean age for all individuals was ~58 years. Twelve percent of participants did not have a high school diploma, 22% were unemployed, 8% received SNAP benefits and 65% had a spouse or partner living in the household. On average, individuals lived with 2.5 household members. Ninety-four percent of all participants used their own personal vehicle for primary food shopping.
Thirty-two percent of individuals reported having a supermarket in their neighborhood, defined as a 1-mile buffer around their home, whereas only 19% had a GIS-verified supermarket within a 1-mile Euclidean-based buffer, and 12% had one within a 1-mile network-based buffer (Table 3). Similar discrepancies were observed for convenience stores (54% vs. 36% and 29%, respectively), drug and pharmacy stores (29% vs. 16% and 13%, respectively), dollar and variety stores (39% vs. 22% and 15%, respectively) and franchised fast food restaurants (27% vs. 22% and 15%, respectively). For all food outlet types, the Euclidean-based buffer was closer in percentage value to the perceived data as compared to the network-based buffer.
Participants who had a specific retail food outlet located within 1 mile of their home were aware of its presence or absence, as indicated by percent agreements ranging from 72% for convenience stores to 86% for fast food restaurants using a Euclidean-based 1-mile buffer (Table 4). Percent agreements ranged from 68% to 83% using a network-based 1-mile buffer. Increasing the buffer size did not result in any significant changes in percent agreement using either Euclidean- or network-based buffers. Thus, there seemed to be no advantage in comparing the perceived survey results to a larger GIS-based neighborhood buffer size. Kappa values ranged from 0.44 (moderate agreement) for convenience stores to 0.61 (substantial agreement) for fast food restaurants using a 1-mile Euclidean-based buffer. Kappa values were lower when using a 1-mile network-based buffer, ranging from 0.35 (fair agreement) for dollar and variety stores to 0.50 (moderate agreement) for fast food restaurants. In addition, the Kappa statistic for the presence of supermarkets was significantly lower when using a 1-mile network-based buffer (0.38, fair agreement) compared to a 1-mile Euclidean-based buffer (0.52, moderate agreement).
When using different GIS-based neighborhood buffer sizes (i.e., 2-, 3- and 5-mile) as objective measures of presence, there were some statistically significant differences between Kappa values compared to the standard 1-mile Euclidean- or network-based neighborhood buffer sizes (Table 4). Specifically, agreement for the presence of a supermarket, drug and pharmacy store or dollar and variety store was significantly higher when using either the 2- or 3-mile network-based buffer sizes compared to the 1-mile network-based buffer, indicating that the 1-mile buffer may underestimate a participant’s perceived neighborhood. In addition, agreement for the presence of a fast food restaurant was significantly higher when using the 2-mile network-based buffer compared to the 1-mile buffer.
We also noted that the measures of agreement for the 3- and 5-mile buffers were significantly lower than those for the 1-mile Euclidean- and network-based buffers. However, this decrease in agreement is most likely due to the inevitable increase in the number of stores introduced mathematically into the denominator when calculating agreement statistics for much larger GIS-based buffer sizes. Thus, these findings are not necessarily comparable to the 1-mile buffer size after a certain mathematical threshold.
Finally, agreement was examined by stratifying participants by urban or non-urban designation (Table 5). Overall, the percent agreement was significantly lower for supermarkets and dollar and variety stores in urban participants compared to non-urban participants using either 1-mile Euclidean- or network-based buffers. In addition, the percent agreement was significantly lower for drug and pharmacy stores and fast food restaurants in urban participants compared to non-urban participants using a 1-mile network-based buffer. However, there were few significant differences between urban and non-urban participants when examining Kappa values, as only the agreement for drug and pharmacy stores differed significantly using either Euclidean- or network-based buffers, indicating a significantly lower agreement in non-urban participants compared to urban participants. Lastly, urban participants had a significantly lower chance-adjusted agreement for dollar and variety stores compared to non-urban participants using the network-based buffer.
In this study, we examined the relationship between perceived and GIS-based presence of retail food outlets in a sample of adults living in South Carolina. Overall, an individual’s perception of the presence or absence of a food store located within 1 mile of his/her home was moderately agreeable with objective, GIS-based measures of food outlet presence. Kappa values ranged from 0.44 to 0.61 when comparing a participant’s perceived presence of food outlets to the GIS-based presence of food outlets using a 1-mile Euclidean-based neighborhood buffer and 0.35 to 0.50 using a 1-mile network-based buffer.
Our findings also suggest that the performance of an individual’s perceptions was most optimal when using a 2- or 3-mile geographic-based neighborhood boundary. Kappa values for comparisons of these perceptions to standard 1-mile Euclidean- or network-based neighborhood buffer sizes were significantly higher for supermarkets, drug and pharmacy stores and dollar and variety stores when using either 2- or 3-mile network-based buffer sizes compared to a 1-mile network-based buffer. In addition, the agreement for fast food restaurants was significantly higher using the 2-mile network-based buffer. These findings suggest that researchers should consider the neighborhood boundaries selected as a reference in food environment measures. Although Euclidean-based measures were not significantly different, when analyzing network-based measures, researchers should consider the most appropriate boundary size: it could be that a 1-mile network-based buffer does not necessarily best characterize a participant’s perceived neighborhood.
To the best of our knowledge, this is the first study to explore the impact of varying neighborhood buffer sizes on the relationship between perceived and actual presence of various types of food outlets. However, we do not wish to overstate the contribution of our findings, as we did not have varied neighborhood boundaries when administrating the perception questionnaire. A future study comparing both perceived and GIS-based measures and utilizing varied neighborhood boundaries would be ideal.
Agreement statistics also differed significantly when considering urbanicity. Specifically, the percent agreement was significantly lower for supermarkets and dollar and variety stores in urban participants compared to non-urban participants using either 1-mile Euclidean- or network-based buffers. In addition, the percent agreement was significantly lower for drug and pharmacy stores and fast food restaurants in urban participants using a 1-mile network-based buffer. However, given that we observed few significant differences by urbanicity after accounting for chance agreement via Kappa statistics, this finding suggests that for supermarkets, convenience stores and franchised fast food outlets, residents of urban and non-urban areas do not differ substantially in terms of their reported agreement.
Two recent studies compared perceived and GIS-based presence of food outlets (Williams et al., 2011; Caspi et al., 2012b). In a sample of women aged 18–65 years in Melbourne, Australia, Williams et al. found that the match (percent agreement) between perceived and objective food environments was poor, reporting that approximately 50% of women had accurate reports between their perceptions and objective measures of supermarket presence within 800 m (~0.5 miles) of their home (Williams et al., 2011). For fast food stores, this agreement was only 40%. This outcome is much different than that in our study, where we report percent agreements of 78% using a 1-mile network-based buffer size and 82% using a 1-mile Euclidean-based buffer. Possible discrepancies between our results and those of Williams et al. may be traced to the nature of the perception questions used and the choice of GIS-based measure. In our study, we asked study participants to think of their neighborhood environment specifically as a 1-mile buffer or 20-minute walk around their home, whereas participants in the study by Williams et al. were asked “are the following within walking distance of your home?” without any guide to what is meant by walking distance. Moreover, Williams et al. classified participants via GIS as having or not having each type of store nearby using an 800-m (~0.5-mile) definition for walkable distance.
In another study, Caspi et al. reported that 69% of residents matched, via percent agreement, between objective and perceived presence of a supermarket within 1 kilometer (~0.6 miles) in a sample of low-income housing residents in three urban areas in the greater Boston area. This may suggest that Caspi et al. used a buffer size that was too small to optimize concordance between a person’s perceived and objectively measured food environment. Moreover, Caspi et al. chose their cut point for a neighborhood buffer at 1 kilometer because the researchers were concerned about artificially high levels of discordance based on previous buffers used in the literature and because most of their participants reported a supermarket within walking distance (Caspi, Kawachi, Subramanian, Adamkiewicz, & Sorensen 2012a). It is important to point out, however, that in our study, the majority of food shoppers traveled by car (>90%) and did not walk to food outlets, even in urban neighborhoods.
Our study contributes to food environment research by not only exploring the agreement between an individual’s perceived and actual presence of supermarkets and other food retail outlets but also examining how this relationship changes using different boundaries to define a person’s neighborhood. It could be that the Williams et al. and Caspi et al. studies using smaller cut points to define a person’s neighborhood may have affected the relationship between perception and the GIS-based environment. In our study, we found slight improvement, via significant increases in Kappa statistics, when increasing the network-based neighborhood definition from the 1-mile to the 2- or 3-mile buffer sizes. It is possible that individuals are not able to mentally conceptualize the extent of a 1-mile buffer around their home based on personal and behavioral factors. Thus, additional studies are needed that vary the spatial frame of the perception questions (i.e., 2 miles, 3 miles, etc.) and subsequently compare respondents’ perceptions with validated GIS-based measures with the same buffers as those used in the perception questions.
Our study has several limitations. First, women constituted the majority of the sample because we selected the primary food shopper of the household. This may limit the generalizability of our findings. Second, our landline-based telephone sample yielded an age distribution with an average age in the middle-to-older category, which does not represent all residents. Third, the perception data were collected nearly 1 year after the completion of the validated field census. However, it should be noted that this time gap between data collection is comparable to those reported in other similar studies (Moore et al., 2008a; Gustafson et al., 2011).
Strengths of the study included the use of a set of reliable food environment questions examining the perceived presence of various types of food outlets (Ma et al., 2013). Reliability ranged from 0.58 to 0.96 in a test-retest analysis (Ma et al., 2013). Secondly, our measure of GIS-based presence was based on a validated field census of our study region (Liese et al., 2010, 2013a). In addition, our study area contained both urban and non-urban communities. Thus, these findings may be beneficial and comparable to new studies examining populations in the southeastern United States, where there is a mix of urban and non-urban neighborhoods. Williams et al. and Caspi et al. both examined residents living in urban communities only.
GIS is an important and useful tool for describing a food environment and relating this environment to an individual’s diet, weight status and neighborhood characteristics; however, measures based on GIS are susceptible to a number of errors and inaccuracies (Liese et al., 2010, 2013a; Powell et al., 2011; Forsyth et al., 2010). This susceptibility is due partially to the fact that few datasets and resources have been conceived with the sole purpose of capturing food outlet access. Moreover, the effort to validate commercial and secondary datasets is often not feasible because of a lack of resources and the expense of research staff traveling into the field (Forsyth et al., 2010). There is debate as to whether it may be more economical and accurate to utilize perceptions measures, either alone or in tandem with GIS-based measurements (Moore et al., 2008a; Caspi et al., 2012a). For studies that are still in the planning stage, our results could suggest that a simple survey question alone may be advantageous, thereby saving a study team the effort of lengthy data cleaning and merging, ground truthing and/or GIS analyses. Moreover, our study demonstrates there is a moderate correspondence between what retail food outlets an individual perceives in his/her neighborhood compared to what is actually present, especially in terms of supermarkets. However, our study also points out that there is still room to evaluate appropriate neighborhood boundaries for both GIS-based and perceived presence of retail food outlets. More evidence is needed to determine whether to rely on perception-based measures of retail food outlets exclusively.
This project was supported by grant R21CA132133-02S1 from the National Cancer Institute. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health. The authors would like to thank James Hibbert for all GIS data management and mapping.
Conflicts of Interest
The authors declare no conflicts of interest.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.