PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Am Diet Assoc. Author manuscript; available in PMC 2011 February 22.
Published in final edited form as:
PMCID: PMC3042797
NIHMSID: NIHMS167479

Evidence-based development of a mobile telephone food record

Bethany L Six, RD, Research Assistant,1,3 TusaRebecca E Schap, MSc, RD, Research Assistant,1,3 Fengqing M Zhu, MS, Research Assistant,2,3 Anand Mariappan, MS, Research Assistant,2,3 Marc Bosch, MS, Research Assistant,2,3 Edward J Delp, PhD, The Charles William Harrison Distinguished Professor of Electrical and Computer Engineering,2,3 David S Ebert, PhD, Professor,2,3 Deborah A Kerr, PhD, Associate Professor,4 and Carol J Boushey, PhD, MPH, RD, Associate Professor1,3,*

Abstract

Mobile telephones with an integrated camera can provide a unique mechanism for collecting dietary information that reduces burden on record keepers. Objectives for this study were: (1) to test whether participant's proficiency with the mobile telephone food record (mpFR) improved after training and repeated use, and (2) to measure changes in perceptions regarding use of the mpFR after training and repeated use. Seventy-eight adolescents (26 males, 52 females) ages 11–18 y were recruited to use the mpFR for one or two meals. Proficiency with the mpFR was defined as capturing a useful image for image analysis and self-reported ease of use. Positive changes in perceptions regarding use of the mpFR were assumed to equate to potentially improved proficiency with the mpFR. Participants received instruction for using the mpFR prior to their first meal, and captured an image of their meals before and after eating. Following the first meal, participants took part in an interactive session where they received additional training on capturing images in various snacking situations and responded to questions about user preferences. Changes in the participants' abilities to capture useful images and perceptions about the usability of the mpFR were examined using McNemar, Wilcoxon rank-sum test, and paired t-test. After using the mpFR, the majority of participants (79%) agreed that the software was easy to use. Eleven percent of participants agreed taking images before snacking would be easy. After additional training, the percent increased significantly to 32% (p<.0001). For taking images after snacking, there was also improvement (21% before training and 43% after, p<.0001). Adolescents readily adopt new technologies; however the mpFR design needs to accommodate the lifestyles of its users to ensure useful images and continuous use. Further, these results suggest that additional training in using a new technology may improve the accuracy among users.

Keywords: dietary assessment, technology, adolescents, mobile telephones

Introduction

Dietary intake provides some of the most valuable insights for mounting intervention programs for the prevention of chronic diseases. However, accurate assessment of diet is problematic, especially in adolescents (1). Emerging technology in mobile telephones with higher resolution pictures, improved memory capacity, and faster processors allow these devices to process information not previously possible. Mobile telephones are widely used throughout the world and can provide a unique mechanism for collecting dietary information that reduces burden on record keepers. A dietary assessment application on a mobile telephone would be of value to practicing dietitians and researchers.

Previous results among adolescents showed that dietary assessment methods using technology, e.g., a Personal Digital Assistant with or without a camera or a disposable camera, were preferred over the traditional paper food record (2). This suggests that for adolescents, dietary methods that incorporate technology may improve cooperation and accuracy. Additionally, adolescents have expressed difficulty describing and estimating portions of foods and beverages consumed (2). Even with the use of portion size estimation aids, children's estimates are considered a source of error in quantifying food and energy intake (35). The development of a mobile telephone application for dietary assessment that fits into the lifestyle of young people may address many of the barriers identified above (6;7).

Most technology development emphasizes the requirements of the system as determined by the engineer or software programmer rather than the interaction of the user with the system (8;9). The study described here is part of the evidence-based development of a mobile telephone Food Record (mpFR) for adolescents (2;7;10). Evidence-based development of technology is a novel idea intended to focus on the effects of the system on its users and their lifestyles (9;11). Using this process will allow the design of the mpFR from the perspective of the user or the client rather than the perspective of the engineer or dietitian.

A form of evidence-based development is interaction design which is the discipline of defining the behavior of products that a user can interact with. This process of developing interactive products supports the way people communicate and interact in their everyday and working lives. The goals of interaction design are to develop usable products, in this case the mpFR, that are easy to learn, effective to use and provide an enjoyable experience. This is accomplished by involving users in the design process and receiving feedback about the product. Therefore, it is an iterative cycle of usability testing in which the user feedback is applied to the next version of the mpFR and tested again.

The objectives of this study informed the next development phase of the mpFR. The two objectives were: (1) to test whether participants' proficiency with the mpFR improved after training and repeated use, and (2) to measure changes in perceptions regarding use of the mpFR after training and repeated use. Proficiency with the mpFR was defined as capturing a useful image for image analysis and self-reported ease of use. Further, positive changes in perceptions regarding use of the mpFR were assumed to equate to potentially improved proficiency with the mpFR.

Methods

System Design

The client-server configuration for the mpFR is in Figure 1. In this system, a mobile telephone with an integrated camera can be used to capture images of food before and after eating and sent to the server (Figure 1, #1). A digital image is different from a photograph in that useful information, called metadata, is captured that is not visible, such as the time stamp and digital codes. Image analysis uses metadata to automatically identify characteristic features in food, such as color, texture, and intensity that are then used to identify a food. Methods for automatic identification of food using image analysis (Figure 1, #2) have been previously described (7;12).

Figure 1
Current client server configuration of the mobile telephone food record (mpFR). 1: User acquires an image of the food before and after eating with the mpFR. The image along with metadata (time stamp and digital code) are sent to the server. 2–3: ...

An object of known dimensions and markings, referred to as a fiducial marker, must be included in the image as a size reference. The inclusion of a fiducial marker in the image (e.g., the checkerboard square in Figures 2A and 2B) aids in the reconstruction of a 3-dimensional environment which allows for the estimation of the volume of the foods and beverages (Figure 1, #3). The image is then sent back to the user (Figure 1, #4) for the user to confirm that the foods are identified correctly (Figure 1, #5). Together the information from image analysis and volume estimation can be linked to a nutrient database (Figure 1, #6) to estimate energy and nutrients consumed (10). In the final step, the nutrient analysis can be sent to researchers or dietitians.

Study Design and Recruitment

Data were collected from two samples of participants. For Sample 1 and Sample 2, the study methods were approved by the Blinded University Institutional Review Board, and informed assent and consent were obtained from the participants and their parents, respectively.

Sample 1

The first sample was drawn from summer camps for adolescents, ages 11–18 years, conducted by a variety of university departments and taking place on the university campus. Participants from these camps participated in one lunch (n=63) and 55 (87%, 55/63) returned for breakfast the next morning. Following lunch (i.e., their first meal after using the mpFR) all of the participants took part in an interactive session.

Sample 2

The second sample was a convenience sample drawn from the local community (n=15). Recruitment was limited to those individuals between 11–18 years. Participants received all meals and snacks for a 24-hour period while being monitored under controlled conditions. These participants took part in the interactive session after breakfast (i.e., their first meal after using the mpFR); and data from their first and second meals are included in this analysis.

Eating Occasions

A systematic method was used to design the menus using the most common foods reported during two 24-hour recalls administered among 162 adolescents between 10–18 years (13;14), as well as foods from local school lunch menus. For each meal, foods and beverages were weighed, plated, and arranged using a predetermined place setting (See Figure 2A). Three menus for breakfasts and three menus for lunches were cycled between the sessions. All participants received instruction for using the mpFR prior to their first meal. In order to obtain an image useful for image analysis, participants were asked to include two items in each image: 1) all food and beverages, and 2) the fiducial marker. Participants ate to satiation and they were served seconds, if requested. The procedures for capturing images were then repeated for any additional portions.

HTC p4351 mobile telephones running Windows Mobile 6.0 were used. The software described previously (7;10) guided the user to select the meal occasion and capture an image of foods and beverages (See Figure 2B). After capturing the image, the user was prompted to review the image and then given a choice to retake the image or save the image. Once the user was satisfied with the image, the mobile telephone prompted the user to eat before proceeding to the next screen. At the next screen, the user was prompted to take an image of the place setting regardless of whether food and beverages remained. The final screen showed the before and after images prior to exiting the program. The participants were assisted during the meal by trained dietetic students and for Sample 1 only, staff recorded the number of images taken by each participant, as well as position (sitting/standing) while taking an image.

Interactive Session

Eleven forced-choice questions with appropriate visuals regarding the interaction design of mpFR were shown using PowerPoint®. Response choices ranged from `strongly agree' to `strongly disagree' and responses were collected with the eInstruction™ Classroom Performance System (CPS) (Cincinnati, Ohio). Three open-ended questions were presented: 1) If the program, cannot identify a food, how would you want the program to tell you that?; 2) If you need to then label the food with the correct name, how would you want to do that?; and 3) Would you want to narrate into the telephone out loud the names of the food?. The oral responses from the participants were recorded by staff. The participants received additional training through activities where they practiced capturing images in potentially problematic snacking scenarios, e.g., on the school bus, in the movie theater.

Data Analysis

For analysis, participants were separated into early and late adolescent age groups to address cognitive development: 11–14 years old (n=44) and 15–18 years old (n=33). Descriptive analysis included frequencies and percents. Differences in responses by categorical characteristics, i.e., age group and gender, were examined using chi-square, and ANOVA for quantitative variables. Differences within participants were examined using McNemar (categorical variables), Wilcoxon rank-sum test (ordinal variables) or paired t-test (quantitative variables). The qualitative responses to the open-ended questions were examined using content analysis (15).

Results and Discussion

A total of 78 participants (26 males, 52 females) ages 11–18 years used the mpFR for a first meal and 70 of those also used the mpFR again for a second meal. Characteristics of the samples are in Table 1. The participants were either in middle school or high school, and the mean age was 14.2 years. The participants were of diverse ethnic backgrounds.

Table 1
Characteristics of volunteers testing the usability of the mobile telephone food record (n=78)

With regard to being able to acquire a useful image, the majority of the participants saved images that included all of the foods and beverages in both the before and after images for the first meal (56/70, 80%) and the second meal (59/70, 84%). A smaller proportion included the entire fiducial marker in both the before and after images for the first meal (50/70, 71%) and slightly more for the second meal (54/70, 77%). After repeated use of the mpFR, there was no significant change in the number of participants who included the entire fiducial marker in their images. There were numerous items (e.g., condiments, beverages) to include in each image (See Figure 2A). Participant comments during the meal sessions suggest that it was a challenge to include all items in the image. Height of the adolescents may also have contributed to the challenge of including all items in each image. In one instance, a short statured participant stood on a chair in order to capture the best image. A possible solution is to ask the user to take multiple images if they cannot capture all food items in a single image, or push the tableware closer together. The individuals that consumed all foods and beverages recommended the addition of an option, “ate all food and beverages”; rather than capturing an image of completely empty plates and glasses.

Repeated use allowed participants to familiarize themselves with the mpFR. Just under half of Sample 1 took more than 1 image before the first meal (21/50, 42%), however this declined significantly (p=.033) to 22% (11/50) after the second meal. In all cases, the likelihood of taking one image was positively associated with standing rather than sitting to take an image. A small fraction of participants changed the orientation of the camera between the before and after meals, i.e., portrait versus landscape. This indicated the need for a prompt to take images in landscape providing consistency for image analysis.

Responses to the questions regarding use of the mpFR are shown in Table 2. A larger proportion of the participants agreed it would be easy to use a credit card sized fiducial marker as opposed to using a USB flash drive sized fiducial marker. Although a fiducial marker sized like a USB flash drive would be smaller, the credit card sized fiducial marker might be more convenient since it would fit in a wallet. Most adolescents have standard sized school identification cards, thus a credit card sized fiducial marker may be easier to incorporate into their current lifestyles.

Table 2
Responses from adolescents (11–18 y) after using a mobile telephone food record (n=78)

The majority of participants agreed that the software was easy to use. Just over one third of the participants agreed that it would be easy to remember to take images before meals and a slightly higher proportion agreed that it would be easy to remember to take images after meals (See Table 2). Despite the ease of using the software, the perception of being less likely to remember to take an image before eating versus after eating is consistent with individuals being hungry and eager to eat. Responses before additional training reflected the lowest proportion of individuals indicating that taking an image before a snack would be easy to remember (See Table 2). Following additional training in which the participants practiced taking snack images in awkward situations, significantly more participants agreed that taking images before (t=3.78, df=67, p =<.0001) and after snacking (t=3.89, df=69, p <.0001) would be easy.

Adolescents are assumed to have a high level of technology readiness and most of the adolescents were participating in technology themed camps. The results support that improved use of the mpFR can be achieved with additional training activities and interaction design changes. Specifically, alterations were initiated to ensure speedier capture of images of foods and beverages prior to eating.

Over half of the participants agreed that they preferred to stand while taking their images. Many participants found it difficult to capture all food items and the fiducial marker while sitting. These results in a controlled environment need to be confirmed in a variety of free-living situations. Adolescents may not readily stand to take images of their meals in more realistic situations.

Overall, there were no significant differences in the respondents' perceptions of the mpFR and their proficiency with the mpFR by age group or gender. This suggests that the interaction design for the mpFR would be the same across the adolescent age span.

The open-ended questions addressed the interface issues of the automatic identification of the foods through image analysis (Figure 1, #2) and the subsequent confirmation with the user (Figure 1, #5). The participants stated their suggestions for notification to confirm the food and beverage items. Most preferred that a pop-up message appear on the screen if additional information was needed. Other ideas included the appearance of an “X” or an arrow on or by the food in question. Also, several suggested having a list appear or a space; each of which could be edited with the correct information. As a means of identifying a misidentified item, the majority preferred filling in a text box or selecting from a list of possible options. There was a predominance of negative responses for narrating the food identification into the mobile telephone. Participants noted social awkwardness (e.g., “weird”, “creepy”, “they will think you are an idiot”) and under development of voice recognition software (e.g., “voice recognition is too iffy”, “Voice recognition doesn't work. You say `call Bob,' and it doesn't even know who to call.”). These suggestions informed the next phase of the development of the mpFR.

Conclusions

Evidence-based development entails a shift from focusing on the product to focusing on the process and the interaction of the user with the product. Using this process will allow the design of the mpFR from the perspective of the user or the client rather than the perspective of the engineer or dietitian. Formative evaluation outcomes and user feedback are important components of evidence-based development used when designing applications (11). In the same way that evidence based guidelines help dietitians perform more effectively, we anticipate that developing a product that users can comfortably use throughout the day will lead to cooperation that will translate to an accurate and less burdensome record of dietary intake.

Results from this study suggest that additional training in using this new technology may improve user cooperation. Adolescents' proficiency for capturing all foods, beverages, and the fiducial marker with the mpFR improved after the first use. Adolescents readily adopt new technologies; however the design of the mpFR needs to accommodate the lifestyles of its users to ensure useful images and continuous use throughout the day or multiple days. The results from this study will be reflected in future versions of the mpFR, thus providing an accurate, inexpensive, easy-to-use method for dietary assessment in dietetic practice and research.

ACKNOWLEGEMENT

Support for this work comes from the National Cancer Institute (1U01CA130784-01) and the National Institute of Diabetes, Digestive, and Kidney Disorders (1R01-DK073711-01A1).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Reference List

1. Livingstone MBE, Robson PJ, Wallace JMW. Issues in dietary intake assessment of children and adolescents. Br J Nutr. 2004;92:S213–S222. [PubMed]
2. Boushey CJ, Kerr DA, Wright J, Lutes KD, Ebert DS, Delp EJ. Use of technology in children's dietary assessment. Eur J Clin Nutr. 2009;63:S50–S57. [PMC free article] [PubMed]
3. Frobisher C, Maxwell SM. The estimation of food portion sizes: A comparison between using descriptions of portion sizes and a photographic food atlas by children and adults. J Hum Nutr Diet. 2003;16:181–8. [PubMed]
4. Matheson DM, Hanson KA, McDonald TE, Robinson TN. Validity of children's food portion estimates. Arch Pediatr Adolesc Med. 2002;156:867–71. [PubMed]
5. Lillegaard ITL, Overby NC, Andersen LF. Can children and adolescents use photographs of food to estimate portion sizes? Eur J Clin Nutr. 2005;59:611–7. [PubMed]
6. Winter J, Boushey CJ. Workshop 1: Use of technology in dietary assessment. Eur J Clin Nutr. 2009;63:S75–S77. [PMC free article] [PubMed]
7. Zhu F, Mariappan A, Boushey CJ, Kerr D, Lutes KD, Ebert DS, Delp EJ. Technology-assisted dietary assessment. Proceedings SPIE-The International Society for Optical Engineering. 2008;6814:681411.1–10. [PMC free article] [PubMed]
8. Sharp H, Rogers Y, Preece J. Interaction Design: Beyond Human-Computer Interaction. John Wiley & Sons, Ltd.; Chichester, West Sussex: 2007. What is interaction design? pp. 1–40.
9. Hertzum M, Simonsen J. Evidence-based development: A viable approach? Nordic Conference on Human Computer Interaction. Proceedings of the third Nordic conference on Human-computer interaction. 2004;82:385–388.
10. Mariappan A, Ruiz MB, Zhu F, Boushey CJ, Kerr DA, Ebert DS, Delp EJ. Personal dietary assessment using mobile devices. Proceedings SPIE-The International Society for Optical Engineering. 2009;7246:72460Z.1–12. [PMC free article] [PubMed]
11. Subar AF, Thompson FE, Potischman N, Forsyth BH, Buday R, Richards D, McNutt S, Hull SG, Guenther PM, Schatzkin A, Baranowski T. Formative research of a quick list for an automated self-administered 24-hour dietary recall. J Am Diet Assoc. 2007;107:1002–7. [PubMed]
12. Mariappan A, Igarta M, Taskiran C, Gandhi B, Delp EJ. A low-level approach to semantic classification of mobile multimedia content. Proceedings of SPIE-IS&T Electronic Imaging, SPIE. 2006;6074:607408.1–12.
13. Jensen JK, Gustafson D, Boushey CJ, Auld G, Bock MA, Bruhn CM, Gabel K, Misner S, Novotny R, Peck L, Read M. Development of a food frequency questionnaire to measure calcium intake of Asian, Hispanic, and White youth. J Am Diet Assoc. 2004;104:762–9. [PubMed]
14. Novotny R, Boushey C, Bock MA, Peck L, Auld G, Bruhn C, Gustafson D, Jensen JK, Misner S, Read M. Calcium intake of Asian, Hispanic and White youth. J Am Coll Nutr. 2003;22:64–70. [PubMed]
15. Harris JE, Gleason PM, Sheean PM, Boushey C, Beto JA, Bruemmer B. An introduction to qualitative research for food and nutrition professionals. J Am Diet Assoc. 2009;109:80–90. [PubMed]