PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmchsrBioMed Centralsearchsubmit a manuscriptregisterthis articleBMC Health Services Research
 
BMC Health Serv Res. 2013; 13: 258.
Published online Jul 4, 2013. doi:  10.1186/1472-6963-13-258
PMCID: PMC3708753
Design of an online health-promoting community: negotiating user community needs with public health goals and service capabilities
Joakim Ekberg,corresponding author1 Toomas Timpka,1,2 Marianne Angbratt,2 Linda Frank,3 Anna-Maria Norén,4 Lena Hedin,3 Emelie Andersen,3 Elin A Gursky,5 and Boel Andersson Gäre6,7
1Department of Medical and Health Sciences, Linköping University, SE-581 83, Linköping, Sweden
2Centre for Public Health Sciences, Östergötland County Council, Linköping, Sweden
3Department of Health, Jönköping County Council, Jönköping, Sweden
4Centre for Public Health, Kalmar County Council, Oskarshamn, Sweden
5Analytic Services Inc, Arlington, VA, USA
6Futurum, Jönköping County Council, Jönköping, Sweden
7Jönköpings Academy for Improvement of Health and Welfare, Jönköping University, Jönköping, Sweden
corresponding authorCorresponding author.
Joakim Ekberg: joakim.ekberg/at/liu.se; Toomas Timpka: toomas.timpka/at/liu.se; Marianne Angbratt: marianne.angbratt/at/lio.se; Linda Frank: linda.frank/at/lj.se; Anna-Maria Norén: annamaria.noren/at/ltkalmar.se; Lena Hedin: lena.hedin/at/lj.se; Emelie Andersen: emelie.andersen/at/lj.se; Elin A Gursky: elin.gursky/at/anser.org; Boel Andersson Gäre: boel.andersson.gare/at/lj.se
Received October 20, 2012; Accepted June 28, 2013.
Background
An online health-promoting community (OHPC) has the potential to promote health and advance new means of dialogue between public health representatives and the general public. The aim of this study was to examine what aspects of an OHPC that are critical for satisfying the needs of the user community and public health goals and service capabilities.
Methods
Community-based participatory research methods were used for data collection and analysis, and participatory design principles to develop a case study OHPC for adolescents. Qualitative data from adolescents on health appraisals and perspectives on health information were collected in a Swedish health service region and classified into categories of user health information exchange needs. A composite design rationale for the OHPC was completed by linking the identified user needs, user-derived requirements, and technical and organizational systems solutions. Conflicts between end-user requirements and organizational goals and resources were identified.
Results
The most prominent health information needs were associated to food, exercise, and well-being. The assessment of the design rationale document and prototype in light of the regional public health goals and service capabilities showed that compromises were needed to resolve conflicts involving the management of organizational resources and responsibilities. The users wanted to discuss health issues with health experts having little time to set aside to the OHPC and it was unclear who should set the norms for the online discussions.
Conclusions
OHPCs can be designed to satisfy both the needs of user communities and public health goals and service capabilities. Compromises are needed to resolve conflicts between users’ needs to discuss health issues with domain experts and the management of resources and responsibilities in public health organizations.
Keywords: Community-based participatory research, Health promotion, Adolescents, Health service
Health promotion denotes a set of principles for endorsement of the health of the population from an inclusive perspective. These principles differ from corresponding approaches for disease prevention, where epidemiological knowledge of specific health risks is used for prevention and early detection of particular diseases. The theoretical basis for health promotion is that health is determined in interaction with communities and their residents, and that changes in the physical and social living environment influence the health status [1]. Health is thus seen not only to reflect the absence of disease but also encompasses aspects such as capacity to fulfil vital life goals [2] and psychological well-being and outlook [3].
Due to the advances in web technology, Internet-based applications are now inexpensive alternatives for health promotion, e.g. by self-directed and informal learning in online communities. An online community refers to a gathering of individuals in a virtual space who form a network for long-term public discussions on common interests or experiences [4]. In contrast to geographic communities who share interests in physical tasks and activities, the emphasis in online communities is on learning and information exchange [5,6]. Self-directed learning, where the individual is in command of what should be learned, stands in contrast to health education, where health experts provide information that they think the recipients need [7].
An online health-promoting community (OHPC) has the potential to promote health and advance new means of dialogue between public health representatives and the general public. In Sweden more than 90% of adolescents have Internet access and go online several times a week [8]. However, an OHPC must be managed and even if advances in web technologies simplify the initiation of online communities, management and maintenance must be sustainable. An online community involving the health care service needs to be in accordance with public health goals and service capacity. Public health representatives cannot be expected to invest time and funding in an intervention at odds with the goals of the public health service.
In Sweden, obesity among children has not yet matched the epidemic proportions reported from other parts of the world [9-12]. However, among adolescents, overweight and self-consciousness regarding body shape, diet and exercise influence their social, psychological and physical health [13-15]. Obese children may be in need of secondary prevention because of the adverse effects related to obesity, such as development of coronary heart disease, related to long-term obesity [16], but it is less obvious what to prevent in the remaining population. General interventions, such as health education, to prevent overweight and obesity are problematic because of the lack of procedures that have general application [13,17]. There is a need for personalized community-based health promotion [18]. Online interventions are especially suitable for this purpose when considering the amount of time adolescents spend online [19].
The aim of this study was to examine what aspects of an OHPC that are critical for satisfying the needs of the user community and public health goals and service capabilities. The study was performed in a health care region (population 1 200 000) in the south east of Sweden, where analyses of obesity data had shown that the need for obesity prevention among adolescents was greater than the resources available [20]. The results indicated that obese adolescents would benefit from more specialized interventions; a need for an online public health intervention addressing the non-obese adolescent population was also identified.
Community-based participatory research methods [21,22] were used for the study and participatory design principles were used for the OHPC systems development [23]. Community-based participatory research methods provide an arrangement for balanced influence and control over research processes by non-academic researchers and support structures and processes for academic–community partnerships to improve health [24].
Qualitative data from adolescents on health appraisals and perspectives on health information were collected in a Swedish health service region. Using a three-step analysis, the data were transformed to specify the structure and functions of an OHPC intervention (Figure 1). In the first step, the crude data were analysed and structured into health information needs. The identified needs were then transformed into requirements for an OHPC intervention by an interdisciplinary expert team. In this context, needs refer to problems expressed that warrant a solution; requirements are associated with the design of an information system [25]. In the third step, a prototype based on the user-derived requirements was compared with regional public health goals and service capabilities by public health official representatives from the region, and modified accordingly. Conflicts between the system requirements derived from the adolescents and public health goals and resources were identified.
Figure 1
Figure 1
A three-step analysis to specify the structure and functions of an OHPC intervention.
Data collection
The model population for the development of the OHPC intervention consisted of high school students in their senior year (17–18 years of age). A convenience sample (n = 65) representing the population of adolescents aged 15–20 years (n  50 000) in the study region was recruited. Focus group interview sessions were arranged with adolescents according to two criteria: home community sociocultural profile (industrial–technological, small business–church dominated, small business–agricultural) and high school program profile (high school program in science, arts or nursing). High schools corresponding to the listed program profiles in each study community were contacted and asked for assistance with recruiting students for focus group meetings. Qualitative data were collected using two sets of focus group interviews. The topics for the first set of interviews were physical health, psychological health, and health products; the second set of interviews covered health information in the media, information needs and information-seeking behaviour. The protocol for the focus group sessions is provided in Additional file 1. Initially, each student was asked to participate in both focus groups but some cancellations occurred and stand-ins were accepted. Of the 65 participating adolescents, 48 participated in both sets of focus group interviews, 12 only in the first, and 5 only in the second. In total 18 focus group sessions took place.
Data from the focus groups were collected over a 2-month period in 2010. The focus groups were video recorded and lasted for approximately 90 minutes for the first occasion and 60 minutes for the second. In total, approximately 20 hours of video were collected. In each focus group, two moderators were present: one moderating the discussion and the other making observations. The data were collected by means of formal natural group discussion defined as a group interview with invited people who already know each other [26]. The resulting requirements and various prototypes were demonstrated and discussed with representatives from regional public health services in meetings and telephone conferences.
The focus groups took place after hours in the students’ free time; informed consent was obtained from the participants and with the permission of schools and teachers. The project was approved by the Ethics Committee of Linköping University, Dnr: 479–31.
Data analysis
Step 1. Identification of health information exchange needs
An analysis of health information exchange needs was performed using data from the focus group meetings. Summaries from each focus group were compiled as transcripts from the video recordings by one of the moderators and sent to the second moderator of the focus group to make sure that the summary was accurate. The transcribed content was coded based on themes identified in the material by the first moderator.
Statements from the focus groups related to health information exchange were identified and a summative content analysis of these units was performed to identify specific user needs as related to an information system. In addition, statements regarding content and topics of interest were used to assist in drafting the content for the information system.
Step 2. User-derived requirements for the information system
A process to determine the user-derived system requirements was undertaken based on the principles of design space analysis [27,28] with design rationales captured using an informal representation [29]. A design team consisting of two computer scientists, an interaction designer, and a cognitive scientist drafted the requirements for an OHPC based on the health information exchange needs identified. The design decisions were then constructed into a composite design rationale. A design rationale is an explicit list of decisions made during a design process, and the reasons why those decisions were made [30]. The design rationale primarily consisted of the identified user needs, the user-derived requirements, technical and organizational system solutions, alternative solutions, and the reason for their rejection when applicable. The design rationale was also presented as a prototype OHPC.
Step 3. Conflicts between user-derived requirements and regional public health goals and service capability
In the final step, the design rationale document and prototype were presented to a panel of six public health professionals working with adolescent obesity, representing the health service providers in the region. The task presented to the panel was to assess the prototype with regard to the regional public health goals and service capability for an OHPC intervention. Design solutions were reviewed in sessions at which the expected positive and negative consequences of each design solution were examined in light of its hypothetical implementation in the study setting. When conflicts between requirements, public health goals, and service capabilities were found, the solutions were adjusted in order to neutralize the conflicts. These adaptations were compiled, analysed for common features, and documented.
Step 1. Identification of health information exchange needs
All participants studied at their senior high school year (17–18 years of age), of which 69% were female. The focus group transcript was divided into 505 statement units related to health information exchange. These units were classified into ten categories of user health information exchange needs (Table 1); content-related needs were excluded. The most prominent content-related statements were about food, exercise, and well-being.
Table 1
Table 1
User health information exchange needs
Step 2. User-derived requirements for the information system
The structures and functions identified as requirements to satisfy the adolescents’ health information exchange needs included both technical and organizational components. A composite design rationale for the OHPC was completed by linking the identified user needs, user-derived requirements, and technical and organizational system solutions. When applicable, alternative solutions and the reason for their rejection were also listed (Table 2).
Table 2
Table 2
Reconstructed design rationales
Step 3. Conflicts between the user-derived requirements and regional public health goals and service capability
The assessment of the design rationale document and prototype in light of the regional public health goals and service capabilities raised concerns about organizational aspects of the design solutions. No technical aspects of the design solution were found to be problematic. In brief, compromises were needed to resolve conflicts involving the management of organizational resources and responsibilities.
Two conflicts arose about making a health panel available for questions from adolescents and engaging practitioners in online discussions to meet the adolescents’ needs (personal advice from health professionals and discussions in which experts participate). With regard to the resources at hand, monitoring the potentially vast number of questions and discussions was deemed to be too high a workload for the practitioners. The compromise found was to use a previously rejected design solution of using vodcasts. To produce the vodcasts, it was decided to appoint a community manager to monitor the discussion, collect questions and hot topics, and regularly interview the practitioners. A similar problem occurred when the practitioners were expected to produce editorial content as comments on current health-related events and media reports to meet the need for dismantling of myths and misconceptions. Because of the need for special care when producing material for the public from health services, these editorials were deemed to demand too much work. The compromise found was to transfer this responsibility to public health academics because they are more familiar with producing editorial material, and public communications from them would not be seen to come from the regional health service.
The fourth conflict was on the choice of OHPC editorial content based on monitoring of online discussions in order to meet the adolescents’ need for information about our interests. It was speculated that the editorial content would then be focused exclusively on topics with low priority in public health policies, e.g. make-up and skincare products. However, the practitioners were sympathetic to an approach to adapt the editorial content to adolescents’ interests and a compromise was decided whereby monitoring was to be accompanied by a clear statement of the purpose and focus of the OHPC.
The fifth conflict was about the moderators keeping up with the online activity if the OHPC intervention was used extensively, regarding the need to maintain a civil discourse. The compromise agreed on involved recruiting moderators from members of the adolescent user community based on exemplary conduct.
This study set out to examine what aspects of an OHPC that are critical for satisfying the needs of user communities and public health goals and service capabilities. The most important observation was that compromises in the OHPC design were needed to resolve conflicts between needs to discuss health issues with domain experts in the end-user population and resources and responsibilities related to online activities in the public health organization. In a comparable OHPC development initiative in Ontario, Canada (http://www.youthspark.ca), a method for systems development and evaluation of an OHPC has been comprehensively documented [31]. The current results add knowledge about the need to negotiate public health interests and end-user needs in the OHPC design process.
Web technology is becoming increasingly prevalent in health services [32,33]. In a recent review of computer and web-based interventions to increase physical activity among adolescents, most reported interventions had led to a significant increase in physical activity, but the effects were short lived [34]. Experiences from smoking cessation programs suggest that using only Internet initiatives based on social cognitive theory is not effective [35], but that a combination of cell phone-based “push” technologies, peer-to-peer contact, and web-based information on health and coping strategies can be a promising long-term treatment option [36]. In parallel, online communities providing several of the latter services to users have been used in health services for self-management of health problems [37] and establishment of communities of practice [38]. An OHPC can be regarded as a community-based, long-term, and inclusive health promotion program [1], although mediated over the Internet. When such OHPCs are developed on a large scale, the existing models for health promotion program implementation and evaluation must also be extended to fit the virtual setting. The OHPCs have to be evaluated with regard to behavioural patterns and health effects in the intended population. In these evaluations, there is thus a need to measure the outreach, usage, and effectiveness of the online system, using methods ranging from web surveys and focus groups to epidemiological studies. Such large-scale evaluations of OHPC designs in practice settings are much warranted, but also associated with considerable methodological challenges.
Participatory design proponents suggest that because social factors important in the implementation process are highlighted in a participatory approach, the resulting system designs are more sustainable compared with those developed using traditional systems development methods [39]. However, participatory methods have also been criticized for neglecting the later technical development stages [40] and being unrealistic with regard to the time investments demanded by practitioners [41]. Techniques such as the Voice of the Customer table and House of Quality [42] are well-established but resource-consuming approaches to the technical system accurateness problem. For instance, a possible problem with an OHPC initiative is that it may encourage young people to post and discuss health issues and problems in a manner that they later may come to regret. Even though it is easy to censor content after the fact, information online tends to persist. The use of design rationales in this study enabled public health practitioners to document this design problem, and analyse their views on the consequences of possible solutions. However, these views needed to be scrutinized also by engineers and legal expert before being implemented. This example illustrates that a participatory design process must be structured to include not only a representative group of end-users, but also the necessary expertise. If the experts not are available in the primary process, they have to be recruited downstream to adjust and approve the final systems design. For less complicated design problems, consultation of the scientific literature may be satisfactory. For the example problem, it is possible that consultation review of privacy guidelines for social media and maturation of online behaviour may have been sufficient to reduce the risks associated with sharing online [43].
The study has several limitations that need to be taken into account when interpreting the results. Allowing practitioners and the target community to influence the qualitative analysis process may be interpreted as surrendering control of the study. However, in accordance with the suggested increased use of qualitative data from in-depth interviews and observations for evaluating the context, process, impact, and outcome of community-based interventions [21,44-48], we contend that the participatory method used in this study is adequate for capturing the context and process of the community development. In such interventions, experimental control may be incongruous when taking the different sociopolitical structures of communities into account [44,46,49]. To ensure contextual soundness, the practitioners were in this study included in their actual professional roles, and the adolescents were regarded as the primary end-users, rather than merely informants. Also, including adolescents from the end-user community from several municipalities and school programs in the research was an attempt to decrease the threats to soundness that are associated with a convenience sample and that the adolescents would be disingenuous.
Conclusions
The study shows that an OHPC can be designed to satisfy both the needs of the user community and public health goals and service capabilities. The main results are that compromises in OHPC design are needed to resolve conflicts between needs and requirements to discuss health issues with domain experts in the end-user population and the management of resources and responsibilities related to online activities in the public health organization.
Abbreviations
OHPC: Online health-promoting community.
Competing interests
The authors declare that they have no competing interests.
JE participated in the design of the study, performed the analysis and drafted the manuscript. TT participated in the design of the study, performed the analysis and drafted the manuscript. MA participated in the design of the study, performed the analysis and revised the manuscript. LF participated in the design of the study, performed the analysis and revised the manuscript. AN participated in the design of the study, performed the analysis and revised the manuscript. LH participated in the design of the study, performed the analysis and revised the manuscript. EA participated in the design of the study, performed the analysis and revised the manuscript. EAG participated in the design of the study, performed the analysis and revised the manuscript. BA participated in the design of the study, performed the analysis and revised the manuscript. All authors read and approved the final manuscript.
Pre-publication history
The pre-publication history for this paper can be accessed here:
Supplementary Material
Additional file 1
Protocol for the focus group sessions.
Acknowledgements
The study was funded by a research grant from the Research Council for South-East Sweden (FORSS). The funding agency did not influence the research process in any aspect. We are grateful to all public health officials and adolescents in the region who made this study possible.
  • Green J, Thorogood N. Qualitative Methods for Health Research (Introducing Qualitative Methods Series) 2. London: Sage; 2004.
  • Nordenfelt L. On the Nature of Health. Amsterdam: Kluwer; 2nd, revised edn. Dordrecht, The Netherlands: Kluwer Academic Publishers; 1995.
  • Snyder CR, Lopez SJ. Positive Psychology: The Scientific and Practical Exploration of Human Strengths. Thousand Oaks, CA: Sage; 2007.
  • Rheingold H. The Virtual Community. New York: Addison-Wesley; 1993.
  • Lave J, Wenger E. Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press; 1991.
  • Ekberg J, Ericson L, Timpka T, Eriksson H, Nordfeldt S, Hanberger L, Ludvigsson J. Web 2.0 systems supporting childhood chronic disease management: design guidelines based on information behaviour and social learning theories. J Med Syst. 2010;13:107–117. doi: 10.1007/s10916-008-9222-0. [PubMed] [Cross Ref]
  • Brydges R, Nair P, Ma I, Shanks D, Hatala R. Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ. 2012;13(7):648–656. doi: 10.1111/j.1365-2923.2012.04268.x. [PubMed] [Cross Ref]
  • Medierådet. Ungar & Medier: Fakta om barns och ungas användning och upplevelser av medier. Stockholm: Kulturdepartementet; 2010. ISSN-1102-447X.
  • Lissner L, Sohlström A, Sundblom E, Sjöberg A. Trends in overweight and obesity in Swedish schoolchildren 1999–2005: has the epidemic reached a plateau? Obes Rev. 2010;13:553–559. [PubMed]
  • Bergström E, Blomquist HK. Is the prevalence of overweight and obesity declining among 4-year-old Swedish children? Acta Paediatr. 2009;13:1956–1958. doi: 10.1111/j.1651-2227.2009.01490.x. [PubMed] [Cross Ref]
  • Sjöberg A, Lissner L, Albertsson-Wikland K, Mårild S. Recent anthropometric trends among Swedish school children: evidence for decreasing prevalence of overweight in girls. Acta Paediatr. 2008;13:118–123. doi: 10.1111/j.1651-2227.2007.00613.x. [PubMed] [Cross Ref]
  • Fåhraeus C, Wendt L, Nilsson M, Isaksson H, Alm A, Andersson-Gäre B. Overweight and obesity in twenty-year-old Swedes in relation to birthweight and weight development during childhood. Acta Paediatr. 2012;13:637–642. doi: 10.1111/j.1651-2227.2012.02623.x. [PubMed] [Cross Ref]
  • Shaw SM, Kemeny L. Fitness promotion for adolescent girls: the impact and effectiveness of promotional material which emphasizes the slim ideal. Adolescence. 1989;13:677–687. [PubMed]
  • Setiloane K. Being overweight: negative outcomes for African American adolescents. Nutr Health. 2004;13:61–66. doi: 10.1177/026010600401800106. [PubMed] [Cross Ref]
  • Powers PS, Bruty H. Pharmacotherapy for eating disorders and obesity. Child Adolesc Psychiatr Clin North Am. 2009;13:175–187. doi: 10.1016/j.chc.2008.07.009. [PubMed] [Cross Ref]
  • Owen CG, Whincup PH, Whincup PH, Cook DG, Osmond C, Eriksson JG, Kaye SJ, Rudnicka AR, Chou QA, Orfei L. Is body mass index before middle age related to coronary heart disease risk in later life? Evidence from observational studies. Int J Obes (Lond) 2009;13:866–877. doi: 10.1038/ijo.2009.102. [PMC free article] [PubMed] [Cross Ref]
  • Carter OB. The weighty issue of Australian television food advertising and childhood obesity. Health Promot J Austr. 2006;13:5–11. [PubMed]
  • Anderson AS. How to implement dietary changes to prevent the development of metabolic syndrome. Br J Nutr. 2000;13:165–168. [PubMed]
  • Kind T. The Internet as an adjunct for pediatric primary care. Curr Opin Pediatr. 2009;13:805–810. doi: 10.1097/MOP.0b013e328331e7b4. [PubMed] [Cross Ref]
  • Ekberg J, Angbratt M, Valter L, Nordvall M, Timpka T. History matters: childhood weight trajectories as a basis for planning community-based obesity prevention to adolescents. Int J Obes (Lond) 2012;13:524–528. doi: 10.1038/ijo.2011.263. [PubMed] [Cross Ref]
  • Israel BA, Schultz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 1998;13:173–202. doi: 10.1146/annurev.publhealth.19.1.173. [PubMed] [Cross Ref]
  • Andrews JO, Newman SD, Meadows O, Cox MJ, Bunting S. Partnership readiness for community-based participatory research. Health Educ Res. 2012;13:555–571. doi: 10.1093/her/cyq050. [PMC free article] [PubMed] [Cross Ref]
  • Sjöberg C, Timpka T. Participatory design of information systems in health care. J Am Med Inform Assoc. 1998;13:177–183. doi: 10.1136/jamia.1998.0050177. [PMC free article] [PubMed] [Cross Ref]
  • Israel BA, Schultz AJ, Parker EA, Becker AB. Community-based participatory research: policy recommendations for promoting a partnership approach in health research. Educ Health. 2001;13:182–197. doi: 10.1080/13576280110051055. [PubMed] [Cross Ref]
  • Löwgren J. Proceedings of the Symposium on Designing Interactive Systems (DIS ’95) New York: ACM Press; 1995. Applying design methodology to software development; pp. 87–95.
  • Green J, Tones K. Health Promotion. 2. London: Sage; 2010.
  • Maclean A, Young RM, Moran T. Design rationale: the argument behind the artifact, CHI '89 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Wings for the Mind. New York: ACM Press; 1989. pp. 247–252.
  • Bellotti VME, MacLean A, Moran T. Structuring the design space by formulating appropriate design rationale questions. SIGCHI Bull. 1991;13:80–81. doi: 10.1145/126729.1056051. [Cross Ref]
  • Lee J. Design rationale systems: understanding the issues. IEEE Expert. 1997;13:78–85.
  • Jarczyk AP, Löffler P, Shipman III FM. Design rationale for software engineering: a survey. Proceedings of the 25th Hawaii International Conference on System, Sciences. 1992;13:577–586.
  • Valaitis R, O’Mara L, Bezaire S. Tailoring Online Health Promotion: A Toolkit for Communities. University School of Nursing: Produced by Huron County Health Unit and McMaster; 2007.
  • Nicholas J. The role of internet technology and social branding in improving the mental health and wellbeing of young people. Perspect Public Health. 2010;13:86–90. doi: 10.1177/1757913908101797. [PubMed] [Cross Ref]
  • Mockford C, Stansizewska S, Griffiths F, Herron-Marx S. The impact of patient and public involvement of UK NHS health care: a systematic review. Int J Qual Health Care. 2012;13:28–38. doi: 10.1093/intqhc/mzr066. [PubMed] [Cross Ref]
  • Hamel LM, Robbins LB, Wilbur J. Computer- and web-based interventions to increase preadolescent and adolescent physical activity: a systematic review. J Adv Nurs. 2010;13:251–268. [PubMed]
  • Mason D, Gilbert H, Sutton S. Effectiveness of web-based tailored smoking cessation advice reports (iQuit): a randomised trial. Addiction. 2012;13(12):2183–2190. doi: 10.1111/j.1360-0443.2012.03972.x. [PubMed] [Cross Ref]
  • Bredryen H, Drozd F, Kraft P. A digital smoking cessation program delivered through internet and cell phone without nicotine replacement (happy ending): randomized controlled trial. J Med Internet Res. 2008;13:e51. doi: 10.2196/jmir.1005. [PMC free article] [PubMed] [Cross Ref]
  • Li LC, Townsend AF, Badley EM. Self-management intervention in the digital age: new approaches to support people with rheumatologic conditions. Best Pract Res Clin Rheumatol. 2012;13:321–333. doi: 10.1016/j.berh.2012.05.005. [PubMed] [Cross Ref]
  • Barnett RS, Jones SC, Bennett S, Iverson D, Bonney A. General practice training and virtual communities of practice – a review of the literature. BMC Fam Pract. 2012;13:87. doi: 10.1186/1471-2296-13-87. [PMC free article] [PubMed] [Cross Ref]
  • Floyd C, Mehl WM, Reisin FM, Schmidt G, Wolf G. Out of Scandinavia: alternative approaches to software design and system development. Human–Comput Interact. 1989;13:253–350. doi: 10.1207/s15327051hci0404_1. [PubMed] [Cross Ref]
  • Tollmar K. Towards CSCW design in the Scandinavian tradition. Doctoral Thesis. Stockholm: Department of Numerical Analysis and Computer Science, Stockholm University; 2001.
  • Hirschheim R. Assessing participative systems design: some conclusions from an exploratory study. Inform Manage. 1983;13:13–32.
  • Hallberg N, Timpka T, Eriksson H. The medical software quality deployment method. Methods Inform Med. 1999;13:66–73. [PubMed]
  • Subrahmanyam K, Greenfield P. Online communication and adolescent relationships. Future Child. 2008;13:119–146. [PubMed]
  • Susser M. The tribulations of trials-intervention in communities. Am J Public Health. 1995;13:156–158. doi: 10.2105/AJPH.85.2.156. [PubMed] [Cross Ref]
  • Israel BA, Cummings KM, Dignan MB, Heaney CA, Perales DP, Simons-Morton BG, Zimmerman MA. Evaluation of health education programs: current assessment and future directions. Health Educ Q. 1995;13:364–389. [PubMed]
  • Mittelmark MB, Hunt MK, Heath GW, Schmid TL. Realistic outcomes: lessons from community-based research and demonstration programs for the prevention of cardiovascular diseases. J Public Health Policy. 1993;13:437–462. doi: 10.2307/3342877. [PubMed] [Cross Ref]
  • Koepsell TD, Wagner EH, Cheadle AC, Patrick DL, Martin DC, Diehr PH, Perrin EB, Kristal AR, Allan-Andrilla CH, Dey LJ. Selected methodological issues in evaluating community-based health promotion and disease prevention programs. Annu Rev Public Health. 1992;13:31–57. doi: 10.1146/annurev.pu.13.050192.000335. [PubMed] [Cross Ref]
  • Sorensen G, Emmons K, Hunt MK, Johnston D. Implications of the results of community intervention trials. Annu Rev Public Health. 1998;13:379–416. doi: 10.1146/annurev.publhealth.19.1.379. [PubMed] [Cross Ref]
  • Green LW, Richard L, Potvin L. Ecological foundations of health promotion. Am J Health Promot. 1996;13:270–281. doi: 10.4278/0890-1171-10.4.270. [PubMed] [Cross Ref]
Articles from BMC Health Services Research are provided here courtesy of
BioMed Central