PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jamiaJAMIA - The Journal of the American Medical Informatics AssociationInstructions for authorsCurrent TOC
 
J Am Med Inform Assoc. 2012 Sep-Oct; 19(5): 719–727.
Published online May 12, 2012. doi:  10.1136/amiajnl-2011-000433
PMCID: PMC3422818

Impact of a web-based personally controlled health management system on influenza vaccination and health services utilization rates: a randomized controlled trial

Abstract

Objective

To assess the impact of a web-based personally controlled health management system (PCHMS) on the uptake of seasonal influenza vaccine and primary care service utilization among university students and staff.

Materials and methods

A PCHMS called Healthy.me was developed and evaluated in a 2010 CONSORT-compliant two-group (6-month waitlist vs PCHMS) parallel randomized controlled trial (RCT) (allocation ratio 1:1). The PCHMS integrated an untethered personal health record with consumer care pathways, social forums, and messaging links with a health service provider.

Results

742 university students and staff met inclusion criteria and were randomized to a 6-month waitlist (n=372) or the PCHMS (n=370). Amongst the 470 participants eligible for primary analysis, PCHMS users were 6.7% (95% CI: 1.46 to 12.30) more likely than the waitlist to receive an influenza vaccine (waitlist: 4.9% (12/246, 95% CI 2.8 to 8.3) vs PCHMS: 11.6% (26/224, 95% CI 8.0 to 16.5); χ2=7.1, p=0.008). PCHMS participants were also 11.6% (95% CI 3.6 to 19.5) more likely to visit the health service provider (waitlist: 17.9% (44/246, 95% CI 13.6 to 23.2) vs PCHMS: 29.5% (66/224, 95% CI: 23.9 to 35.7); χ2=8.8, p=0.003). A dose–response effect was detected, where greater use of the PCHMS was associated with higher rates of vaccination (p=0.001) and health service provider visits (p=0.003).

Discussion

PCHMS can significantly increase consumer participation in preventive health activities, such as influenza vaccination.

Conclusions

Integrating a PCHMS into routine health service delivery systems appears to be an effective mechanism for enhancing consumer engagement in preventive health measures.

Trial registration

Australian New Zealand Clinical Trials Registry ACTRN12610000386033. http://www.anzctr.org.au/trial_view.aspx?id=335463.

Keywords: Personal health record, influenza, vaccination, preventive health, e-health, consumer, university, consumer informatics, human behavior modeling, social influence, decision support, internet intervention, patient safety, incident reporting, interruption, patient safety, bioinformatics, communication, decision support, safety

Background and significance

Personal health records (PHRs) offer an opportunity to directly engage consumers in preventive healthcare.1 Consumers are increasingly using a range of online systems to inform their decisions and manage their care.2–5 However, few studies have measured the impact of these systems on consumer behavior or the uptake of preventive measures.

In a recent review, only three of 10 randomized controlled trials (RCTs) involved electronic implementations of PHRs,6 and none demonstrated significant differences in health service utilization rates or the uptake of preventive measures.7–9 The review called for further trials to evaluate the effectiveness and sustainability of PHRs, and to examine the role of primary care service providers. Indeed, there is now a broad call for more RCTs to evaluate all classes of healthcare information systems.10

We describe an RCT of a personally controlled health management system (PCHMS) called Healthy.me, developed at the University of New South Wales (UNSW).11 This PCHMS integrates (i) an untethered PHR, (ii) consumer care pathways called ‘journeys,’ and (iii) social forums and messaging that allow consumers to interact with each other and with healthcare professionals.

Our PCHMS aims to provide an integrated platform for consumers to manage their health, in partnership with their health service providers. It seeks to minimize knowledge-based (eg, lack of awareness) and system-based (eg, inconvenience) barriers associated with accessing health services, making consumers more likely to engage in preventive health measures such as influenza vaccination.12

Influenza is an important contributor to loss of workforce productivity and is a significant cause of seasonal mortality (eg, among the elderly13). Influenza vaccination has been estimated to save US$46.85 per person vaccinated,14 which from a population perspective affords substantial community benefit. A meta-analysis of interventions to increase adult vaccination and cancer screening rates concluded that involving patients in self-management through reminders can positively improve uptake.15

The hypotheses tested in the current RCT are (i) that consumers using a PCHMS are more likely to comply with public health recommendations, as measured by rates of seeking and obtaining influenza vaccination, and (ii) that providing online facilities within a PCHMS to schedule encounters with a health service provider will increase the utilization of those services.

Methods

Trial design and participants

We designed a 2010 CONSORT-compliant two-group (6-month waitlist control vs PCHMS) parallel RCT (allocation ratio 1:1).16

Box 1 outlines participant selection criteria. The study recruited 855 students and staff in an Australian university setting from May 2010, where participants were either randomized to the PCHMS, or a wait–list control arm that could use the system after 6-months, when the trial had concluded. Written material advertising the study did not mention influenza, vaccination, or service utilization.

Box 1

Eligibility criteria for participants

Inclusion criteria

  1. Aged 18 or above
  2. Access to the internet, and email at least on a monthly basis

Exclusion criteria

  1. Did not complete the registration process (excluded before randomization)
  2. Self-reported having obtained an influenza vaccination in 2010 prior to enrolment in the study (excluded from analysis at post-study)
  3. Self-reported to be influenced by other participants during the study to obtain (or not obtain) influenza vaccination (excluded from analysis at post-study)

Study protocol

Written informed consent was sought from each participant. Ethics approval was obtained from the UNSW ethics committee.

Students and staff were approached via mailing lists and advertisements in online print publications, which described the study and invited participants to complete an online pre-study survey. Participants randomized to the intervention group then completed a 5-min mandatory online tutorial about Healthy.me prior to using the site. All participants (control and intervention) then received an email in the first week of each month inviting them to complete a 1-min survey (four questions) about influenza-like illness (ILI) symptoms and health activities from May to October 2010. At study completion, all participants received an email asking them to complete a post-study survey (20 questions). Two follow-up emails 5 days apart were sent to non-completers. Those who completed all surveys entered into a draw for one $A500 prize. Supplementary online appendix table A gives the completion rates for each survey.

A researcher was available via a dedicated telephone line and email to answer participant concerns and address any unintended effects during the study. Participants could also provide feedback via the monthly surveys.

Intervention

Healthy.me

The intervention was a web-based PCHMS called Healthy.me (figure 1).11 Central to the system's design are consumer specific care pathways called ‘journeys’ that provide disease or task specific knowledge in an actionable way. A journey can be viewed as a health service engagement protocol for patients and consumers, outlining the steps and activities associated with a specified service or task. For example, at the point that a consumer encounters advice to seek influenza vaccination, they can immediately book an appointment with a doctor from the journey page, or set themselves a reminder to do so. Journeys are computationally active and can personalize other PCHMS elements like the PHR to reflect the specific content of the journey. For example, commencing a vaccination journey can trigger the creation of a vaccination record in the PHR.

Figure 1
Features of Healthy.me (© University of New South Wales, 2009–2011).

Influenza vaccine journey

The influenza vaccine journey in Healthy.me contained two elements:

  • A consumer vaccination care pathway (figure 2), which described (i) the types of influenza vaccine currently available (ii) steps to obtain vaccination at the University Health Service (UHS; the university primary care service) or elsewhere, and (iii) vaccine costs, adverse effects, and contraindications
    Figure 2
    Top page of Healthy.me influenza vaccine journey (© University of New South Wales, 2009–2011).
  • Online appointment booking (figure 3), whereby participants could click a ‘Book now’ button on the journey page, thus sending an email to the UHS to book an appointment for influenza vaccination or other medical issues. A dedicated UHS administrative staff member would telephone participants by the next working day to confirm appointments.
    Figure 3
    Booking an appointment with the University Health Service on Healthy.me (© University of New South Wales, 2009–2011).

The journey was designed in consultation with UHS primary care physicians, utilizing government-endorsed evidence-based consumer education material, and was tested in the previous year for seasonal and pandemic H1N1 influenza.17–20

Outcomes

Table 1 outlines study outcome measures. The protocol and outcomes were not changed after study commencement.

Table 1
Primary and secondary outcome measures collected at different time points

Influenza-like illness case definition

ILI symptoms were based upon case definitions of influenza (fever with cough or a sore throat) issued by the state health department (NSW Health) and the US Centers for Disease Control and Prevention (CDC) as of 26 March 2010.21 22 Febrile upper respiratory tract illnesses occurring during the peak influenza period was identified as the most specific clinical case definition with the highest positive predictive value for true influenza.23

Sample size

Six hundred participants with 300 in each arm were needed to detect a 10% difference in vaccination rate between the waitlist control and the PCHMS groups, calculated at 5% level of significance, 80% power (two-sided test), with an anticipated participant dropout rate of 10%.24 The effect size estimate is based on a review of patient reminder systems showing improved immunization rates of 5%–20%.25 A baseline estimate of influenza vaccination at the UHS showed 16% of patients received FluVax or PanVax between March 5, 2009 and November 5, 2009. The literature reports that 18%–30% of university students and healthy adults (18–49 years old) obtain an influenza vaccination each year.26 27

Randomization

Sequence generation

Eligible consumers were randomly assigned to the PCHMS or waitlist control by a random number sequence, pre-generated externally to the research team with a computerized random-number generator with randomly assigned blocks (block sizes 2, 4, and 8) and an intervention allocation ratio of 1:1.28

Allocation concealment, mechanism, and implementation

Allocation occurred automatically at enrolment. Participants received the next consecutive allocation in the random number sequence, assigning them to the PCHMS or waitlist control.

Blinding

Since a PCHMS is a behavioral intervention, it was not possible to completely blind participants. Group allocation was revealed to participants after obtaining their consent to participate and completion of the pre-study survey. Investigators and clinicians were blinded to group allocation. To minimize contamination of the control group, the intervention group was asked to not share their PCHMS access details with others. Four participants who reported being influenced by other participants were excluded from the analysis.

Statistical methods

Statistical significance was defined a priori as a p value of >0.05 (two-tailed test). Data were collected by online survey software KeySurvey29 and analyzed using PASW Statistics 18.30

Baseline comparisons to assess randomization effectiveness were conducted using the Student t tests and χ2 tests. Adjustments for baseline characteristics and potential confounders were made using sequential logistic regression31 to provide a stratified estimate of intervention effect.32 33 Primary analysis examined differences in the proportion of participants obtaining influenza vaccination during the study in the waitlist and PCHMS groups using the χ2 test, including participants who had the opportunity to use the PCHMS but did not do so. Secondary outcomes were assessed using the χ2 test (table 1). Differences in average number of days of absence per participant were compared using the Student t test.

Results

Participation flow

A total of 855 participants were recruited and 604 followed up between May and October 2010 (figure 4; supplementary online appendix table A). Of 742 participants who met the inclusion criteria, 372 were randomly allocated to the 6-month waitlist and 370 to the PCHMS.

Figure 4
Participant flowchart in the randomized controlled trial. PCHMS, personally controlled health management system.

Baseline data

Randomization resulted in an even distribution of the pre-study baseline measures across the waitlist and PCHMS groups (p>0.05), and at post-study primary analysis (p>0.05) (table 2).

Table 2
Baseline characteristics of study participants eligible for primary analysis in the 6-month waitlist and PCHMS groups

Numbers analyzed

Primary and secondary analyses were conducted on 470 participants who met eligibility criteria. Data for ancillary analyses were available for 86.8% (644/742) of participants who completed at least one monthly survey, and subgroup analyses were conducted on 345 participants who accessed the PCHMS. No significant differences in post-study survey completion rates were found between the waitlist and PCHMS groups (χ2 (1, n=604)=1.6, p=0.207).

Influenza vaccination (primary analysis)

Primary analysis comparing influenza vaccination between the waitlist and PCHMS recipients is presented in Figure 5 and supplementary online appendix table C.

Figure 5
Health activities by study group during the randomized controlled trial. PCHMS, personally controlled health management system group; UHS, University Health Service; Waitlist, group that could use the system after 6 months.

In absolute terms, participants assigned to the PCHMS were 6.7% (95% CI 1.5 to 12.3) more likely than waitlist recipients to receive an influenza vaccine (χ2 (1, n=470)=7.1, p=0.008; waitlist: 4.9% (12/246, 95% CI 2.8 to 8.3) vs PCHMS: 11.6% (26/224, 95% CI 8.0 to 16.5)). Relative to the waitlist control, the proportion of participants receiving an influenza vaccine was 137% higher for the PCHMS group (RR: 2.4 (95% CI 1.2 to 4.6)). Overall, the PCHMS had a small but significant effect (Φ=0.123) on influenza vaccination rates.

Sequential logistic regression assessed whether baseline characteristics (box 1) or other post-study factors (ie, contact with children during the study, past history of influenza vaccine) affected vaccination rate. The final logistic regression model explained between 8.3% (Cox and Snell R square) and 19.3% (Nagelkerke R square) of variance in the uptake of influenza vaccine, and correctly classified 92.1% of influenza vaccination cases. All three independent variables (past history of influenza vaccine, group allocation to PCHMS, and age) made a unique statistically significant contribution to the model (χ2 (3)=40.6, p<0.001) (supplementary online appendix table D).

Health service utilization (secondary analysis)

Secondary analysis of visits to the UHS is outlined in figure 5 and supplementary online appendix table C. Access to PCHMS attracted 11.3% (17/150, 95% CI 7.2 to 17.4%) of participants who were not current patients to visit the service during the study.

In absolute terms, participants assigned to PCHMS were 11.6% (95% CI 3.6 to 19.5) more likely than those assigned to the waitlist control to visit the UHS during the study (χ2 (1, n=470)=8.8, p=0.003; waitlist: 17.9% (44/246, 95% CI 13.6 to 23.2) vs PCHMS: 29.5% (66/224, 95% CI 23.9 to 35.7)). Relative to the waitlist control, the proportion of participants visiting the UHS was 65% higher for the PCHMS group (RR: 1.6 (95% CI 1.2 to 2.3)). Overall, PCHMS use had a small but significant effect (Φ=0.137) on visits to the UHS.

Ancillary analyses

Influenza-like illness symptoms

Cumulative rates of monthly ILI symptoms and health activities from May through October 2010 are summarized in supplementary online appendix table B. No significant differences in ILI symptoms were detected between PCHMS and waitlist participants (p>0.05). During the study, 27.0% (174/644) of participants across the waitlist control and PCHMS groups had ≥1 episode of ILI symptoms (fever/fever-like, with coughs or sore throat), 14.0% (90/644) had ≥1 episode of ILI symptoms with vomiting or diarrhea, and 80.7% (520/644) experienced at least one of these symptoms (fever/fever-like, cough, sore throat, vomiting or diarrhea).

Impact on health activities according to frequency of PCHMS access

There appears to be a dose–response effect associated with the use of the PCHMS on health service engagement behaviors (figure 6, supplementary online appendix table E). Compared to accessing the PCHMS only once, accessing the PCHMS more than once was associated with:

Figure 6
Health activities according to frequency of accessing the personally controlled health management system (PCHMS). PHR, personal health record.
  • Significantly higher use of the PHR to add, update, or delete health data (χ2 (1, n=345)=66.5, p<0.001; used PCHMS once only: 16.5% (41/248); used PCHMS more than once: 60.8% (59/97))
  • Significantly greater use of the online feature to book appointments with the UHS (χ2 (1, n=345)=68.2, p<0.001; used PCHMS once only: 1.2% (3/248); used PCHMS more than once: 29.9% (29/97)).

Among those who also reported their health service usage, accessing the PCHMS more than once was associated with:

  • Significantly higher rates of influenza vaccination (χ2 (1, n=184)=11.7, p=0.001; used PCHMS once only: 8.5% (11/130); used PCHMS more than once: 27.8% (15/54)).
  • Significantly higher rates of visiting the UHS (χ2 (1,n=184)=9.1, p=0.003; used PCHMS once only: 23.8% (31/130); used PCHMS more than once: 46.3% (25/54)).

Reasons for influenza vaccination

Reasons for obtaining (or not obtaining) an influenza vaccine were categorized using past history of influenza vaccination and post-study vaccination status (Past–Post). The most frequently cited reason for vaccination among Yes–Yes participants was ‘don't want to get sick’ (70.8%, 17/24); Yes–No participants: ‘forgot’ (30.9%, 25/81); No–No participants: ‘low-risk or low impact of getting ill’ (27.1%, 95/351); and No–Yes participants: ‘PCHMS’ (57.1%, 8/14) (supplementary online appendix tables F and G).

Harms

No harms or unintended effects were reported by participants during the study. The trial did not end earlier than planned.

Discussion

To our knowledge, this is the first trial of a PCHMS to have shown a statistically significant increase in vaccination rates and health service utilization by consumers. The intervention's effect size (an absolute difference of 6.7% in influenza vaccination rate between PCHMS and waitlist control) is similar to the 5%–20% increase in vaccination rates reported for recall and reminder systems targeted at clinicians.25 It does appear that PCHMS can significantly increase consumer participation in preventive health activities, such as influenza vaccination.

We have previously tested the suitability of our PCHMS to support women undergoing fertility treatment. In that clinical setting, patients were highly motivated to engage in their care, and required complex scheduling of procedures and tests over a relatively short period (eg, see Stone et al15). In contrast, the present study focuses on a preventive clinical setting which has a much lower intensity of activity, where building motivation for engagement with clinical services is one of the challenges. Demonstrating the effectiveness of the PCHMS across these diverse settings and patient groups provides initial evidence that this approach is indeed generalizable and of broad utility.

A previous study using a web-based PHR incorporating reminders, weekly influenza risk maps, and provision of respiratory illness advice, did not report significant improvements in vaccination rates.9 That study did not include online booking, or a consumer specific health service engagement protocol, which are possible reasons why the present intervention was more effective. Online consumer systems in practice offer a ‘bundle’ of e-health services and features well beyond the PHR, and we now need to focus our efforts to identify the right mix of these features that motivate consumer behavior change in different clinical settings.

One approach to bundling e-health services and features is to recognize the ability of a PCHMS to overcome the barriers consumers experience when accessing primary care services.34 This study provides evidence that a PCHMS which (i) addresses knowledge barriers (eg, via disease and task specific service description) and (ii) reduces system barriers (eg, embedding action tools like online booking within these service descriptions) is a promising approach to engage consumers in health service utilization for preventive health activities.

A one-size-fits-all strategy is thus unlikely to result in sustained change in consumer behavior.12 PCHMS service bundles will need to accommodate varying levels of consumer health literacy, motivation, and willingness to engage in preventive behaviors. For example, effective interventions to influence health behaviors in young people will undoubtedly require the use of social networks, social influences, viral communication, and social recommendations,35 36 given the ubiquitous nature of such channels in this population.

In this study, the relative simplicity of the PCHMS design, along with the use of immediately actionable tools (such as the ‘Book now’ button), embedded within consumer specific content (ie, the journey), may account for the improved uptake of influenza vaccination and increased visits to the health service. The Health Belief Model, in particular, discusses how cues in an environment can motivate action.37 Developed in the early 1950s, the HBM remains one of the most widely used and cited conceptual frameworks for understanding why individuals did or did not engage in a variety of health-related actions.37 The HBM uses perceived susceptibility, perceived severity, perceived benefits and perceived barriers to explain and predict individuals' acceptance of health and medical care recommendations.37 With our web-based PCHMS, by providing informational cues that are directly linked to action, we may have overcome some of the perceived barriers that participants experience when deciding to obtain an influenza vaccine, and thus increased the likelihood that consumer intention translated to action.

There are several explanations for the dose–response effect we observed between increased PCHMS access and increased health service utilization. There are two broad causal interpretations—either that increased exposure leads to increased service utilization, or second that those who are high service utilizers are also likely to use the PCHMS more frequently, reflected in higher motivation for engagement.38 39 If increased exposure indeed leads to higher service utilization, then the combination of increased information through content in journeys and the use of actionable tools in the PCHMS, are likely explanations for this effect. However, the dose–response effect between use of web interventions and health behaviors is complex.38 Having more features on a website does not necessarily increase participant engagement, and may sometimes create adherence challenges as more features require more effort from participants.39

Research is now at the stage where we have evidence that web interventions can indeed trigger health behavior changes, but the empirical and theoretical basis for e-health service design is still weak, especially concerning which ‘bundles’ of website features would lead to behavior change. Future studies should employ more theoretical approaches to designing e-health services, recognizing that uptake and outcome changes may be highly dependent on population, disease group, and socio-economic factors.

Cost effectiveness

Not much is currently known about the cost effectiveness of PCHMS as preventive health strategies. Using results from this trial to calculate the underlying return on investment in PCHMS to improve vaccination rates is difficult for a number of reasons. First, the return on investment depends upon an assessment of the costs of deploying and maintaining a system in a working clinical setting, and the current trial uses a research grade system rather than one designed for routine use. Second, a system like this is likely to concurrently support multiple clinical conditions and tasks across a variety of settings, meaning that the cost of system operation would be distributed across all these parallel uses. We would not anticipate a system like ours would be used exclusively to improve vaccination rates.

Limitations

The study relied on self-reports by participants, which has been shown to be acceptably accurate in studies of days of absence,40 influenza symptoms, and vaccination status for diverse patient cohorts.41–46 We minimized the risk of recall bias by conducting short 1-min monthly follow-up surveys during the first week of each month. In addition, we validated influenza vaccination and health service utilization rates by matching self-reports from study participants with their medical records at the UHS (supplementary online appendix table C).

In 2010 the uptake of vaccination and the number of confirmed influenza diagnoses were lower than expected due to a relatively mild winter in Australia,47 and the controversies around the adverse effects of the 2010 seasonal influenza vaccine. It is possible that in a more severe season of influenza, the impact of PCHMS on vaccination rates and health service utilization could be higher than observed in our study. Integrating PCHMS into routine health service delivery systems does appear to be an effective mechanism for enhancing consumer engagement in preventive health measures.

Conclusions

This RCT provides evidence that a web-based PCHMS improves the uptake of influenza vaccination and utilization of health services. The nature of the intervention is sufficiently general that it is likely to be applicable to a wide variety of preventive health tasks, and future work should focus on understanding how to design the right “bundle” of E-Health services and features for different clinical settings and different health tasks.

Worldwide, governments have made multi-billion dollar investment in e-health to modernize health services delivery, with many questions still unanswered about the uptake, benefits, and cost effectiveness of these investments.48 49 Finding approaches that effectively engage consumers in e-health, with minimal attrition rates, remains a high priority.50–52 There is lack of a systematic approach to guide the design of consumer E-Health systems that would encourage the uptake and engagement amongst patients and clinicians. For this reason, unpacking the features and factors that drive effective online engagement remains a crucial area for future research.

Supplementary Material

Supporting Statement:
Supporting Statement:
Supplementary Data:

Acknowledgments

The authors thank Farshid Anvari, Jingbo Liu, Vitaliy Kim, and Jay Liu for their contributions to software development, and the staff and students at UNSW who assisted or participated in the study.

Footnotes

Contributed by

Contributors: Study conceptualization: AL, EC. Study design: AL, EC, JC, VS. Journey design: JC, AL. Data collection: AL, JC. Data analysis: AL, VS, BG. First draft: AL, VS. Draft revision and approval: AL, VS, JC, FM, BG, EC.

Funding: This research is supported in part by grants received from the National Health and Medical Research Council (NHMRC) Centre for Research Excellence (1032664), and the HCF Health and Medical Research Foundation. The funding bodies did not have a role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.

Competing interests: The university and some of the researchers involved in this project could benefit from any commercialization of the personally controlled health management system (PCHMS).

Ethics approval: Ethics approval was provided by the University of New South Wales Human Research Ethics Committee (HREC10109).

Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. Steinbrook R. Personally controlled online health data–the next big thing in medical care? N Engl J Med 2008;358:1653–6. [PubMed]
2. Frost JH, Massagli MP, Wicks P, et al. How the Social Web Supports patient experimentation with a new therapy: the demand for patient-controlled and patient-centered informatics. AMIA Annu Symp Proc 2008:217–21. [PMC free article] [PubMed]
3. Smith CA, Wicks PJ. PatientsLikeMe: consumer health Vocabulary as a Folksonomy. AMIA Annu Symp Proc 2008:682–6. [PMC free article] [PubMed]
4. Wicks P, Massagli M, Frost J, et al. Sharing health data for better outcomes on PatientsLikeMe. J Med Internet Res 2010;12:e19. [PMC free article] [PubMed]
5. Frost JH, Massagli MP. Social uses of personal health information within PatientsLikeMe, an online patient community: what can Happen when patients have access to one Another's data. J Med Internet Res 2008;10:e15. [PMC free article] [PubMed]
6. Archer N, Fevrier-Thomas U, Lokker C, et al. Personal health records: a scoping review. J Am Med Inform Assoc 2011;18:515–22. [PMC free article] [PubMed]
7. Tobacman JK, Kissinger P, Wells M, et al. Implementation of personal health records by case managers in a VAMC general medicine clinic. Patient Educ Couns 2004;54:27–33. [PubMed]
8. Leveille SG, Huang A, Tsai SB, et al. Health coaching via an internet portal for primary care patients with chronic conditions: a randomized controlled trial. Med Care 2009;47:41–7. [PubMed]
9. Bourgeois FT, Simons WW, Olson K, et al. Evaluation of influenza prevention in the workplace using a personally controlled health record: randomized controlled trial. J Med Internet Res 2008;10:e5. [PMC free article] [PubMed]
10. Liu JL, Wyatt JC. The case for randomized controlled trials to assess the impact of clinical information systems. J Am Med Inform Assoc 2011;18:173–80. [PMC free article] [PubMed]
11. Coiera E, Lau AY, Anvari F, et al. Healthy.me: An Online Research Platform to Support Consumer Health Management. Melbourne: Health Informatics Conference (HIC), 2010.
12. Dixon-Woods M, Brown H, Arthur A, et al. Organising services for influenza vaccination for older people. J Health Serv Res Policy 2004;9:85–90. [PubMed]
13. Simonsen L, Reichert TA, Viboud C, et al. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med 2005;165:265–72. [PubMed]
14. Nichol KL, Lind A, Margolis KL, et al. The effectiveness of vaccination against influenza in healthy, working adults. N Engl J Med 1995;333:889–93. [PubMed]
15. Stone EG, Morton SC, Hulscher ME, et al. Interventions that increase use of adult immunization and cancer screening services. Ann Intern Med 2002;136:641–51. [PubMed]
16. A Randomised Controlled Trial to Assess the Effectiveness of an Online Decision Support System on Uptake of Influenza Vaccination and Consumer Healthcare Management (ACTRN12610000386033). Australian New Zealand Clinical Trials Registry (ANZCTR), 2010. http://www.anzctr.org.au/trial_view.aspx?id=335463(accessed 9 May 2011).
17. Influenza. 2011. http://www.health.nsw.gov.au/factsheets/infectious/influenza.html(accessed 30 May 2011).
18. Influenza—The Spread of flu is up to You. 2011. http://www.health.nsw.gov.au/publichealth/infectious/influenza/index.asp(accessed 30 May 2011).
19. What is Influenza? 2011. http://www.flupandemic.gov.au/internet/panflu/publishing.nsf/Content/whatis-1(accessed 30 May 2011).
20. How does a Pandemic Develop? 2011. http://www.flupandemic.gov.au/internet/panflu/publishing.nsf/Content/how-develop-1(accessed 30 May 2011).
21. NSW Health Influenza Case Definition. 2010. http://www.emergency.health.nsw.gov.au/swineflu/professionals/gp.asp(accessed 26 Mar 2010).
22. Novel Influenza A virus Infections—2010 Case Definition. Centers for disease control and prevention (CDC), 2010. http://www.cdc.gov/ncphi/disss/nndss/casedef/novel_influenzaA.htm(accessed 26 Mar 2010).
23. van Hal SJ, Foo H, Blyth CC, et al. Influenza outbreak during Sydney World Youth Day 2008: the utility of laboratory testing and case definitions on mass gathering outbreak containment. PLoS One 2009;4:e6620. [PMC free article] [PubMed]
24. Lwanga SK, Lemeshow S. Sample Size Determination in Health Studies: A Practical Manual. Geneva: World Health Organization, 1991.
25. Szilagyi PG, Bordley C, Vann JC, et al. Effect of patient Reminder/Recall interventions on immunization rates: a review. JAMA 2000;284:1820–7. [PubMed]
26. Johnson DR, Nichol KL, Lipczynski K. Barriers to adult immunization. Am J Med 2008;121(7 Suppl 2):S28–35. [PubMed]
27. Smith NM, Bresee JS, Shay DK, et al. Prevention and control of influenza: recommendations of the Advisory committee on immunization practices (ACIP). MMWR Recomm Rep 2006;55:1–42. [PubMed]
28. Random Number Generator http://www.randomization.com(accessed 24 May 2011).
29. Online Survey Software—KeySurvey http://www.keysurvey.com/(accessed 15 Mar 2011).
30. PASW Statistics 18-Content Guide http://support.spss.com/productsext/statistics/documentation/18/clientindex.html(accessed 15 Mar 2011).
31. Tabachnick BG, Fidell LS. Using Multivariate Statistics. 5th edn Boston, MA: Allyn & Bacon, 2006.
32. Assmann SF, Pocock SJ, Enos LE, et al. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000;355:1064–9. [PubMed]
33. Pocock SJ, Assmann SE, Enos LE, et al. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med 2002;21:2917–30. [PubMed]
34. Harrell E. Do flu vaccines Really work? A Skeptic's view. 2010. http://www.time.com/time/health/article/0, 8599, 1967306,00.html(accessed 30 May 2011).
35. Rao N, Mobius MM, Rosenblat T. Social Networks and Vaccination Decisions. FRB of Boston working paper No. 07-12. SSRN, 2007. http://ssrn.com/abstract=1073143.
36. Wilson SL, Huttlinger K. Pandemic flu knowledge among dormitory housed university students: a need for informal social support and social networking strategies. Rural Remote Health 2010;10:1526. [PubMed]
37. Janz NK, Becker MH. The health Belief model: a decade later. Health Educ Q 1984;11:1–47. [PubMed]
38. Danaher BG, Seeley JR. Methodological issues in research on web-based behavioral interventions. Ann Behav Med 2009;38:28–39. [PubMed]
39. Glasgow RE, Nelson CC, Kearney KA, et al. Reach, engagement, and retention in an Internet-based weight loss program in a multi-site randomized controlled trial. J Med Internet Res 2007;9:e11. [PMC free article] [PubMed]
40. Revicki DA, Irwin D, Reblando J, et al. The accuracy of self-reported disability days. Med Care 1994;32:401–4. [PubMed]
41. Nichol KL, D'Heilly SJ, Greenberg ME, et al. Burden of influenza-like illness and effectiveness of influenza vaccination among working adults aged 50-64 years. Clin Infect Dis 2009;48:292–8. [PubMed]
42. Nichol KL, D'Heilly S, Ehlinger EP. Influenza vaccination among college and university students: impact on influenzalike illness, health care use, and impaired school performance. Arch Pediatr Adolesc Med 2008;162:1113–18. [PubMed]
43. Mac Donald R, Baken L, Nelson A, et al. Validation of self-report of influenza and pneumococcal vaccination status in elderly outpatients. Am J Prev Med 1999;16:173–7. [PubMed]
44. Zimmerman RK, Raymund M, Janosky JE, et al. Sensitivity and specificity of patient self-report of influenza and pneumococcal polysaccharide vaccinations among elderly outpatients in diverse patient care strata. Vaccine 2003;21:1486–91. [PubMed]
45. Carlson SJ, Dalton CB, Tuyl FA, et al. Flutracking surveillance: comparing 2007 New South Wales results with laboratory confirmed influenza notifications. Commun Dis Intell 2009;33:323–7. [PubMed]
46. Carlson SJ, Durrheim DN, Dalton CB. Flutracking provides a measure of field influenza vaccine effectiveness, Australia, 2007-2009. Vaccine 2010;28:6809–10. [PubMed]
47. Dalton C, Durrheim D, Fejsa J, et al. Flutracking: a weekly Australian community online survey of influenza-like illness in 2006, 2007 and 2008. Commun Dis Intell 2009;33:316–22. [PubMed]
48. Greenhalgh T, Stramer K, Bratan T, et al. Adoption and non-adoption of a shared electronic summary record in England: a mixed-method case study. BMJ 2010;340:c3111. [PubMed]
49. Greenhalgh T, Hinder S, Stramer K, et al. Adoption, non-adoption, and abandonment of a personal electronic health record: case study of HealthSpace. BMJ 2010;341:c5814. [PMC free article] [PubMed]
50. Khadjesari Z, Murray E, Kalaitzaki E, et al. Impact and costs of incentives to reduce attrition in online trials: two randomized controlled trials. J Med Internet Res 2011;13:e26. [PMC free article] [PubMed]
51. Eysenbach G. The law of attrition. J Med Internet Res 2005;7:e11. [PMC free article] [PubMed]
52. Poirier J, Cobb NK. Social influence as a driver of engagement in a web-based health intervention. J Med Internet Res 2012;14:e36. [PMC free article] [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of American Medical Informatics Association