Factors affecting home care patients' acceptance of a web-based interactive self-management technology

doi:10.1136/jamia.2010.007336

J Am Med Inform Assoc. 2011 Jan-Feb; 18(1): 51–59.

Published online 2010 December 3. doi: 10.1136/jamia.2010.007336

PMCID: PMC3005875

Factors affecting home care patients' acceptance of a web-based interactive self-management technology

Calvin K L Or,¹ Ben-Tzion Karsh,² Dolores J Severtson,³ Laura J Burke,⁴ Roger L Brown,³ and Patricia Flatley Brennan^2,³

¹Department of Industrial and Manufacturing Systems Engineering, The University of Hong Kong, Pokfulam, Hong Kong

²Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin, USA

³School of Nursing, University of Wisconsin-Madison, Madison, Wisconsin, USA

⁴System Nursing Research and Scientific Support, Aurora Health Care, Milwaukee, Wisconsin, USA

Corresponding author.

Correspondence to Calvin K L Or, Room 8-07, Haking Wong Building, Department of Industrial and Manufacturing Systems Engineering, The University of Hong Kong, Pokfulam, Hong Kong; Email: klor/at/hku.hk

Received July 10, 2010; Accepted November 3, 2010.

See "What's new in informatics" on page 1.

This article has been cited by other articles in PMC.

Abstract

Objective

With the advent of personal health records and other patient-focused health technologies, there is a growing need to better understand factors that contribute to acceptance and use of such innovations. In this study, we employed the Unified Theory of Acceptance and Use of Technology as the basis for determining what predicts patients' acceptance (measured by behavioral intention) and perceived effective use of a web-based, interactive self-management innovation among home care patients.

Design

Cross-sectional secondary analysis of data from a randomized field study evaluating a technology-assisted home care nursing practice with adults with chronic cardiac disease.

Measurement and analysis

A questionnaire was designed based on validated measurement scales from prior research and was completed by 101 participants for measuring the acceptance constructs as part of the parent study protocol. Latent variable modeling with item parceling guided assessment of patients' acceptance.

Results

Perceived usefulness accounted for 53.9% of the variability in behavioral intention, the measure of acceptance. Together, perceived usefulness, health care knowledge, and behavioral intention accounted for 68.5% of the variance in perceived effective use. Perceived ease of use and subjective norm indirectly influenced behavioral intention, through perceived usefulness. Perceived ease of use and subjective norm explained 48% of the total variance in perceived usefulness.

Conclusion

The study demonstrates that perceived usefulness, perceived ease of use, subjective norm, and healthcare knowledge together predict most of the variance in patients' acceptance and self-reported use of the web-based self-management technology.

Introduction

Recent advances in consumer health information technologies (CHITs), which have increasingly gained popularity over the past decade, have complemented traditional healthcare delivery and expanded support for patient self-care. In this study, CHITs refer to patient-focused interactive web- or technology-mediated applications that are designed to improve information access and exchange, enhance decision making, provide social and emotional support, and facilitate behavior changes that promote health and well being¹; examples include CHESS² and ComputerLink.³ CHITs are frequently developed to be used at home by patients for self-care, home care support, or teleconsultation. Such technologies benefit patients by providing them with prompt access to various education, communication, and decision support materials and tools. These readily available resources at the time of need can facilitate the care delivery process, which in turn can lead to better patient health outcomes, well-being, and quality of life, as well as lower hospital readmission rates and mortality rates.^4–10 However, patients do not always accept and use the technologies.^11–16 Lack of CHIT acceptance and subsequent use is a significant concern for patients and healthcare organizations; patients who do not accept or use CHITs will not realize the full benefits of them, and non-acceptance or non-use means a loss of return on investment for the organizations. This makes the study of patient technology acceptance a national priority.

In fact, the U.S. Agency for Healthcare Research and Quality sponsored an Evidence Report to identify drivers and barriers related to successful use of CHITs among the elderly and underserved.¹⁷ The qualitative review of the literature found that: (1) use of CHITs among the elderly, chronically ill, and underserved varied widely and was difficult to assess; (2) ease of use and usefulness of the CHITs were important determinants of use of the technologies; (3) the most frequent barriers were lack of perceived benefit, lack of convenience, and cumbersome data entry; and (4) the most frequent driver of use was the patient's perception of a health benefit. One of the other conclusions was that “in most cases our evidence for usability, barriers, and drivers came from studies where these issues were not a key part of the study design, but rather qualitative evidence that accompanied an outcomes study” (p 54).¹⁷

In another literature review of factors affecting patients' CHIT acceptance, Or and Karsh¹⁸ found that most studies focused on patient-related predictors, such as socio-demographic and health variables. Most of the reviewed studies did not test relevant and reliable predictors of acceptance, such as ease of use and usefulness of CHITs and social influence. The review also found that only a few studies^19–23 were grounded in a theory that could explain technology acceptance, suggesting that model misspecification could be a significant limitation of previous research. The significant lack of theoretically driven empirical models is a concern because it leaves designers and decision makers without clear guidance of what to do to promote patient acceptance of CHIT. This study is an attempt to fill that gap by positing a patient technology acceptance model that draws upon variable categories from an established theory of technology acceptance—the Unified Theory of Acceptance and Use of Technology (UTAUT).²⁴ The UTAUT is a recent extension of the most widely studied theory of technology acceptance, the Technology Acceptance Model,²⁵ ²⁶ which has been used in over a dozen studies modeling healthcare providers' acceptance of health information technology,²⁷ but rarely to model patient acceptance of CHIT. Our purpose in using the UTAUT was to have a principled set of variables from which to predict CHIT acceptance and use. We do not suggest that these variables are the only possible predictors, or that our model is a ‘correct’ model. Rather, we view this as a theoretically supported starting point.

This present study examined acceptance after only 4 weeks of patient use because early stage acceptance has strong bearing on later stage use.²⁸ This study also measured and tested variables outside of the UTAUT that may be especially important for the study population of interest—a home care patient population that includes mostly elderly patients. This study then analyzed the relationships among the variables of the model using data from a larger study²⁹ (hereafter referred to as the HeartCare II study) that examined technology-enhanced nursing practice in home care. While the HeartCare II study focused on a new model of home care nursing practice that capitalized on a web-based technology to support patient self-care and disease management, this present study investigated patients' acceptance of the web technology itself.

Theoretical background and research hypotheses

The patient technology acceptance model that was tested is presented in figure 1. The model relies substantially on the UTAUT, as well as the findings of other related studies.²³ ²⁵ ³⁰ The UTAUT is a well-established model and has been successfully used for analyzing acceptance of computer technologies among healthy individuals, such as general internet users as well as healthcare professionals. Evidence has shown that the UTAUT demonstrated a substantial improvement over the other acceptance models and explained 69% of the variance in behavioral intention,²⁴ which is the most common indicator of acceptance. We therefore measure patient acceptance of CHIT by self-reported behavioral intention. Other extensions of the Technology Acceptance Model have had good success predicting acceptance among healthcare professionals as well.²⁷ Based on the UTAUT, our model posited that four core determinants were associated with acceptance and perceived effective use; they were performance expectancy, effort expectancy, social influence, and facilitating conditions. Figure 1 shows the variables that were used to measure each of those core determinants. Behavioral intention is the measure of acceptance in this present study, and the reasons are as follows: (1) the variable has been widely used to measure individuals' acceptance of (health) technology²³ ²⁴ ²⁶ ^30–34; (2) individuals' formation of an intention to perform a behavior may mean that the individual has accepted the behavior; and (3) behavioral intention considerably predicts actual behavior, such as using a technology.²⁴ ³⁰ ³⁵ ³⁶ Because behavioral intention is believed to predict use behavior, we further measured perceived effective use. Also, we complemented the UTAUT with four patient-centered factors addressing psychomotor, visual, and cognitive aspects, including perceived upper extremity functional ability, perceived visual functional status, health information seeking preference, and healthcare knowledge.

Figure 1

The Patient Technology Acceptance Model. Rectangles represent the core determinants from the Unified Theory of Acceptance and Use of Technology and the circles represent the measured variables representing the core determinants.

Perceived usefulness

Perceived usefulness pertains to performance expectancy in the UTAUT. Based on the original definition by Davis,²⁵ perceived usefulness in this study refers to “the degree to which the participant believes that using the self-management technology would improve his or her ability and enhance the effectiveness in managing his or her disease.” Studies outside and within the consumer health informatics discipline have consistently found that perceived usefulness of a technology is associated with the acceptance of the technology.¹⁷ ¹⁹ ²¹ ²³ ²⁴ ²⁷ ³⁷ ³⁸ This suggests that patients will be more likely to accept the technology if they believe that the technology is useful as it can convey health benefits or facilitate self-management.

Perceived ease of use

Perceived ease of use pertains to effort expectancy in the UTAUT. Based on Davis,²⁵ perceived ease of use in this study refers to “the degree to which a participant believes that using the self-management technology would be free of effort.” CHIT acceptance studies supported that perceived ease of use positively predicted acceptance¹⁹ ^21–23; the effect was also found to be mediated by perceived usefulness. This suggests that patients are more likely to accept a CHIT and perceive the technology as useful if they feel that the technology is easy to use.

Subjective norm

Subjective norm has been frequently used to capture the essence of social influence and refers to one's appraisal of how important others feel about the target behavior.³⁹ In the context of this present study, subjective norm refers to the participant's perceptions that people who are important to him or her think he or she should use the self-management technology. Although the impacts of subjective norm on acceptance are not always consistent, evidence has shown that the factor has been a significant determinant of acceptance within the information systems literature³⁰ ⁴⁰ ⁴¹ as well as in research studying acceptance among healthcare providers, such as physicians.²⁷ We are not aware of any previous studies of patient CHIT acceptance that have examined subjective norm; however, there is merit in exploring the possible effect of subjective norm in this present study.

Perceived behavioral control

Perceived behavioral control, a construct that pertains to facilitating conditions in the UTAUT, reflects individuals' perceptions of internal (ie, self-confidence in his or her ability to perform a given behavior) and external (ie, the availability of resources, such as technical support, needed to perform the behavior) constraints on the behavior of interest, such as CHIT use.²⁴ ³⁰ ³⁹ We are unaware of any previous studies that have examined the impact of perceived behavioral control on patient CHIT acceptance. However, studies of healthcare professionals and studies outside of healthcare have shown that perceived behavioral control significantly predicts acceptance and use of information technology.³⁰ ⁴²

Perceived visual functional status and perceived upper extremity functional ability

Perceived visual functional status and perceived upper extremity functional ability were examined. The CHIT involved in this study was an interactive web-based technology and our participants were home care patients with chronic cardiac disease who were mostly elderly. Due to chronic disabling conditions and ageing, the visual and upper extremity functioning of many elderly patients and patients with chronic cardiac disease is reduced, which leads to their inability to see well and/or to easily manipulate computers.⁴³ Having difficulty seeing text and graphics on a computer screen, using a keyboard, or manipulating a mouse can in turn lead to non-acceptance of technology.^44–46 Psychomotor and visual capacity, therefore, should be components of any user acceptance model, and they are especially important in models attempting to explain technology acceptance of lay people.

Health information seeking preference

Health information seeking preference, which refers to “the degree to which the participant believes that he or she should be told the information about his or her health and the care for the disease,” was included based on Wilson and Lankton.²³ Using CHITs is one way to receive health information since the technologies offer an alternative mechanism for getting prompt lab test results and up-to-date health and treatment knowledge. Therefore, patients who want to receive more information about their condition or treatments may be more likely to accept the technology.

Healthcare knowledge

Healthcare knowledge, which refers to “the amount of knowledge participants feel they have regarding their health condition and care for their disease,” was also examined based on Wilson and Lankton.²³ Patients who think that they have little knowledge about caring for their health problems may be more likely to accept the health technology. This may occur because the participants wish to obtain more relevant health information and supportive resources from the technology to better manage their symptoms and activities.⁴⁷ ⁴⁸

Behavioral intention

Behavioral intention refers to the strength of an individual's intention to perform a behavior.³⁶ It is the most common measure of acceptance, and is the one used in this present study. Previous studies have shown that behavioral intention predicted usage behavior, such as using a technology.²⁴ ²⁶ ³⁰ ³⁴ Moreover, behavioral intention typically fully mediated the effects of individuals' beliefs and perceptions about a technology, such as perceived ease of use and usefulness, on usage behavior.²⁴ ²⁶ ²⁸ ⁴⁰

Methods

Study design and sample

We conducted a secondary analysis of data collected from the HeartCare II study, a randomized field study involving home care patients with chronic cardiac disease, to test the research model and hypotheses. For the HeartCare II study, five offices of a multi-site home healthcare agency in a midwestern state were randomly assigned to the experimental or control condition. The nurses who worked for offices in the control group provided care as usual to their home care patient participants. Nurses who worked for offices in the experimental group provided their patient participants with technology-enhanced care.²⁹ Consenting participants in this group either used their own computers or were provided with free computers and internet connection so that they (or their nurses) could access our specially designed web site for self-management (or nursing practice). The technology acceptance survey questionnaire examining patient participants' perceptions of using the web site for self-management was administered only to the experimental group's participants since those in the control group were not exposed to the technology. To ensure the participants used a consistent referent (ie, using the technology for self-management), they were instructed that the questions were used to ask about “your feeling of using the web site in doing the self-management” before they answered the questions.

Participant inclusion criteria were as follows: (1) admitted for home care registered nursing service by the home care agency; (2) any diagnosis of chronic cardiac disease related to coronary artery disease and/or congestive heart failure; (3) normal vision with corrective lenses; (4) non-cognitively impaired; (5) able to read and write in English; and (6) home dwelling within the service area of the participating home care agency offices. Eligible participants had to have sufficient network connectivity (phone line or cable network) in their home. Exclusion criteria included significant sensory or motor disabilities, mental incapacity, or requiring in-home continuous professional care. Those people were excluded because they were likely to be unable to use the technology intervention. In the parent study, 146 eligible participants were enrolled in the experimental group. This technology acceptance study began data collection about 7 months after the parent study had begun enrollment. At that time, 22 participants had either withdrawn from or completed part or all of the time points of the parent study, and, therefore, only 124 of the 146 experimental participants were involved in this acceptance study. The HeartCare II research and this acceptance study received institutional review board approval and obtained participant informed consent.

Procedure

Following enrollment in the study, participants in the experimental-designated offices were either permitted to use their home computers or provided with a computer terminal and a modem for accessing the technology at no cost if they did not have their own computer. During the first or second week after the technology implementation, participants received training from their visiting nurses on how to use the technology to improve their self-management of their heart condition(s). Participants were also provided with a written manual and a pocket guide illustrating access and use of the system and its features.

All participants completed an assessment for demographic data during a home visit by a research registered nurse at baseline. At the participants' request, data were collected either via telephone or mail survey. For the telephone survey, a trained research assistant administered the survey to participants over the phone. Although CHIT acceptance survey data were collected at 1, 4, 8, 12, and 24 weeks post-implementation, in this paper we examined only cross-sectional data collected at week 4 to examine predictors of early acceptance and use.

The HeartCare web site

The technology used to test acceptance in this study was an interactive web-based health information system (henceforth called the HeartCare web site). The HeartCare web site allowed home care patients with chronic cardiac disease access to the resources for self-care in their homes. The web site was created with a graphical user interface. Participant users needed to use the mouse and keyboard to interact with the system. They were also required to login by typing their user name and password before navigating the web site. The web site offered resources to support three key functions for self-care: health information access, communication, and disease self-monitoring and self-management.

Measurements

We carefully reviewed the literature and chose validated measurement scales to operationalize the theoretical constructs. All measurement items were adjusted to reflect the specific target behavior of interest in this study. Study variables were measured through subscales that were conceptually or semantically congruent. Both intact scales and modifications of existing scales were employed. Modifications, which were based on two rounds of cognitive interviews with senior citizens, included eliminating irrelevant items or rewording existing items for better acceptability to the target population.⁴⁹ A total of 31 items were used to measure the 10 constructs tested in the conceptual model. Items for all constructs, except the perceived upper extremity functional ability and perceived visual functional status, were rated on 7-point Likert-type scales, ranging from “not at all” (0) to “a great deal” (6). Perceived upper extremity function was assessed with four items from the upper extremity subscale of the 32-item Late-Life Function and Disability Instrument.^50–52 The items were phrased, “how much difficulty do you have doing a particular physical activity?” were answered on a 5-point scale with response options of “cannot do” (1), “quite a lot” (2), “some” (3), “a little” (4), and “none” (5).^50–52 Perceived visual functional status was measured using the three-item near vision subscale of the 25-item National Eye Institute Visual Function Questionnaire.⁵³ The items were rated on a 5-point scale, ranging from “no difficulty at all” (1) to “stopped doing this because of your (the participant's) eyesight” (5).⁵³ The overall composite visual scores were calculated based on the standard algorithm.⁵⁴ The scores range from 0 to 100 where a higher score represents less difficulty with near vision activities. The four-item usefulness and four-item ease of use scales from Davis et al²⁶ were adapted to assess perceived usefulness and ease of use, respectively. Adapted from Venkatesh et al²⁴ and Taylor and Todd,³⁰ subjective norm and perceived behavioral control were measured with the three- and four-item scale, respectively. Health information seeking preference and healthcare knowledge were assessed based on the four- and two-item scales, respectively, developed by Wilson and Lankton.²³

Behavioral intention, our measure of acceptance, was assessed with two items drawn from Venkatesh et al.²⁴ Perceived effective use, our measure of use, was measured with one item developed within this study, “To what extent do you use the HeartCare web site as much as you should use it?” Most CHIT studies measure subjective use with frequency measures or simple yes/no measures. However, neither frequency nor duration of use is necessarily relevant to self-care or meaningful. A patient might use a health web site very infrequently and for short durations of time, but get exactly what they need from that web site. Another patient might spend hours searching for information, only to quit disappointed. Subjective measures of frequency or duration would correctly find the later patient used the web site more; but as demonstrated, that may not be the most relevant outcome. Because of such concerns, we used a new ‘effective use’ measure. Even though all measures except for perceived effective use were derived from previously validated scales, all items were still subjected to expert review by two engineering and four nursing researchers as well as two rounds of pilot testing with senior citizens using cognitive interview techniques. Cronbach's α was used to assess the internal consistency of the items for each construct.

Data analysis

Latent variable modeling with item parceling^55–57 was used to determine the significant predictors of CHIT acceptance, starting with the proposed model (figure 1) and to examine the multivariate relationships among the variables using MPlus v 5.⁵⁸ Maximum likelihood estimation was used for model fitting and parameter estimation. Measurement errors were computed for each latent variable and were included in model estimation. As suggested by Bollen⁵⁹ (also see Coffman and MacCallum⁵⁵), measurement errors were set to equal (1−α)×σ², where α is the reliability estimate (Cronbach's α coefficient) of the items and σ² is the variance of the mean scores of the latent construct. Model modification was done using a ‘backward search’ followed by a ‘forward search’ approach.⁶⁰ The backward search continued until no further non-significant path was detected.⁶⁰ ⁶¹ Following that, the forward search identified paths that could be added (ie, reducing constraints) based on the MPlus modification indices.⁵⁸ We intended to only add paths that were theoretically meaningful,⁶⁰ ⁶¹ however no additional paths were suggested for improved fit. Five commonly used goodness-of-fit indices were employed to assess the overall model fit; good fit was indicated by the ratio of χ² to degrees of freedom (χ²/df<3), the comparative fit index (CFI≥0.90), the Tucker–Lewis index (TLI≥0.90), root mean square error of approximation (RMSEA<0.06; although RMSEA values as high as 0.08 are considered acceptable), and standardized root mean square residual (SRMR<0.08).^62–67

Results

Descriptive statistics of the sample

Of the 124 acceptance study's participants, 101 (81%) completed the acceptance survey at week 4. Of the 23 participants who did not complete the survey at week 4, seven were hospitalized, seven did not return the mail survey, six withdrew from the study, one was too ill to complete the survey, one was lost to follow-up, and one refused to complete the survey for personal reasons. Forty-eight of the 101 surveys were through phone interview and 53 were via mail. Participants' ages ranged from 28 to 90 years with a mean of 62.5 (SD 12.3). The sample had 61 males and 40 females. Their education ranged from 3 to 28 years with a mean of 14.1 (SD 3.6). The majority (79.2%) of the participants were Caucasian. Self-rated computer experience was none, beginner, competent, and expert in 14.9%, 32.7%, 40.6%, and 11.9% of the participants, respectively. Slightly more than half (57%) of the sample used their own computers versus computers provided by our project. Cronbach's α's, means, and SDs for the variables, and correlations between the acceptance constructs and behavioral intention and perceived effective use are shown in table 1.

Table 1

Cronbach's alphas, means, SDs, and correlation coefficients

Model testing

The significant predictors are presented in figure 2. All five goodness-of-fit indices indicated that this reduced model fit the data well: χ²/df=1.42; CFI=0.99; TLI=0.98; RMSEA (90% CI)=0.064 (0.00–0.15); and SRMR=0.034. Perceived usefulness accounted for 53.9% of the variability in behavioral intention. The amount of variance in perceived effective use accounted for by perceived usefulness, healthcare knowledge, and behavioral intention was 68.5%. Perceived ease of use and subjective norm explained 48% of the total variance in perceived usefulness.

Figure 2

The final model and parameter estimates of the paths (standardized path coefficients).

In the model, perceived usefulness significantly predicted behavioral intention. There was a significant un-hypothesized finding which emerged; it was found that perceived usefulness significantly directly influenced perceived effective use. Moreover, perceived usefulness had an indirect effect on perceived effective use through behavioral intention (indirect effect=0.34, p<0.001). The direct effect of perceived ease of use on behavioral intention was not significant, but perceived ease of use had an indirect influence on behavioral intention through perceived usefulness (indirect effect=0.41, p<0.001). Subjective norm had no direct effect on behavioral intention. However, subjective norm exerted an indirect influence on behavioral intention through perceived usefulness (indirect effect=0.20, p<0.05). Perceived behavioral control had no significant influence on behavioral intention or perceived effective use. Perceived upper extremity functional ability, perceived visual functional status, health information seeking preference, and healthcare knowledge were not associated with behavioral intention. However, healthcare knowledge significantly predicted perceived effective use. As expected, behavioral intention significantly predicted perceived effective use.

Discussion

This study used the UTAUT as a theoretical basis and integrated variables related to health to examine the acceptance of a web-based, interactive self-management technology among home care patients. The model with significant paths fit the data well and showed the enduring predictive power of the core UTAUT variables—subjective norm, perceived ease of use, perceived usefulness—on acceptance (measured as behavioral intention) and self-reported use (measured as effective use). Consistent with previous CHIT acceptance studies,²⁰ ²¹ ²³ ³⁸ perceived usefulness was the most important factor that explained a significant proportion of the variance in behavioral intention. Patients had stronger intention to use a health technology if they believed that using the technology would enhance their ability and effectiveness in managing their disease. This study also found that perceived usefulness significantly influenced self-reported use, consistent with previous research.³⁷ Patients who felt that the health technology was useful for their disease reported a higher usage of the technology.

Contrary to what was expected, perceived ease of use had no significant direct effect on behavioral intention. Davis et al²⁶ noted that, over time, as users learn to use a technology, ease of use of the technology becomes less salient; thus, the effect of ease of use diminishes. In this study, we examined acceptance 4 weeks post-implementation. Ease of use may not have been a concern as patients may have learned to use the technology during that time; and thus, the effect of ease of use was not found. However, additional research is needed to understand any temporal relationships among ease of use, acceptance, and use with CHITs. Our results, like those of others,²³ ²⁵ showed that patients' perceptions of usefulness, which strongly influenced their intention to use the technology, depended in part on the ease of use of the technology. This demonstrates ease of use is still an important variable for influencing acceptance of CHIT.

Studies on information technology acceptance have shown various results for the relationship between subjective norm and behavioral intention. However, this was the first study examining the relationship within a patient population. Our results match the findings of previous studies, demonstrating that subjective norm has no direct effect on behavioral intention.⁴⁰ ⁶⁸ Nevertheless, subjective norm was still an important factor for patients' CHIT acceptance since our result showed that the factor indirectly influenced behavioral intention through perceived usefulness, consistent with other studies.⁴⁰ ⁶⁹ This indicated that the influences of people who were important to the patients were essential in the development of patients' beliefs about the usefulness of the technology for managing their disease.

Healthcare knowledge had no significant impact on behavioral intention, but it positively predicted perceived effective use. Patients who felt that they were less knowledgeable regarding care for their disease were less likely to use the technology. Perhaps the issue is that low healthcare knowledge lowers patients' self-efficacy for using technology to obtain health information and to manage their health, and therefore lowers their use of the technology. In contrast, patients who have higher healthcare knowledge may benefit more readily from CHIT because they already understand health and can benefit from further information. However, the causal mechanism can only be speculated upon, and additional work is needed to examine the assertion.

Perceived behavioral control was hypothesized to exert direct effects on both behavioral intention and perceived effective use. However, perceived behavioral control had no effect on the variables. Moreover, contrary to what was expected, perceived visual functional status, perceived upper extremity functional ability, and health information seeking preference had no significant influence on behavioral intention. However, the findings should be interpreted with caution. Our data indicated that there was low variability in the responses for those variables (ie, a substantial number of participants responded having good visual ability, high upper extremity functional status, and high preference for health information seeking). This may have occurred because the overall study design excluded those people with significant limitations to their sensory or motor abilities. Also, it is possible that patients who had a high preference for health information seeking were more willing to agree to participate in the overall study. The low variability in these data elements could limit the possibility of finding significant effects for these variables. More research on these variables is needed.

There are a number of limitations in this study. First, the sample size was small. This current study represents a rare attempt to model patient technology acceptance using a theoretically grounded model. However, this study is exploratory. We only studied one group of patients and had a sample size of only 101. We note, however, that Or and Karsh's review of 52 previously published studies of patient acceptance of CHIT only had large samples when the CHIT under study was a health internet web site. On the other hand, the six previous studies of CHIT similar to ours had sample sizes of 12,⁴⁴ 32,²¹ 35,²² 38,⁴⁵ 100,⁷⁰ 107,¹⁹ and 163,²³ making our sample not unusually small. To address this limitation, we had to restrict the number of a priori parameters to estimate in order to maintain a minimum of 5:1 subject-to-parameter ratio.⁷¹ Because of that, potential confounders, which also might have served as moderators, were not included in the model. These variables include age, gender, education, computer experience, and computer ownership. No other studies of CHIT acceptance have assessed those moderators¹⁸ and it is unfortunate our sample size precluded us from doing so. The sample size may also call into question the null results; perhaps with larger samples some of those might have been significant. Nevertheless, the model fit the data well, perceived usefulness accounted for over 50% of the variation in acceptance, and perceived usefulness, healthcare knowledge, and behavioral intention accounted for 68% of the variance in perceived effective use. The UTAUT likely holds promise in explaining significant variation in acceptance and use of CHIT. Second, this study evaluated technology usage behavior using a perceived effective use measure as opposed to objective measures. Objective measures may result in more accurate estimates. However, most previous CHIT acceptance studies have employed self-reported measures,¹⁸ and ample research exists demonstrating that that self-reported measures correlate well with objective measures.³⁰ ⁴⁰ Third, the generalizability of this study should be evaluated in light of the fact that it was conducted on a sample of home care patients with chronic cardiac disease. Fourth, several variables that might help further explain acceptance and use were not explored in this study. Those factors, including computer affect, intrinsic motivation, satisfaction with medical care, computer self-efficacy, and internet skill training,¹⁸ ⁷² ⁷³ may have the potential to determine CHIT acceptance among chronically ill patients and merit future exploration.

Future research in this area should focus on several areas. First, larger samples will be needed to add the variables we could not test, but also to further develop the left side of the model. That is, it will be important to model the determinants of usefulness and ease of use more precisely as those determinants can translate directly into design. Second, moderator analyses will be necessary to tease out the details of how disability, age, gender, and other variables might affect the model. Third, longitudinal models should be explored to determine whether and how the model changes with time.

Conclusions

To conclude, CHITs are expected to bring benefits to their users, such as patients. Evidence shows that CHITs can improve patients' quality of life and well-being, increase medication adherence, facilitate the healthcare delivery process, and support home care patient self-management.^4–6 ⁷⁴ ⁷⁵ However, those benefits will only accrue to patients if the technology is accepted and used. There are many ways in which CHITs can fail to gain users' acceptance. Current patient users of health technology will be less likely to accept the technology when they perceive it as disadvantageous or functionally incompatible with their needs, existing values, or past experiences (ie, systems are not useful).¹² ¹³ Our results suggest that, first and foremost, in order to increase the likelihood that patients will experience the potential benefits of CHITs, the technology must be designed such that system applications are in consonance with patients' needs. This is of course easier said than done, but the results have design and development implications. Based on the items measuring usefulness, the data have suggested that CHITs that are designed to support disease management must improve patients' ability to manage their disease, save them time managing their disease, and make them more effective at managing their disease. Achieving these goals will likely require cognitive analyses, such as cognitive work and task analyses,¹⁵ ^76–78 followed by careful attention to human factors engineering⁷⁹ and cognitive engineering principles for designing automation to support cognitive work (eg, managing a disease).^80–84 Moreover, experiences within and outside of healthcare have advocated active involvement of intended target users in the process of technology design and development (ie, user participation) as one of the principal methodologies for technology designers or health informatics practitioners to address actual user needs and preferences for a new system.⁶⁸ ^85–87 Such analyses and user involvement can ensure that necessary and desired content and functions are present to support disease management. As Demiris et al pointed out, “The targeted users of the system can provide useful feedback… They can point out features they perceived as undesirable for such a system and redirect the design efforts” (p 470).⁸⁷ Applying usability principles is necessary but not sufficient to achieve useful CHIT; cognitive analyses and user involvement provide the ‘what’ to design; usability principles and principles of data representation for problem solving⁸⁸ can provide the ‘how.’

Our study also showed that subjective norm and patients' perception of ease of use significantly determined the perception of usefulness. The subjective norm measure asked the patients the extent to which their nurses and other people important to them wanted them to use the web site. This highlights the need for caregivers and important others, perhaps family members, to be champions of the technology to help increase perceptions of usefulness. The perceived ease of use measure was comprised of items measuring ease of learning as well as ease of manipulating the web site, both of which can be addressed through attention to methods and principles of usability⁸⁹ ⁹⁰ and application of scientific principles of training.⁹¹ ⁹² The literature has emphasized the importance of training for elderly patients for using computer or web-based health technology.⁸⁷ ^93–95 A series of operational and computer/internet skills training sessions may be necessary to improve (elderly) patients' skills and level of confidence in their ability to use technical innovations, which may improve the feelings of ease of use. Proactively considering the issues should help create acceptance for the technology, which in turn should increase the likelihood of technology implementation success.¹⁸ ⁹⁶ ⁹⁷ We believe these factors are especially important for the population we studied, a patient population that includes mostly frail elderly patients, because of their unique design requirements.⁸⁷ ⁹⁸ As has been pointed out, “elderly patients may have difficulty reading electronic sources because of the small size of the font, irrelevant or excessive use of information and graphical representations… Challenges can also arise if the technology is not made with a user-friendly design” (p 205).⁹³ This is especially important for those health informatics practitioners who want to achieve higher usability and accessibility for technology designed for elderly patients. Using a poorly designed web system (eg, overwhelming interface or inadequate feedback) can be a challenge for healthy users. The challenge becomes even larger when a non-user-friendly system is pushed onto the elderly patient users who have functional decline and disability. Designing a system for elderly patient users should aim to improve its interface usability as well as functional and physical accessibility, which can be achieved by applying a specific set of design guidelines and recommendations.⁸⁷ ⁹³ ^98–102

Acknowledgments

The authors thank all of the nurses and patient participants who were involved in the study. The authors also thank Pascale Carayon, Gail Casper, and Marge Sebern for their assistance on the study and helpful comments on the manuscript.

Footnotes

Funding: This study was funded by a grant from the National Library of Medicine (R01-LM6249, ‘HeartCare II-Custom Support for Home Care’).

Competing interests: None.

Ethics approval: This study was conducted with the approval of the institutional review board of the University of Wisconsin-Madison and Aurora Health Care.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. Science Panel on Interactive Communication and Health Wired for health and well-being: the emergence of interactive health communication. In: Eng TR, Gustafson DH, editors. , eds. Washington, DC: US Department of Health and Human Services, U.S. Government Printing Office, 1999.

2. Gustafson DH, Bosworth K, Hawkins RP, et al. CHESS: a computer-based system for providing information, referrals, decision support and social support to people facing medical and other health-related crises. Paper presented at: Annual Symposium on Computer Applications in Medical Care, 1992. Baltimore, MD: American medical Informatics Association.

3. Brennan PF, Ripich S, Moore SM. The use of home-based computers to support persons living with AIDS/ARC. J Community Health Nurs 1991;8:3–14. [PubMed]

4. Eysenbach G. The impact of the Internet on cancer outcomes. CA Cancer J Clin 2003;53:356–71. [PubMed]

5. Gustafson DH, Hawkins R, Boberg E, et al. Impact of a patient-centered, computer-based health information/support system. American Journal of Preventive Medicine 1999;16:1–9. [PubMed]

6. Martinez A, Everss E, Rojo-Alvarez JL, et al. A systematic review of the literature on home monitoring for patients with heart failure. J Telemed Telecare 2006;12:234–41. [PubMed]

7. Scott RL, Park MH, Uber PA, et al. Electronic home monitoring reduces medical resource utilization in chronic heart failure: A prospective investigation of the HomMed-Sentry telemonitoring system. J Card Fail 2004;10:S110–11.

8. Hailey D, Roine R, Ohinmaa A. Systematic review of evidence for the benefits of telemedicine. J Telemed Telecare 2002;8:1–30. [PubMed]

9. Louis AA, Turner T, Gretton M, et al. A systematic review of telemonitoring for the management of heart failure. Eur J Heart Fail 2003;5:583–90. [PubMed]

10. Kashem A, Droogan MT, Santamore WP, et al. Web-based Internet telemedicine management of patients with heart failure. Telemed J E Health 2006;12:439–47. [PubMed]

11. Stoop AP, Van't Riet A, Berg M. Using information technology for patient education: realizing surplus value? Patient Educ Couns 2004;54:187–95. [PubMed]

12. Williams P, Nicholas D, Huntington P. Non use of health information kiosks examined in an information needs context. Health Info Libr J 2003;20:95–103. [PubMed]

13. Rogers A, Mead N. More than technology and access: primary care patients' views on the use and non-use of health information in the Internet age. Health Soc Care Community 2004;12:102–10. [PubMed]

14. Liddy C, Dusseault JJ, Dahrouge S, et al. Telehomecare for patients with multiple chronic illnesses: Pilot study. Can Fam Physician 2008;54:58–65. [PMC free article] [PubMed]

15. Kaufman DR, Patel VL, Hilliman C, et al. Usability in the real world: Assessing medical information technologies in patients' homes. J Biomed Inform 2003;36:45–60. [PubMed]

16. Patterson TL, Shaw WS, Masys DR. Improving health through computer self-help programs: theory and practice. In: Brennan PF, Schneider SJ, Tornquist E, editors. , eds. Information Networks for Community Health. New York: Springer, 1997:219–46.

17. Jimison H, Gorman P, Woods S, et al. Barriers and Drivers of Health Information Technology Use for the Elderly, Chronically Ill, and Underserved. Rockville, MD: Agency for Healthcare Research and Quality, 2008. Evidence Report/Technology Assessment No. 175 (Prepared by the Oregon Evidence-based Practice Center under Contract No. 290-02-0024), AHRQ Publication No. 09-E004.

18. Or CKL, Karsh B. A systematic review of patient acceptance of consumer health information technology. J Am Med Inform Assoc 2009;16:550–60. [PMC free article] [PubMed]

19. Boberg EW, Gustafson DH, Hawkins RP, et al. Development, acceptance, and use patterns of a computer-based education and social support system for people living with AIDS HIV infection. Comput Human Behav 1995;11:289–311.

20. Klein R. Internet-based patient-physician electronic communication applications: patient acceptance and trust. e-Service Journal 2007;5:27–51.

21. Lai TY, Larson EL, Rockoff ML, et al. User acceptance of HIV TIDES–Tailored interventions for management of depressive symptoms in persons living with HIV/AIDS. J Am Med Inform Assoc 2008;15:217–26. [PMC free article] [PubMed]

22. Lu HP, Gustafson DH. An empirical study of perceived usefulness and perceived ease of use on computerized support system use over time. Int J Inf Manage 1994;14:317–29.

23. Wilson EV, Lankton NK. Modeling patients' acceptance of provider-delivered e-health. Journal of the American Medical Informatics Association 2004;11:241–8. [PMC free article] [PubMed]

24. Venkatesh V, Morris MG, Davis GB, et al. User acceptance of information technology: toward a unified view. MIS Quarterly 2003;27:425–78.

25. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 1989;13:319.

26. Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Management Science 1989;35:982–1003.

27. Holden RJ, Karsh B. The technology acceptance model: its past and its future in healthcare. J Biomed Inform 2010;43:159–72. [PMC free article] [PubMed]

28. Venkatesh V, Speier C, Morris MG. User acceptance enablers in individual decision making about technology: toward an integrated model. Decision Sciences 2002;33:297–316.

29. Brennan PF, Casper GR, Kossman S, et al. HeartCareII: Home care support for patients with chronic cardiac disease. Medinfo 2007;129:988–92. [PubMed]

30. Taylor S, Todd PA. Understanding information technology usage: a test of competing models. Information Systems Research 1995;6:144–76.

31. Hu PJ, Chau PYK, Sheng ORL, et al. Examining the technology acceptance model using physician acceptance of telemedicine technology. Journal of Management Information Systems 1999;16:91–112.

32. Legris P, Ingham J, Collerette P. Why do people use information technology? A critical review of the technology acceptance model. Inf Manage 2003;40:191–204.

33. Wu JH, Shen WS, Lin LM, et al. Testing the technology acceptance model for evaluating healthcare professionals' intention to use an adverse event reporting system. Int J Qual Health Care 2008;20:123–9. [PubMed]

34. Sheppard BH, Hartwick J, Warshaw PR. The Theory of Reasoned Action: A meta-analysis of past research with recommendations for modifications and future research. J Consum Res 1988;15:325–43.

35. Ajzen I, Fishbein M. Understanding Attitudes and Predicting Social Behavior. Englewood Cliffs, NJ: Prentice-Hall, 1980.

36. Fishbein M, Ajzen I. Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley, 1975.

37. Diaz JA, Griffith RA, Ng JJ, et al. Patients' use of the Internet for medical information. J Gen Intern Med 2002;17:180–5. [PMC free article] [PubMed]

38. Mead N, Varnam R, Rogers A, et al. What predicts patients' interest in the Internet as a health resource in primary care in England? J Health Serv Res Policy 2003;8:33–9. [PubMed]

39. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process 1991;50:179–211.

40. Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: four longitudinal field studies. Management Science 2000;46:186–204.

41. Moore GC, Benbasat I. Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research 1991;2:192–222.

42. Mathieson K. Predicting user intentions: comparing the technology acceptance model with the theory of planned behavior. Information Systems Research 1991;2:173–91.

43. Hermanova H. New challenges in ageing: home care. Disabil Rehabil 1997;19:142–6. [PubMed]

44. Finkelstein J, Khare R, Ansell J. Feasibility and patients' acceptance of home automated telemanagement of oral anticoagulation therapy. Paper presented at: Annual Symposium of American Medical Informatics Association, 2003. Washington, DC: American medical Informatics Association.

45. Lober WB, Zierler B, Herbaugh A, et al. Barriers to the use of a personal health record by an elderly population. Paper presented at: Annual Symposium of American Medical Informatics Association, 2006. Washington, DC: American medical Informatics Association.

46. Tak SH, Hong SH. Use of the Internet for health information by older adults with arthritis. Orthop Nurs 2005;24:134–8. [PubMed]

47. Ferguson T. Online patient-helpers and physicians working together: a new partnership for high quality health care. Br Med J 2000;321:1129–32. [PMC free article] [PubMed]

48. Fowles JB, Kind AC, Craft C, et al. Patients' interest in reading their medical record: relation with clinical and sociodemographic characteristics and patients' approach to health care. Arch Intern Med 2004;164:793–800. [PubMed]

49. Or CKL, Severtson DJ, Karsh B, et al. Development of an instrument to measure technology acceptance among homecare patients with heart disease. Paper presented at: Annual Symposium of American Medical Informatics Association, 2006. Washington, DC: American medical Informatics Association.

50. Haley SM, Jette AM, Coster WJ, et al. Late life function and disability instrument: II. Development and evaluation of the function component. J Gerontology A Biol Sci Med Sci 2002;57:M217–22. [PubMed]

51. Sayers SP, Jette AM, Haley SM, et al. Validation of the late-life function and disability instrument. J Am Geriatr Soc 2004;52:1554–9. [PubMed]

52. Haley SM, Ludlow LH, Kooyoomjian JT. Extending the range of functional assessment in older adults: development of the late-life function and disability instrument. J Aging Phys Act 2002;10:453–65.

53. Mangione CM, Lee PP, Gutierrez PR, et al. Development of the 25-item national eye institute visual function questionnaire (VFQ-25). Arch Ophthalmol 2001;119:1050–8. [PubMed]

54. Mangione CM. The National Eye Institute 25-item Visual Function Questionnaire (VFQ-25)–Scoring algorithm. 2000:1–15 http://www.rand.org/health/surveys_tools/vfq/vfq25_manual.pdf (accessed 22 November 2010).

55. Coffman DL, MacCallum RC. Using parcels to convert path analysis models into latent variable models. Multivariate Behavioral Research 2005;40:235–59.

56. Cattell RB, Burdsal CA. The radial parcel double factoring design: a solution to the item-vs-parcel controversy. Multivariate Behavioral Research 1975;10:165–79.

57. Little TD, Cunningham WA, Shahar G, et al. To parcel or not to parcel: exploring the question, weighing the merits. Structural Equation Modeling 2002;9:151–73.

58. Muthén LK, Muthén BO. Mplus User's Guide. 5th edn Los Angeles, CA: Muthén & Muthén, 1998–2007.

59. Bollen KA. Structural Equations with Latent Variables. New York: Wiley, 1989.

60. Chou CP, Bentler PM. Model modification in structural equation modeling by imposing constraints. Comput Stat Data Anal 2002;41:271–87.

61. MacCallum RC. Specification searches in covariance structure modeling. Psychol Bull 1986;100:107–20.

62. Kline RB. Principles and Practice of Structural Equation Modeling. 2nd edn New York: The Guilford Press, 2005.

63. Steiger JH. Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research 1990;25:173–80.

64. Bentler PM. Comparative fit indexes in structural models. Psychol Bull 1990;107:238–46. [PubMed]

65. Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika 1973;38:1–10.

66. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling 1999;6:1–55.

67. MacCallum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychol Methods 1996;1:130–49.

68. Hartwick J, Barki H. Explaining the role of user participation in information system use. Management Science 1994;40:440–65.

69. Horst M, Kuttschreuter M, Gutteling JM. Perceived usefulness, personal experiences, risk perception and trust as determinants of adoption of e-government services in The Netherlands. Computers in Human Behavior 2007;23:1838–52.

70. Dugaw JE, Civello K, Chuinard C, et al. Will patients use a computer to give a medical history? J Fam Pract 2000;49:921–3. [PubMed]

71. Bentler PM, Chou CP. Practical issues in structural modeling. Sociol Methods Res 1987;16:78–117.

72. Davis FD, Bagozzi RP, Warshaw PR. Extrinsic and intrinsic motivation to use computers in the workplace. J Appl Soc Psychol 1992;22:1111–32.

73. Igbaria M, Guimaraes T, Davis GB. Testing the determinants of microcomputer usage via a structural equation model. Journal of Management Information Systems 1995;11:87–114.

74. Friedman RH, Kazis LE, Jette A, et al. A telecommunications system for monitoring and counseling patients with hypertension. Impact on medication adherence and blood pressure control. Am J Hypertens 1996;9:285–92. [PubMed]

75. Leirer VO, Morrow DG, Tanke ED, et al. Elders' nonadherence: its assessment and medication reminding by voice mail. Gerontologist 1991;31:514–20. [PubMed]

76. Schraagen JM, Chipman SF, Shalin VL. Cognitive Task Analysis. Mahwah, NJ: Lawrence Arlbaum, 2000.

77. Vicente K. Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work. Mahwah, NJ: Lawrence Erlbaum, 1999.

78. Weir CR, Nebeker JJR, Hicken BL, et al. A cognitive task analysis of information management strategies in a computerized provider order entry environment. Journal of the American Medical Informatics Association 2007;14:65–75. [PMC free article] [PubMed]

79. Karsh B, Holden RJ, Alper SJ, et al. A human factors engineering paradigm for patient safety: designing to support the performance of the healthcare professional. Qual Saf Health Care 2006;15:I59–65. [PMC free article] [PubMed]

80. Hollnagel E, Woods DD. Joint Cognitive Systems: Foundations of Cognitive Systems Engineering. New York: CRC Press, 2005.

81. Littner M, Hirshkowitz M, Davila D, et al. Practice parameters for the use of auto-titrating continuous positive airway pressure devices for titrating pressures and treating adult patients with obstructive sleep apnea syndrome. Sleep 2002;25:143–7. [PubMed]

82. Woods DD. Designs are hypotheses about how artifacts shape cognition and collaboration. Ergonomics 1998;41:168–73.

83. Hollnagel E. Cognitive ergonomics: it's all in the mind. Ergonomics 1997;40:1170–82.

84. Woods DD, Roth EM. Cognitive engineering—human problem-solving with tools. Hum Factors 1988;30:415–30.

85. Holden RJ, Or CKL, Alper SJ, et al. A change management framework for macroergonomic field research. Appl Ergon 2008;39:459–74. [PubMed]

86. Lorenzi NM, Riley RT. Preparing the Staff for New Technologies. Organizational Aspects of Health Informatics: Managing Technological Change. New York: Springer-Verlag, 1995.

87. Demiris G, Finkelstein SM, Speedie SM. Considerations for the design of a web-based clinical monitoring and educational system for elderly patients. Journal of the American Medical Informatics Association 2001;8:468–72. [PMC free article] [PubMed]

88. Carayon P, Hundt AS, Karsh B, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006;15(Suppl I):i50–8. [PMC free article] [PubMed]

89. Nielsen J. Usability Engineering. Boston, MA: Academic Press, 1993.

90. Rubin JR. Handbook of Usability Testing. New York, NY: John Wiley & Sons, 1994.

91. Cannon-Bowers JA, Salas E, Tannenbaum SI, et al. Toward theoretically based principles of training effectiveness: a model and initial empirical investigation. Mil Psychol 1995;7:141–64.

92. Salas E, Cannon-Bowers JA. The science of training: a decade of progress. Annu Rev Psychol 2001;52:471–99. [PubMed]

93. Or CK, Valdez RS, Casper GR, et al. Human factors and ergonomics in home care: current concerns and future considerations for health information technology. Work 2009;33:201–9. [PMC free article] [PubMed]

94. Kelley CL, Charness N. Issues in training older adults to use computers. Behav Inf Technol 1995;14:107–20.

95. Fisk AD, Rogers WA. Health care of older adults: the promise of human factors research. In: Rogers WA, Fisk AD, editors. , eds. Human Factors Interventions for the Health Care of Older Adults. Mahwah, NJ: Erlbaum, 2001:1–12.

96. Karsh B. Beyond usability: designing effective technology implementation systems to promote patient safety. Qual Saf Health Care 2004;13:388–94. [PMC free article] [PubMed]

97. Holden RJ, Karsh B. A theoretical model of health information technology behavior. Behav Inf Technol 2009;28:21–38.

98. Rogers WA, Fisk AD. Human Factors Interventions for the Health Care of Older Adults. Mahwah, NJ: Erlbaum, 2001.

99. Vanderheiden GC. Design for people with functional limitations. In: Salvendy G, editor. , ed. Handbook of Human Factors and Ergonomics. 3rd edn New York: John Wiley & Sons, 2006: 1387–417.

100. Nichols TA, Rogers WA, Fisk AD. Design for aging. In: Salvendy G, editor. , ed. Handbook of Human Factors and Ergonomics. 3rd edn New York: John Wiley & Sons, 2006: 1418–45.

101. Czaja SJ, Lee CC. Information technology and older adults. In: Jacko JA, Sears A, editors. , eds. Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. 2nd edn New York: Lawrence Erlbaum Associates, 2007:777–93.

102. Eysenbach G. Design and evaluation of consumer health information web sites. In: Lewis D, Eysenbach G, Kukafka R, et al., editors. , eds. Consumer Health Informatics: Informing Consumers and Improving Health Care. New York: Springer-Verlag, 2005:34–60.

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of
American Medical Informatics Association