Feasibility of an eHealth Service to Support Collaborative Depression Care: Results of a Pilot Study

doi:10.2196/jmir.1510

J Med Internet Res. 2010 Oct-Dec; 12(5): e63.

Published online 2010 December 19. doi: 10.2196/jmir.1510

PMCID: PMC3057312

Copyright ©Matic Meglic, Mirjana Furlan, Marja Kuzmanic, Dejan Kozel, Dusan Baraga, Irma Kuhar, Branko Kosir, Rade Iljaz, Brigita Novak-Sarotar, Mojca Zvezdana Dernovsek, Andrej Marusic, Gunther Eysenbach, Andrej Brodnik. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 19.12.2010

Feasibility of an eHealth Service to Support Collaborative Depression Care: Results of a Pilot Study

Reviewed by Peter Splunteren, Hedda Land, and Jeannet Kramer

Matic Meglic, MD PhD MBA,¹ Mirjana Furlan, MD,² Marja Kuzmanic, MPhil,¹ Dejan Kozel, BSc,¹ Dusan Baraga, MD,³ Irma Kuhar, MD,⁴ Branko Kosir, MD,⁵ Rade Iljaz, MD,⁶ Brigita Novak Sarotar, MD PhD,⁷ Mojca Zvezdana Dernovsek, MD PhD,⁷ Andrej Marusic, MD PhD,¹ Gunther Eysenbach, MD MPH,^8,⁹ and Andrej Brodnik, PhD¹

⁹Centre for Global eHealth Innovation, University Health Network, Toronto, ON, Canada

⁸Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada

⁷University Psychiatric Hospital Ljubljana, Ljubljana, Slovenia

⁶Primary Health Care Centre Brezice, Brezice, Slovenia

⁵Ambulanta Kosir d.o.o., Skofja Loka, Slovenia

⁴Psihiatricna Ambulanta Vrhnika, Vrhnika, Slovenia

³Primary Health Care Centre Cerknica, Cerknica, Slovenia

²Health Care Centre Celjenje, Koper, Slovenia

¹Primorska Institute of Nature Science and Technology, University of Primorska, Koper, Slovenia

Matic Meglic, Primorska Institute of Nature Science and Technology, University of Primorska, Muzejski trg 2, Koper, 6000, Slovenia, Phone: 386 56117591, Fax: 386 56117592, Email: matic.meglic/at/upr.si.

Monitoring Editor: Maria Riper

Corresponding author.

Received February 2, 2010; Revisions requested February 26, 2010; Revised June 8, 2010; Accepted June 9, 2010.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

This article has been cited by other articles in PMC.

Abstract

Background

Treatments and organizational changes supported by eHealth are beginning to play an important role in improving disease treatment outcome and providing cost-efficient care management. “Improvehealth.eu” is a novel eHealth service to support the treatment of patients with depressive disorder. It offers active patient engagement and collaborative care management by combining Web- and mobile-based information and communication technology systems and access to care managers.

Objectives

Our objective was to assess the feasibility of a novel eHealth service.

Methods

The intervention—the “Improvehealth.eu” service—was explored in the course of a pilot study comparing two groups of patients receiving treatment as usual and treatment as usual with eHealth intervention. We compared patients’ medication adherence and outcome measures between both groups and additionally explored usage and overall perceptions of the intervention in intervention group.

Results

The intervention was successfully implemented in a pilot with 46 patients, of whom 40 were female. Of the 46 patients, 25 received treatment as usual, and 21 received the intervention in addition to treatment as usual. A total of 55% (12/25) of patients in the former group and 45% (10/21) in the latter group finished the 6-month pilot. Available case analysis indicated an improvement of adherence in the intervention group (odds ratio [OR] = 10.0, P = .03). Intention-to-treat analysis indicated an improvement of outcome in the intervention group (ORs ranging from 0.35 to 18; P values ranging from .003 to .20), but confidence intervals were large due to small sample sizes. Average duration of use of the intervention was 107 days. The intervention was well received by 81% (17/21) of patients who reported feeling actively engaged, in control of their disease, and that they had access to a high level of information. In all, 33% (7/21) of the patients also described drawbacks of the intervention, mostly related to usability issues.

Conclusions

The results of this pilot study indicate that the intervention was well accepted and helped the patients in the course of treatment. The results also suggest the potential of the intervention to improve both medication adherence and outcome measures of treatment, including reduction of depression severity and patients becoming “healthy.”

Keywords: Depression, patient care management, information systems, Internet, treatment outcome, medication adherence, pilot study, feasibility study, collaborative care

Introduction

Depressive disorders are the second leading cause of disability worldwide with prevalence ranging from 16% to 18% during the entire life span [1]. The majority of people with depression are treated in primary health care [2,3]. It has been shown that treatment of depression in the primary care setting is far from optimal [4-6].

To improve the outcome of depression treatment, we need to improve patient adherence to therapy [7-9] and the care process itself using, for example, collaborative care, which is characterized by enhanced collaboration between the patient and health care professionals involved in the treatment process [10-12]. Glied has shown that particularly in collaborative care it appears possible to sustain net benefits using less costly interventions [10]. The question we asked was: Can we develop eHealth interventions to treat depression in which the net benefits are sustained while further reducing resource utilization and cost?

New eHealth tools and interventions promise to provide care process support (helping patients and health care professionals to comply with the defined care process with less effort) and to actively engage the patients, thus reducing resource usage [13,14]. Online self-treatment interventions have already proven their clinical value [15], and literature also describes eHealth solutions to support collaborative care in depression treatment [16] including eHealth solutions that have already demonstrated significant improvement of outcomes [17,18].

An eHealth system for active patient engagement and care management, called RecoveryRoad, has been described by Robertson [19]. Its features included secure e-consultations, progress-monitoring questionnaires, psycho-education, and evidence-based therapy. It also offered access to patients’ data, automated reminders for patients, and support for case management. In reports of preliminary findings, Robertson described high adherence to the system (53% to 84%) and self-reported medication adherence (over 90%) with a large effect size (Cohen’s d = 1.0) on average depression severity decline [19].

In this paper, we report the results of a pilot study to assess the feasibility of a novel eHealth intervention to support treatment of patients with depression.

Methods

Improvehealth.eu Intervention

The intervention, “Improvehealth.eu” service [20], consisted of (1) a Web-based information and communication technology system, referred to as “the ICT system,” designed to support collaborative care management and active patient engagement, and (2) online and phone-based care management performed by trained psychologists.

The intervention was administered via the Internet (accessible using personal computers and smart phones) and mobile phones. The ICT system, available 24/7, aimed to (1) actively engage the patients in the process of care; (2) increase the availability of information to all involved health care professionals (psychologists/care managers, general physicians, and psychiatrists); (3) automatically detect patient issues like poor or missing treatment response, unwanted side effects, emergence of suicidality, and nonadherence to medication regimens; and (4) provide timely response by care managers [21]. Care managers were available by telephone during service hours (3 hours per day on workdays), and their email response time was not longer than 2 working days. Upon starting the intervention, patients were informed that in case of imminent suicidality outside the intervention’s service hours they were to contact existing urgent psychiatric care providers. Care managers reported all patient-related activities in the ICT system, and physicians were asked to do the same.

Patients were actively engaged by submitting self-reporting questionnaires on symptoms and drug therapy side effects at least once per week in the acute phase (lasting from week 0 to week 9 after the start of therapy) and at least once per month in the continuation phase (lasting from week 10 to week 23). Submitted questionnaires provided real-time evaluation data for the ICT system and care managers.

The ICT system defined and assigned the tasks in the care process automatically for each patient. Some administrative and clinical tasks with well-defined trigger rules were performed by the ICT system in an automated way using rule sets and an evaluation matrix [21,22]. These tasks included (1) sending reminder text messages to patients and/or care managers if patients forgot to submit questionnaires on due dates, (2) quantitatively analyzing submitted questionnaires, and (3) creating questionnaire-related tasks for care managers (ie, to call a patient who had discontinued treatment).

The ICT system could only be accessed using secure hypertext transfer protocol (HTTPS) and digital certificates. All data were stored on the University of Primorska server in an encrypted file system. Access of health care professionals to data was only granted on patient consent.

Generic functionalities available to all users (patients, care managers, and care providers) included a personal calendar; an internal messaging system (a system of mailboxes); a forum as well as questions and answers; personal profile settings; information on depressive disorder, treatment, and emergency facilities; and the ICT system instructions.

Additional functionalities of the ICT system were available to patients, care managers, and care providers. These are shown in Table 1. A schematic description of functionalities available to patients is given in Figure 1 (not all are shown). In addition, a screenshot of “Improvehealth.eu” in which a patient is submitting a questionnaire is depicted in Figure 2.

Table 1

Additional functionalities of the ICT system available to patients, care providers, and care managers

Figure 1

Simplified patient view of the intervention in which arrows describe the direction of information flow, boxes represent ICT system functionalities, and icons above arrows denote available channels (personal computer, mobile phone)

Figure 2

Screenshot of “Improvehealth.eu” in which a patient is submitting a questionnaire (the red semibrackets indicate particular modules, explained in boxes on the right hand side)

Study Design

The pilot study to explore the feasibility of the intervention was approved by the Slovene National Medical Ethics Committee. Inclusion criteria were: a diagnosis of depression (ICD10 group F32) or mixed anxiety and depression disorder (ICD10 code F41.2) for the first time or after a remission of at least 6 months; introduction of antidepressant treatment in the last 10 days; regular use of Internet and mobile phone; 14 or more points on Beck Depression Inventory-II (BDI-II) questionnaire [23,24].

The control group received treatment as usual, that is, physician visits and antidepressant treatment. The intervention group received the intervention “Improvehealth.eu” service as an addition to treatment as usual. Systematic alternating order (unweighted even-odd distribution) without blinding was used to assign patients to the control group or to the intervention group [25].

After giving informed consent, the patients filled in 2 paper questionnaires, one at the beginning of the pilot (referred to as Time 0) and the other after 24 weeks, that is, at the end of the pilot (referred to as Time 1). The Time 0 assessment consisted of a questionnaire assessing demographics and BDI-II [23,24]. The Time 1 assessment consisted of a repeated BDI-II assessment and a 26-question questionnaire exploring the duration of antidepressant therapy, side effects, adherence, patients’ perceived quality of care, and, for the intervention group only, the overall perception of the intervention.

Patients were recruited and enrolled by 7 physicians upon initial assessment. The observation period for each individual patient was 6 months. Primary outcome measures were patients’ medication adherence and clinical outcome measures (reduction in depression severity according to BDI-II and reaching “healthy” criteria, described below). User acceptance and usage patterns were explored as secondary outcome measures. No incentives were offered to patients for finishing the pilot.

Demographics and Pilot Participation

Demographics included age, gender, marital status, education, and employment. Attrition rate was measured as the share of patients responding to the Time 1 questionnaire.

Medication Adherence, BDI-II Improvement, and Outcome Measures

For self-assessment of medication adherence at Time 1, we used a questionnaire combining 3 previously reported measures: (1) regularity of administration over the defined medication period, (2) taking the medication at the same time of the day, and (3) regular use of correct dosage [26-28]. “Adherent” was defined as adherent to 2 or 3 of these criteria.

To assess reduction in depression severity, we calculated the difference between patient-reported Beck Depression Inventory-II (BDI-II) scores at Time 0 and Time 1 [23,24]. To assess clinically important change (the patient becoming “healthy”), we used a combination of BDI-II score of less than 14 points at Time 1 and at least 8 points improvement in BDI-II score from Time 0 to Time 1 as suggested in previous research (rounded from 14.29 and 8.46, respectively) [29-31].

Usage Patterns

Additional data were acquired from the database to explore the duration and frequency of intervention usage and workload on care managers, including time between registration and last submitted questionnaire, number of submitted questionnaires, and number of tasks performed by care managers.

Patient Feedback

For qualitative assessment, the patient questionnaires at Time 1 included open-ended questions on overall satisfaction with the intervention. They also included 12 Likert-type items (statements) on patient perception of care quality, access to care, and access to information.

User Experience With the ICT System

Initial usability testing of the ICT system was performed before the pilot by 6 healthy individuals and 1 usability expert. Users had to perform tasks like registering, filling in the questionnaire etc. Usability issues regarding the ICT system arising during the pilot were reported and listed as such.

Statistical Analysis

Choice of tests was dependant on variable type. Two-sided significance testing was used in all cases. To compare demographic characteristics of the two groups we used the Fisher exact test, the Mann-Whitney test, and the chi-square test. For adherence, Mantel-Haenszel odds ratio estimate was used. For BDI-II improvements, we performed available case analysis using paired and unpaired t tests and Cohen’s d for effect size. For outcome measures, we used the Fisher Exact test and odds ratio estimates.

We employed a simple sensitivity analysis to assess variability due to dropouts’ missing data: available case and intention-to-treat analyses were performed, the latter using simple imputation scenarios [32]. These scenarios were: (1) “healthy” (ie, an assumed BDI-II score of less than 14 and symptom reduction of at least 8) early quitters in the intervention group were also healthy at Time 1, and the average frequency of healthy dropouts in the control group was the same as in the intervention group; (2) all drop-outs in either group were not healthy; or (3) all dropouts in both groups were healthy. These 3 scenarios imputed the same risks in both groups, pulling effect estimates towards the null hypothesis [32], thus avoiding the overestimation of intervention effect. We also added a pessimistic scenario (4) in which all missing patients in the intervention group were imputed as “not healthy,” whereas in the control group they were imputed as “healthy.”

Usage patterns were assessed using Kaplan-Meyer analysis. The 2 scenarios used were (1) treating all patients as events and (2) treating “healthy” quitters (patients quitting with last reported BDI-II values reaching “healthy” criteria) as censored events. Care manager usage patterns were listed depending on classification of tasks.

For patient feedback, the qualitative answers were categorized and compared with the aims of the intervention, whereas Likert-type items were analyzed using the Mann-Whitney test.

Results

Demographics and Pilot Participation

There were no significant differences between the intervention and control groups at the beginning of pilot (Time 0 in Table 2) and after 6 months (Time 1) for age, gender, marital status, education, and employment. Of the 46 patients, 25 (54%) were allocated to control group and 21 (46%) to intervention group. The response rate at 6 months (Time 1) was slightly higher in the intervention group (12 out of 21, 57%) versus the control group (10 out of 25, 40%), but the difference was not statistically significant (χ² ₁ = 1.34, P = .38).

Table 2

Group characteristics at Time 0

In the intervention group, the reasons patients gave for dropping out were dissatisfaction with the intervention (1 patient) and early significant clinical improvement (8 patients). For those who said they dropped out because they were much improved, it is unknown whether they were still healthy at Time 1. Reasons for high attrition at Time 1 in the control group were not known.

Medication Adherence, BDI-II Improvement, and Outcome Measures

In the control group, 3 out of 9 (33%) patients were adherent to antidepressants compared with 10 out of 12 patients (83%) in the intervention group (χ² ₁ = 5.45, P = .03, odds ratio [OR] = 10.0, 95% confidence interval [CI] = 1.28-78.1).

Both groups demonstrated significant within-group reduction of mean BDI-II score from Time 0 to Time 1 (control group: paired t ₉ = 3.95, P = .003, Cohen’s d = 1.23; intervention group: paired t ₁₁ = 7.23, P < .001, Cohen’s d = 2.57), with intervention group seeming to indicate a greater effect size, which is further supported by the between-group comparison shown in Table 3.

Table 3

Between-group comparison of BDI-II for available cases

In outcome sensitivity analysis (Table 4), available case analysis and the 3 intention-to-treat (ITT) scenarios with equal risk imputation [32] resulted in odds ratios in favour of intervention, seeming to indicate an improvement of outcome in the intervention group. The last, pessimistic scenario was insignificantly in favour of the control group. Confidence intervals were wide due to small sample sizes and high dropout ratios.

Table 4

Outcome measures: available cases and intention-to-treat (ITT) analysis scenarios

Usage Patterns

In Figure 3, the Kaplan-Meyer plot depicts the chance of a patient reaching a certain duration of intervention usage. Shown are 2 scenarios: (1) all patients and (2) only patients in need of further intervention where only nonhealthy patients at any given time were taken into account (by treating healthy quitters as censored events). The mean duration of intervention usage by all patients was 107 days (95% CI 90-125 days); for patients in need of further intervention the mean duration was 150 days (95% CI 131-170 days).

Figure 3

Kaplan-Meier survival analysis for use of intervention (dotted vertical line denotes Time 1 at beginning of the 24th week)

The patients submitted a total of 431 questionnaires online of which 198 (46%) were complete. The average number of complete questionnaires per patient was 9.9 (SD 3.35, range 3-14). The remaining 229 submitted questionnaires were only partially completed and were treated as unsuccessful submissions. The patients were required to fill in the missing answers and resubmit the questionnaires.

Of the 21 patients in the intervention group, 6 (29%) required guided registration over the phone by the care manager. Care managers submitted 46 task-resolution reports related to 16 of the 21 patients (76%) (see Table 5).

Table 5

Care manager tasks

In addition, care managers reported that 33 assigned tasks were not resolved because the patient did not answer the phone or reply to an email. Of these, 88% (29/33) were requested for patients who dropped out. The average number of tasks performed was 2.2 per patient and 2.9 per patient actually requiring that the case manager undertake a task. In addition, 1 of the 7 physicians involved in the pilot reported patient visits in the ICT system, and none of the 7 physicians performed the e-learning test.

Patient Feedback

No significant differences were detected between the control group and the intervention group in perception of care quality or accessibility to care and information. Qualitative feedback regarding the intervention provided by patients from the intervention group is shown in Tables 6. Of the 21 patients, 17 (81%) gave positive feedback whereas 7 (33%) gave negative feedback.

Table 6

Positive feedback provided by patients in the intervention group: categories and examples

Table 7

Negative feedback provided by patients in the intervention group: categories and examples

User Experience With the ICT System

Some areas for improvement were identified during the pilot. The following 4 required increased resource utilization and called for a future modification of work processes: (1) Digital certificates (electronic documents required by the ICT system from each user for authentication) required time and were somewhat difficult for both patients and care managers to manage. Further simplifications of certificate handling are necessary and human resources are required to help patients register. In the future, we anticipate that digital certificate “literacy” among users will reduce the importance of this issue. (2) A significant proportion of the care manager workload was due to dropouts not responding to calls and emails. More efficient strategies for interaction with these patients are needed. (3) Physician usage of the ICT system was poor, requiring specific motivational strategies (ie, a reimbursement scheme). (4) Frequently asked questions were available but not used, as the protocol that required the care manager to post these was not strictly enforced.

An additional 4 areas were identified that require changes in the ICT system functionality: (1) The feedback provided when a patient does not complete a questionnaire needs to be improved (ie, that directs users to missing answers). (2) Automated text messages were seen as a disturbance for a minority of patients who experienced fast clinical improvement and wished to finish the intervention early. The solution is for the care manager to have the ability to deactivate these messages for individual patients. (3) The knowledge base of the ICT system needs to be upgraded to increase the pool of available interpretations when questionnaires are submitted. The reason for this is to reduce repetitive answers that are possibly perceived as impersonal by patients. (4) Because the forum was poorly used and the clinical value of forums is unclear [33,34], consideration should be given to discontinuing of this function.

Discussion

The main findings were that (1) user feedback confirmed the ICT system’s alignment with the initial objectives (active patient engagement and improved care management) and (2) the results of the pilot indicate the intervention’s likely influence on improvement of medication adherence and the outcome measures, namely the reduction of depression severity and patients becoming “healthy.”

Overall usability was good, with some feature enhancements necessary to further improve it. Patient feedback about the benefits of the ICT system was in line with the intervention goal and its design. The intervention seemed to support collaborative care [16,35] and active engagement [36] if usability issues are addressed properly in future [37].

We noticed that medication adherence for treatment as usual was low (33%) and comparable to values reported previously (21%) [38]. The intervention group had a significantly higher adherence rate (83%), likely due to the intervention.

The effect size of improvements in scores on the BDI-II in the intervention group compared with the control group was in line with the results of the study by Robertson et al [19] (in both cases Cohen’s d = 1). This seems to indicate an effect of the intervention on treatment success and is further supported by improved outcome measures for intervention group. No significant changes in treatment quality perception were observed.

Limitations

Even though we employed tests that gave the highest statistical power for a given variable type (parametric where applicable, nonparametric otherwise), small sample sizes in most analyses and the additionally high dropout ratio for intention-to-treat analyses resulted in wide confidence intervals. The fixed allocation sequence with even-odd randomization and no blinding [25] likely contributed to bias, and other contributing factors such as depression severity and comorbidities were also not taken into account. We suggest larger sample sizes and more robust methodology (with improved dropout prevention, full randomization, and use of advanced imputation techniques) for further research of the topic.

Conclusions

This pilot study has shown that the intervention—a novel eHealth service offering collaborative care management and active patient engagement—was well received by potential users, seeming to indicate increased patient engagement and feelings of control over treatment progress. The pilot also seems to indicate a likely positive effect of this type of intervention on medication adherence and outcome measures in depression treatment, possibly further improving outcome in addition to interventions offering online cognitive behavioral therapy [39-41].

Acknowledgments

Professor Andrej Marusic, the initial project leader and coauthor of this article, sadly passed away in 2008. The research took place as part of the project “E-depression: improving treatment of people with depressive disorder” financed by Slovene Research Agency (Nr. L3-9306-1669-08) and Ministry of Health of the Republic of Slovenia. The authors would like to thank cordially Mike Galsworthy, PhD, for the comments on statistical analysis. We also thank Matic Ivanovski and Sanja Temnik for their assistance in running the pilot.

Abbreviations


HTTPS	secure hypertext transfer protocol

ICT	information and communication technology

ITT	intention-to-treat

BDI-II	Beck Depression Inventory-II

Footnotes

Conflicts of Interest: None declared.

Editorial note: One of the authors (GE) is also editor of the Journal of Medical Internet Research (JMIR). Because of his involvement in the conduct of this research and writing of this paper, assessment and peer review have been carried out entirely by an associate editor (Riper). GE has not been involved in any editorial decisions related to this paper.

References

1. Kessler RC, Berglund P, Demler O, Jin R, Koretz D, Merikangas KR, Rush AJ, Walters EE, Wang PS., National Comorbidity Survey Replication The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R) JAMA. 2003 Jun 18;289(23):3095–105. doi: 10.1001/jama.289.23.3095. http://jama.ama-assn.org/cgi/pmidlookup?view=long&pmid=12813115.289/23/3095 [PubMed] [Cross Ref]

2. de Graaf LE, Gerhards SA, Evers SM, Arntz A, Riper H, Severens JL, Widdershoven G, Metsemakers JF, Huibers MJ. Clinical and cost-effectiveness of computerised cognitive behavioural therapy for depression in primary care: design of a randomised trial. BMC Public Health. 2008;8:224. doi: 10.1186/1471-2458-8-224. http://www.biomedcentral.com/1471-2458/8/224.1471-2458-8-224 [PMC free article] [PubMed] [Cross Ref]

3. Spitzer RL, Kroenke K, Linzer M, Hahn SR, Williams JB, deGruy FV, Brody D, Davies M. Health-related quality of life in primary care patients with mental disorders. Results from the PRIME-MD 1000 Study. JAMA. 1995 Nov 15;274(19):1511–7. [PubMed]

4. Simon GE, VonKorff M, Rutter C, Wagner E. Randomised trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care. BMJ. 2000 Feb 26;320(7234):550–4. http://bmj.com/cgi/pmidlookup?view=long&pmid=10688563. [PMC free article] [PubMed]

5. Robinson WD, Geske JA, Prest LA, Barnacle R. Depression treatment in primary care. J Am Board Fam Pract. 2005;18(2):79–86. http://www.jabfm.org/cgi/pmidlookup?view=long&pmid=15798136.18/2/79 [PubMed]

6. Simon GE. Evidence review: efficacy and effectiveness of antidepressant treatment in primary care. Gen Hosp Psychiatry. 2002;24(4):213–24.S0163834302001986 [PubMed]

7. Sabate E. Adherence to long-term therapies - Evidence for action. 2003. [2010-01-04]. webcite http://www.who.int/chp/knowledge/publications/adherence_full_report.pdf.

8. Lin EH, Von Korff M, Katon W, Bush T, Simon GE, Walker E, Robinson P. The role of the primary care physician in patients' adherence to antidepressant therapy. Med Care. 1995 Jan;33(1):67–74. [PubMed]

9. Kripalani S, Yao X, Haynes RB. Interventions to enhance medication adherence in chronic medical conditions: a systematic review. Arch Intern Med. 2007 Mar 26;167(6):540–50. doi: 10.1001/archinte.167.6.540. http://archinte.ama-assn.org/cgi/pmidlookup?view=long&pmid=17389285.167/6/540 [PubMed] [Cross Ref]

10. Glied S, Herzog K, Frank R. Review: the net benefits of depression management in primary care. Med Care Res Rev. 2010 Jun;67(3):251–74. doi: 10.1177/1077558709356357.1077558709356357 [PubMed] [Cross Ref]

11. Katon WJ, Seelig M. Population-based care of depression: team care approaches to improving outcomes. J Occup Environ Med. 2008 Apr;50(4):459–67. doi: 10.1097/JOM.0b013e318168efb7.00043764-200804000-00011 [PubMed] [Cross Ref]

12. Gilbody S, Bower P, Fletcher J, Richards D, Sutton AJ. Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Arch Intern Med. 2006 Nov 27;166(21):2314–21. doi: 10.1001/archinte.166.21.2314. http://archinte.ama-assn.org/cgi/pmidlookup?view=long&pmid=17130383.166/21/2314 [PubMed] [Cross Ref]

13. van Straten A, Cuijpers P, Smits N. Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial. J Med Internet Res. 2008;10(1):e7. doi: 10.2196/jmir.954. http://www.jmir.org/2008/1/e7/v10i1e7 [PMC free article] [PubMed] [Cross Ref]

14. Griffiths KM, Christensen H. Depression in primary health care: from evidence to policy. Med J Aust. 2008 Apr 21;188(8 Suppl):S81–3. http://www.mja.com.au/public/issues/188_08_210408/gri11085_fm.html.gri11085_fm [PubMed]

15. Warmerdam L, van Straten A, Twisk J, Riper H, Cuijpers P. Internet-based treatment for adults with depressive symptoms: randomized controlled trial. J Med Internet Res. 2008;10(4):e44. doi: 10.2196/jmir.1094. http://www.jmir.org/2008/4/e44/v10i4e44 [PMC free article] [PubMed] [Cross Ref]

16. Fortney JC, Pyne JM, Edlund MJ, Williams DK, Robinson DE, Mittal D, Henderson KL. A randomized trial of telemedicine-based collaborative care for depression. J Gen Intern Med. 2007 Aug;22(8):1086–93. doi: 10.1007/s11606-007-0201-9. [PMC free article] [PubMed] [Cross Ref]

17. Unützer J, Choi Y, Cook IA, Oishi S. A web-based data management system to improve care for depression in a multicenter clinical trial. Psychiatr Serv. 2002 Jun;53(6):671–3, 678. http://ps.psychiatryonline.org/cgi/pmidlookup?view=long&pmid=12045303. [PubMed]

18. Hunkeler EM, Katon W, Tang L, Williams JW, Kroenke K, Lin EH, Harpole LH, Arean P, Levine S, Grypma LM, Hargreaves WA, Unützer J. Long term outcomes from the IMPACT randomised trial for depressed elderly patients in primary care. BMJ. 2006 Feb 4;332(7536):259–63. doi: 10.1136/bmj.38683.710255.BE. http://www.bmj.com/cgi/pmidlookup?view=long&pmid=16428253.bmj.38683.710255.BE [PMC free article] [PubMed] [Cross Ref]

19. Robertson L, Smith M, Castle D, Tannenbaum D. Using the Internet to enhance the treatment of depression. Australas Psychiatry. 2006 Dec;14(4):413–7. doi: 10.1111/j.1440-1665.2006.02315.x.APY2315 [PubMed] [Cross Ref]

20. improvehealth.eu. [2010-09-30]. webcite http://improvehealth.eu/

21. Meglic M, Brodnik A. Electronic environments for integrated care management: case of depression. In: Mohammed S, Fiaidhi J, editors. Ubiquitous Health and Medical Informatics: The Ubiquity 2.0 Trend and Beyond. Hershey, PA: IGI Global; 2010.

22. Rao MR, Hildebrandt T, Norgaard K, Toth JB. The Resultmaker Online Consultant: Flexible Declarative Workflow Management in Practice. the 12th Enterprise Distributed Object Computing Conference Workshops; Sept 16, 2008; Munich, Germany. 2008. http://www.resultmaker.com/LinkClick.aspx?fileticket=wRgp9f8%2FFW0%3D&tabid=147&mid=647.

23. Beck AT, Steer RA, Ball R, Ranieri W. Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. J Pers Assess. 1996 Dec;67(3):588–97. [PubMed]

24. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961 Jun;4:561–71. [PubMed]

25. Haro JM, Edgell ET, Novick D, Alonso J, Kennedy L, Jones PB, Ratcliffe M, Breier A., SOHO advisory board Effectiveness of antipsychotic treatment for schizophrenia: 6-month results of the Pan-European Schizophrenia Outpatient Health Outcomes (SOHO) study. Acta Psychiatr Scand. 2005 Mar;111(3):220–31. doi: 10.1111/j.1600-0447.2004.00450.x.ACP450 [PubMed] [Cross Ref]

26. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005 Aug 4;353(5):487–97. doi: 10.1056/NEJMra050100.353/5/487 [PubMed] [Cross Ref]

27. Tamburrino MB, Nagel RW, Chahal MK, Lynch DJ. Antidepressant medication adherence: a study of primary care patients. Prim Care Companion J Clin Psychiatry. 2009;11(5):205–11. doi: 10.4088/PCC.08m00694. [PubMed] [Cross Ref]

28. Haynes RB, Taylor DW, Sackett DL, Gibson ES, Bernholz CD, Mukherjee J. Can simple clinical measurements detect patient noncompliance? Hypertension. 1980;2(6):757–64. http://hyper.ahajournals.org/cgi/pmidlookup?view=long&pmid=7007235. [PubMed]

29. Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M. Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. J Med Internet Res. 2009;11(2):e15. doi: 10.2196/jmir.1151. http://www.jmir.org/2009/2/e15/v11i2e15 [PMC free article] [PubMed] [Cross Ref]

30. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991 Feb;59(1):12–9. [PubMed]

31. Seggar LB, Lambert MJ, Hansen NB. Assessing clinical significance: application to the Beck Depression Inventory. BehavTherapy. 2002;33:253–269. doi: 10.1016/S0005-7894(02)80028-4. [Cross Ref]

32. Higgins JP, White IR, Wood AM. Imputation methods for missing outcome data in meta-analysis of clinical trials. Clin Trials. 2008;5(3):225–39. doi: 10.1177/1740774508091600.5/3/225 [PMC free article] [PubMed] [Cross Ref]

33. Griffiths KM, Calear AL, Banfield M. Systematic review on Internet Support Groups (ISGs) and depression (1): Do ISGs reduce depressive symptoms? J Med Internet Res. 2009;11(3):e40. doi: 10.2196/jmir.1270. http://www.jmir.org/2009/3/e40/v11i3e40 [PMC free article] [PubMed] [Cross Ref]

34. Griffiths KM, Calear AL, Banfield M, Tam A. Systematic review on Internet Support Groups (ISGs) and depression (2): What is known about depression ISGs? J Med Internet Res. 2009;11(3):e41. doi: 10.2196/jmir.1303. http://www.jmir.org/2009/3/e41/v11i3e41 [PMC free article] [PubMed] [Cross Ref]

35. Williams JW, Gerrity M, Holsinger T, Dobscha S, Gaynes B, Dietrich A. Systematic review of multifaceted interventions to improve depression care. Gen Hosp Psychiatry. 2007;29(2):91–116. doi: 10.1016/j.genhosppsych.2006.12.003.S0163-8343(06)00223-4 [PubMed] [Cross Ref]

36. Bilsker D, Goldner EM, Jones W. Health service patterns indicate potential benefit of supported self-management for depression in primary care. Can J Psychiatry. 2007 Feb;52(2):86–95. [PubMed]

37. Nijland N, van Gemert-Pijnen J, Boer H, Steehouder MF, Seydel ER. Evaluation of internet-based technology for supporting self-care: problems encountered by patients and caregivers when using self-care applications. J Med Internet Res. 2008;10(2):e13. doi: 10.2196/jmir.957. http://www.jmir.org/2008/2/e13/v10i2e13 [PMC free article] [PubMed] [Cross Ref]

38. Akincigil A, Bowblis JR, Levin C, Walkup JT, Jan S, Crystal S. Adherence to antidepressant treatment among privately insured patients diagnosed with depression. Med Care. 2007 Apr;45(4):363–9. doi: 10.1097/01.mlr.0000254574.23418.f6.00005650-200704000-00014 [PMC free article] [PubMed] [Cross Ref]

39. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007 Mar;37(3):319–28. doi: 10.1017/S0033291706008944.S0033291706008944 [PubMed] [Cross Ref]

40. de Graaf LE, Gerhards SA, Arntz A, Riper H, Metsemakers JF, Evers SM, Severens JL, Widdershoven G, Huibers MJ. Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial. Br J Psychiatry. 2009 Jul;195(1):73–80. doi: 10.1192/bjp.bp.108.054429. http://bjp.rcpsych.org/cgi/pmidlookup?view=long&pmid=19567900.195/1/73 [PubMed] [Cross Ref]

41. Robertson L, Smith M, Tannenbaum D. Case management and adherence to an online disease management system. J Telemed Telecare. 2005;11(Suppl 2):S73–5. doi: 10.1258/135763305775124885. [PubMed] [Cross Ref]

Articles from Journal of Medical Internet Research are provided here courtesy of
Gunther Eysenbach