PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of resprotJMIR PublicationsJMIR Research ProtocolsAboutSearchArchiveCurrent IssueSubmitEditorial Board
 
JMIR Res Protoc. 2016 Apr-Jun; 5(2): e101.
Published online 2016 June 6. doi:  10.2196/resprot.5187
PMCID: PMC4914780

Web Health Monitoring Survey: A New Approach to Enhance the Effectiveness of Telemedicine Systems

Monitoring Editor: Gunther Eysenbach
Reviewed by Eben Afari-kumah, Piero Giacomelli, and Wojciech Glinkowski

Abstract

Background

Aging of the European population and interest in a healthy population in western countries have contributed to an increase in the number of health surveys, where the role of survey design, data collection, and data analysis methodology is clear and recognized by the whole scientific community. Survey methodology has had to couple with the challenges deriving from data collection through information and communications technology (ICT). Telemedicine systems have not used patients as a source of information, often limiting them to collecting only biometric data. A more effective telemonitoring system would be able to collect objective and subjective data (biometric parameters and symptoms reported by the patients themselves), and to control the quality of subjective data collected: this goal be achieved only by using and merging competencies from both survey methodology and health research.

Objective

The objective of our study was to propose new metrics to control the quality of data, along with the well-known indicators of survey methodology. Web questionnaires administered daily to a group of patients for an extended length of time are a Web health monitoring survey (WHMS) in a telemedicine system.

Methods

We calculated indicators based on paradata collected during a WHMS study involving 12 patients, who signed in to the website daily for 2 months.

Results

The patients’ involvement was very high: the patients’ response rate ranged between 1.00 and 0.82, with an outlier of 0.65. Item nonresponse rate was very low, ranging between 0.0% and 7.4%. We propose adherence to the chosen time to connect to the website as a measure of involvement and cooperation by the patients: the difference from the median time ranged between 11 and 24 minutes, demonstrating very good cooperation and involvement from all patients. To measure habituation to the questionnaire, we also compared nonresponse rates to the items between the first and the second month of the study, and found no significant difference. We computed the time to complete the questionnaire both as a measure of possible burden for patient, and to detect the risk of automatic responses. Neither of these hypothesis was confirmed, and differences in time to completion seemed to depend on health conditions. Focus groups with patients confirmed their appreciation for this “new” active role in a telemonitoring system.

Conclusions

The main and innovative aspect of our proposal is the use of a Web questionnaire to virtually recreate a checkup visit, integrating subjective (patient’s information) with objective data (biometric information). Our results, although preliminary and if need of further study, appear promising in proposing more effective telemedicine systems. Survey methodology could have an effective role in this growing field of research and applications.

Keywords: Web questionnaire, Web health monitoring survey, telemedicine, virtual checkup, survey quality, quality indicators, paradata

Introduction

Survey methodology and medical research are becoming more connected: statisticians have always cooperated with clinicians in the analysis of collected data, and in recent years new research questions have arisen, along with new research fields (eg, health technology assessment and health economics). The aging European population and the interest in a healthy population in western countries have contributed to the increase in the number of health surveys, such as the English Longitudinal Study of Ageing (ELSA, since 2002) [1], the Survey of Health, Ageing and Retirement in Europe (SHARE, individuals from 20 European countries aged ≥50 years) [2], and the Health and Retirement Study (HRS) since 1992 in United States [3], to cite only the most well-known and relevant for the number of countries involved or for the systematic nature of the survey.

In all these surveys, as well as in more traditional clinical trials, the role of methodology in survey design, data collection, and data analysis is clear and recognized by the whole scientific community. The role of statisticians in clinical trials is now complemented by survey methodology: patients are often surveyed at the beginning and at the end of the trial to assess their well-being or other characteristics connected to the topic under investigation. Furthermore, patients are often requested to give feedback on their satisfaction with health structures, and, last but not least, health data management and analysis are now mandatory in health services assessment. Therefore, the contact points between medical and statistical research are numerous. Due to the complexity of research topics, the requested skills and knowledge are multidisciplinary and interdisciplinary: the role of researchers has been changing, and researchers now need to interconnect with each other, in order to achieve their research goals.

In the same years, survey methodology has had to couple with the challenges deriving from data collection through information and communications technology, the movement from landline telephones to mobiles and smartphones, and the increasing spread of Wi-Fi connections: modes and techniques to collect responses from people is always changing [4-8]. Guidelines and standards to follow are constantly being updated in order to be able to cover ever-changing and dynamic situations and data gathering modes [9,10], and new metrics derived from macro- and microparadata are being proposed and tested [11-16]. At the same time, the evolution of information and communications technology is giving a strong push to telemonitoring, telemedicine, and home rehabilitation systems [17-21]: new systems are being tested, and are quickly evolving from static to wireless devices, and from obtrusive to less-obtrusive equipment, also making it possible to follow patients and collect their information on their activities and reactions potentially everywhere and continuously.

In our opinion, there are new and common challenges for medicine and survey methodology. Which data are useful to be collected and analyzed? How? And when? What is the data’s value for the well-being of the patient, which is the ultimate aim of a telemonitoring system? What about the quality and usefulness of the data collected?

In the vast scientific literature on telemedicine systems, we have found only a few reports of systems that use data and information coming directly from patients [22-27], although several systems, as for example Philips’ eCareCompanion and eCareCoordinator [28], are now moving on to collecting this information through tablets or smartphones [29,30]. Surely researchers are exploring the implications of how and when these subjective data are collected, their utility for effective monitoring, and, more relevant from our point of view, whether there is a quality control process for this subjective information and how it works, but results have not reached the publication stage.

Our vision is that an effective telemonitoring system would be able to 1) collect objective and subjective data at the same time (biometric parameters and symptoms reported by the patients themselves), 2) control the quality of data collected, and 3) analyze data both at the individual and at an aggregated level.

This ambitious goal can be achieved only by using (and merging) all the technologies and skills available from both survey methodology and health research. In this paper we focus on the contribution of subjective data, that is, data reported by the patients themselves, about themselves, in way that simulates a standard medical examination: this, in our opinion, constitutes the real innovation in the scheme above.

From Acquisition of Vital Signs to an Integrated Telemedicine System

It is quite obvious that remote monitoring and telemedicine systems apply only to patients with chronic conditions. For them, a remote monitoring system is extremely useful, as it is designed to store data collected directly at the patient’s home, with the goal of detecting as soon as possible any worsening of the patient’s health status.

Patient-doctor communication seems to be neglected in home monitoring, and yet it is always present in medical checkups (and in their telephone communications). Data and information are always the objects of the patient-doctor communication in each (first or follow-up) medical visit, and yet it seems that telemedicine systems do not include this relevant source of information. Generally, patient-doctor communication is face-to-face. In the first contact with the patient, the physician starts with a clinical interview: this allows the physician to know how the patient feels and the symptoms that he or she reports, and it precedes the objective physical examination. Thereafter, the patient undergoes further examinations (electrocardiography, echocardiography, chest x-ray, etc, and laboratory tests according to the physician’s request). Only after all the data and information have been collected and read is the physician able to confirm the diagnosis and prescribe suitable therapy.

We underline that—as physicians themselves state—listening to the patient’s account of his or her symptoms and feelings is key to acquiring all of the information about the patient’s health status. To have only biometric or laboratory data is not sufficient for making a correct diagnosis and consequent decisions, for taking actions, and for giving prescriptions. For patients with a chronic disease, there are recognized guidelines (standardized by each disease) that have to be followed in a medical checkup: guidelines recommend not only which laboratory and other tests to take, but also that the symptoms reported by the patients themselves be recorded. Last but not least, patients with chronic diseases are increasingly able to recognize and communicate correct and useful information about their conditions. From these considerations, it follows that a remote monitoring system must have the aim of daily collecting vital signs and biometric data, as well as subjective data, from the patient.

Information and communications technology tools and survey methodology allow for collection of these types of data according to the ideal standards: the questions (asked by a physician) and the responses (given by the patient) can be easily recognized as simulating a questionnaire, whose content and frequency of administration are determined by physicians.

The Role of Survey Methodology in Health Monitoring

The medical checkup can be transformed into a computer-assisted Web interview, and the Web-collected data can be used to generate a longitudinal survey. Subjective data can be analyzed together with objective data in order to have an assessment of the patient’s health status or to detect potential risks.

In chronic diseases, the checkup visit can be more easily transformed into a structured questionnaire, because the clinical interview focuses mainly on presenting symptoms and their variations since the last visit. It is possible to acquire and store on a daily basis (or with a different periodicity; in any case, more frequently than the face-to-face checkup visits) all responses and information given by the patient. More important, it is possible to analyze both clinical and patient variables (measured at the same time) along appropriate longitudinal statistical models.

The aging of the population in western countries calls for the unavoidable and necessary use of a system to automatically assess the health status of patients with chronic conditions, supporting and complementing face-to-face clinic activities, with the aim of focusing the efforts of medical staff on the most serious cases. In order to reliably automate prediction of the state of health of a chronically ill patient, it is also necessary to ensure that data collection is at the highest possible level of reliability and that the data are validated according to metrics established and accepted by the scientific community.

To sum up, in a telemedicine system, questionnaires can be administered (almost) daily for a group of patients for an extended length of time. Are the questionnaires collected in a telemedicine system similar to a longitudinal survey? If we consider that a survey “is a systematic method for gathering information from (a sample of) entities for the purposes of constructing quantitative descriptors of the attributes of the larger population of which the entities are members” [5], then the answer is indeed positive, and we name it a Web health monitoring survey (WHMS).

Quality Measures of a WHMS

If “the quality of a survey is best judged not by its size, scope, or prominence, but by how much attention is given to [preventing, measuring, and] dealing with the many important problems that can arise” [31], then it is relevant to compare the American Association of Public Opinion Research’s (AAPOR) criteria to “produce a quality survey” with the particular situation of a WHMS (Table 1) [31].

Table 1
Comparison between the American Association of Public Opinion Research’s (AAPOR) criteria [31] for a general survey and the features of a Web health monitoring survey (WHMS).

Methods

The main contribution that survey methodologists can make to WHMS is related to the quality of the survey at each stage.

As a first contribution, we offer some indicators, calculated on data collected during the trial of an integrated telemedicine system (Assistenza domiciliare allo SCOmpenso cardiaco attraverso Le Tecniche Avanzate di comunicazione digitale, or ASCOLTA, the Italian word for listen) [32] conducted in 2011 in Pisa, Italy, involving 12 patients with a diagnosis of heart failure. We collected both biometric and subjective data for a period of 2 months for 11 patients and for <1 month for 1 patient, who was the only one hospitalized for noncardiac problems and was thus not included in the study. Patients had to connect daily to a website, at the time most convenient for the patients themselves. A Web questionnaire—dynamically constructed with a different number of questions (and answers) according to the patient’s clinical condition—collected the patients’ health data, including additional physical data (weight and arterial pressures) measured and reported by the patients themselves. At the end of the questionnaire, the system asked the patient to wear wireless electrocardiograph, pulse oximetry oxygen saturation, and respiratory rate recorders for 5 minutes. The patients were free to ask for a new connection at their discretion.

We have previously described and discussed the positive contribution of subjective data to detecting potential risks [32]: variables obtained during a virtual visit substantially contributed to assessment of clinical status (69% of correct classifications), similar to the traditional biometric variables (70%), as assessed by a random forest classification algorithm. The combined use of both variables led to a more correct classification of the patient’s health status (84%).

The questionnaire (see Multimedia Appendix 1 for the English translation) was designed under cardiologists’ supervision, tested, and transformed into a Web questionnaire. The wording of questions and answers and their sequence were in accordance with European Society of Cardiology guidelines [33] for heart failure patients and with the cardiologists’ clinical experience.

We also collected paradata during the study period, along with instrumental and subjective data.

During the 2-month trial, we collected a total of 478 records from 11 patients, 243 for the first month and 235 for the second month.

As described before, a WHMS is a longitudinal survey that is quite different from a panel survey in terms of duration and contents; therefore, we needed more specific indicators than the well-known and consolidated indicators available in the literature [12,34,35].

We therefore propose the use of indicators that measure the response and attrition rates at the patient level. The item nonresponse rate is calculated as a proxy of the patient’s cooperation. To measure whether there was any habituation to a daily questionnaire, we compared the time span to complete the questionnaire between the two halves of the entire study period. Finally, we measured the consistency of the time chosen to complete the questionnaire as a proxy of the patient’s compliance and satisfaction. Focus groups with patients were organized and conducted both at the beginning and at the end of the trial.

Results

Patient Adherence and Response Rate

As expected, patients were very cooperative due to an effective strategy to encourage patient involvement. The number of missing questionnaires was very low. Figure 1 illustrates the response rate by each patient.

Figure 1
Response rate, by patient, during the 2-month ASCOLTA study.

Item Nonresponse

We measured the rate of nonresponse to each item, obtaining good results (Table 2). The 3 biometric variables that the patients were asked to report in the questionnaire all had a nonresponse rate of 0%, which we interpret as denoting a cooperative attitude among all of the patients: during the final focus group the patients expressed positive reactions for their active role.

Table 2
Rate of nonresponse to questionnaire items (N=514).

To measure habituation to the questionnaire, we compared the item nonresponses between the first and the second month. We found no significant differences (Figure 2).

Figure 2
Comparison of nonresponses to questionnaire items by period of the study (period 1: first month; period 2: second month) and by question.

Time to Complete the Questionnaire

From paradata collected, we calculated the time spent to complete the questionnaires both for the daily and the discretionary questionnaires (Table 3). The time required seemed to depend on the patients’ health conditions; in general, time required was very short and acceptable, and at our advice was not perceived as a burden by the patient.

Table 3
Time spent to complete the daily and discretionary questionnaires, by patient.

We also tested for difference in completion time between the first and second period (Figure 3 and Table 4). Although the difference was statistically significant for 3 patients, the overall difference was not.

Table 4
Time (min.s) spent to complete the daily questionnaires: comparison between first and second period of the study, by patient.
Figure 3
Time spent to complete the questionnaires, by patient and by period of the study (period 1: first month; period 2: second month).

Patients’ Chosen Time of Day to Complete the Questionnaire

To accommodate the patient’s lifestyle, each one could choose the hour of the day when they would complete the questionnaire, with an interval of ±1 hour (Table 5).

Table 5
Time of day chosen to complete the questionnaire, by patient.

During the final focus group, the patients expressed their appreciation for this support, and therefore their behavior is a proxy of a cooperative attitude.

The mean difference was about 20 minutes, both from the median and from the mean.

Discussion

It is worth noting that our WHMS is part of a telemedicine system, making its goal different from that of other health surveys such as a Web-based daily questionnaire for health [26]. Telemedicine data are collected and must be analyzed at the patient level, because the main goal is to assess the health status of the single patient. On the contrary, in epidemiological studies the focus is on measuring health parameters for the entire population.

Moreover, a WHMS is designed for patients with chronic conditions, and therefore they may already be greatly involved in the survey or their cooperative attitude can be reinforced: no incentive is needed, because the implicit incentive for each patient is to get better attention from the medical staff and to contribute actively to the management of their chronic disease. As a consequence, all indicators based on response rate (although—or perhaps because—they have values never obtained in a general survey) seem less relevant to measure the quality of the WMHS. Nonetheless, their trend over time for longer study is worth computing and controlling.

WHMS is a longitudinal survey with very strict periodicity, even daily, which could be a burden for the respondent [36], resulting in a possible high attrition rate. The attrition rate at the patient level is therefore relevant in monitoring the quality of data collection. Our results demonstrate that this risk is very low, taking into account that only a few minutes are required to complete the questionnaire.

Even with a diligent patient, there is the risk that patients’ responses will become automatic. The results we obtained led us to reject the hypothesis of the presence of automatic responses; therefore, the patient’s responses are deemed to be reliable.

Signing in to the daily questionnaire and the patients’ choice of when during the day to sign in seem a good measure of their cooperation.

Focus groups with patients confirmed the efficacy of the questionnaire and the positive reactions of patients to the new mode of collecting this information. Patients appreciated their new active role in the ASCOLTA system.

Study Limitations

We acknowledge that our study has some critical limitations, such as the small number of patients studied and the short duration of the longitudinal survey. Furthermore, the study involved only patients with heart failure. Other paradata can be collected, such as those needed to measure response latency or time-to-click [16,37].

In more general terms, the digital skills of potential respondents to a WHMS is a challenge: a sample of patients with chronic conditions would be older, and the greater digital divide in this group could be a barrier to their participation in such a survey [38-40].

Many aspects of a WHMS deserve to be developed, by applying the same approach to other kinds of chronic pathologies (eg, chronic obstructive pulmonary disease, diabetes, and hypertension) [41,42]. Moreover, the expansion of wireless technologies on platforms such as smartphones and tablets will enable the collection of biometric data to extend even to people who are not yet ill (eg, people at risk) or to healthy people who participate in amateur sports. In addition, although this is an open field, the reactions of medical staff to the virtual visit are worth investigating, as patient satisfaction is further explored [43-47].

Information and communications technology solutions can be applied to facilitate and enable patient-doctor communication [48-53]. Many virtual medical visit applications are being developed [54-56] and the increasing number of websites testifies to patients’ willingness to play a more active role in their own health management [57,58].

The main and innovative aspect of our proposal is the use of a Web questionnaire to virtually recreate a checkup visit, integrating subjective (patient information) with objective data (biometric information) in a unique database that can be analyzed with appropriate statistical methods. We suggest here some indicators to control the process of data collection, along with criteria already established by survey methodology, but we are conscious that other specific metrics need to be studied.

Our results, although preliminary, appear promising and, in our opinion, could be of significance to the ongoing debate about the most appropriate type of telemonitoring and remote care of patients. Further studies, in a larger population, could be useful to ultimately confirm our preliminary results and to provide a more efficient cost-benefit ratio, according to health technology assessment.

Survey methodology could have an effective role in this growing field of research and applications.

In agreement with the Couper’s statement that “We constantly need to hone our skills, update our knowledge, and expose ourselves to new developments in other disciplines and fields of research and application.” [59], we think that monitoring of health data is a challenge for survey methodologists. Skills and competencies of survey methodologists and statisticians must guide data collection and data analysis in health monitoring surveys.

Acknowledgments

ASCOLTA Project was funded by the Regione Toscana (BANDO UNICO R&S, anno 2008 POR CREO Attività 1.6 Linea A) to a pool of firms (CAEN SpA - leader, Evidence Srl, EMAC Srl, Develer Srl, Sinergest Srl, Laboratori ARCHA Srl). Sant’Anna School of Advanced Studies (Institute of Economics, Institute of Life Sciences, and Institute of Communication, Information, and Perception Technologies) and the Department Cardiac Thoracic and Vascular Department (University of Pisa) gave scientific support. The project was presented in April 2009, approved in April 2010, and concluded in March 2012.

The study protocol was approved by the Ethical Committee of the Hospital of Pisa (Study No. 3461/2011, presented November 29, 2011, approved February 9, 2012).

Abbreviations

AAPOR
American Association of Public Opinion Research
ASCOLTA
Assistenza domiciliare allo SCOmpenso cardiaco attraverso Le Tecniche Avanzate di comunicazione digitale
ELSA
English Longitudinal Study of Ageing
HRS
Health and Retirement Study
SHARE
Survey of Health, Ageing and Retirement in Europe
WHMS
Web health monitoring survey

Multimedia Appendix 1

Questionnaires used (English translation).

Footnotes

Conflicts of Interest: None declared.

References

1. ELSA English Longitudinal Study of Ageing: Insight Into a Maturing Population. London, UK: The Institute for Fiscal Studies; 2011. [2015-09-16]. http://www.elsa-project.ac.uk/ webcite.
2. Survey of Health Ageing Retirement. Munich, Germany: SHARE-ERIC; 2013. [2015-09-16]. http://www.share-project.org/ webcite.
3. Health and Retirement Study. Ann Arbor, MI: Institute for Social Research, University of Michigan; 2015. [2015-09-16]. http://hrsonline.isr.umich.edu/; webcite.
4. Conrad FG, Schober MF, editors. Envisioning the Survey Interview of the Future. Hoboken, NJ: Wiley-Interscience; 2008.
5. Groves R, Fowler FJ, Couper M, Lepkowski J, Singer E, Tourangeau R. Survey Methodology. New York, NY: Wiley; 2009.
6. Couper MP. The future of modes of data collection. Public Opinion Q. 2011 Dec 15;75(5):889–908. doi: 10.1093/poq/nfr046. [Cross Ref]
7. Callegaro M, Baker R, Bethlehem J, Goritz A, Krosnick J, Lavrakas P, editors. Online Panel Research: A Data Quality Perspective. Chichester, UK: Wiley; 2014. p. A.
8. Callegaro M, Lozar MK, Vehovar V, editors. Web Survey Methodology. Los Angeles, CA: Sage; 2015.
9. Link MW, Murphy J, Schober MF, Buskirk TD, Hunter Childs J, Langer Tesfaye C. Mobile technologies for conducting, augmenting and potentially replacing surveys: executive summary of the AAPOR Task Force on Emerging Technologies in Public Opinion Research. Public Opinion Q. 2014 Nov 25;78(4):779–87. doi: 10.1093/poq/nfu054. [Cross Ref]
10. AAPOR Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 8th edition. [2015-09-30]. http://www.aapor.org/AAPORKentico/Communications/AAPOR-Journals/Standard-Definitions.aspx webcite.
11. Heerwegh D. Explaining response latencies and changing answers using client-side paradata from a Web survey. Soc Sci Comput Rev. 2003 Aug 01;21(3):360–73. doi: 10.1177/0894439303253985. [Cross Ref]
12. Callegaro M, DiSogra C. Computing response metrics for online panels. Public Opinion Q. 2009 Jan 29;72(5):1008–32. doi: 10.1093/poq/nfn065. [Cross Ref]
13. Kreuter F, editor. Improving Surveys With Paradata: Analytic Uses of Process Information. Hoboken, NJ: Wiley & Sons; 2013.
14. Schouten B, Shlomo N, Skinner C. Indicators for monitoring and improving representativeness of responses. J Off Statistics. 2011;27(2):231–53. http://www.jos.nu/Articles/abstract.asp?article=272231 webcite.
15. Sirkis R, Jans M. Using statistical process control to understand variation in computer assisted personal interviewing data. Joint Statistical Meetings; July 30-August 4, 2011; Miami Beach, FL. 2011. pp. 477–89. http://www.amstat.org/sections/srms/proceedings/ webcite.
16. Couper MP, Kreuter F. Using paradata to explore item level response times in surveys. J Royal Stat Soc Ser A (Stat Soc) 2013;176:271–86. doi: 10.1111/j.1467-985X.2012.01041.x. [Cross Ref]
17. Nangalia V, Prytherch DR, Smith GB. Health technology assessment review: remote monitoring of vital signs--current status and future challenges. Crit Care. 2010;14(5):233. doi: 10.1186/cc9208. http://ccforum.biomedcentral.com/articles/10.1186/cc9208. [PMC free article] [PubMed] [Cross Ref]
18. Anker D, Koehler F, Abraham WT. Telemedicine and remote management of patients with heart failure. Lancet. 2011 Aug 20;378(9792):731–9. doi: 10.1016/S0140-6736(11)61229-4. [PubMed] [Cross Ref]
19. Chaudhry SI, Mattera JA, Curtis JP, Spertus JA, Herrin J, Lin Z, Phillips CO, Hodshon BV, Cooper LS, Krumholz HM. Telemonitoring in patients with heart failure. N Engl J Med. 2010;363(24):2301–9. doi: 10.1056/NEJMoa1010029. [PMC free article] [PubMed] [Cross Ref]
20. Desai AS. Home monitoring heart failure care does not improve patient outcomes: looking beyond telephone-based disease management. Circulation. 2012 Feb 14;125(6):828–36. doi: 10.1161/CIRCULATIONAHA.111.031179. http://circ.ahajournals.org/cgi/pmidlookup?view=long&pmid=22331920. [PubMed] [Cross Ref]
21. Bashshur RL, Shannon GW, Smith BR, Alverson DC, Antoniotti N, Barsan WG, Bashshur N, Brown EM, Coye MJ, Doarn CR, Ferguson S, Grigsby J, Krupinski EA, Kvedar JC, Linkous J, Merrell RC, Nesbitt T, Poropatich R, Rheuban KS, Sanders JH, Watson AR, Weinstein RS, Yellowlees P. The empirical foundations of telemedicine interventions for chronic disease management. Telemed J E Health. 2014 Sep;20(9):769–800. doi: 10.1089/tmj.2014.9981. http://europepmc.org/abstract/MED/24968105. [PMC free article] [PubMed] [Cross Ref]
22. Little M, Wicks P, Vaughan T, Pentland A. Quantifying short-term dynamics of Parkinson's disease using self-reported symptom data from an Internet social network. J Med Internet Res. 2013;15(1):e20. doi: 10.2196/jmir.2112. http://www.jmir.org/2013/1/e20/ [PMC free article] [PubMed] [Cross Ref]
23. Bellos C, Papadopoulos A, Rosso R, Fotiadis D. Identification of COPD patients' health status using an intelligent system in the CHRONIOUS wearable platform. IEEE J Biomed Health Inform. 2014 May;18(3):731–8. doi: 10.1109/JBHI.2013.2293172. [PubMed] [Cross Ref]
24. Bellos C, Papadopoulos A, Rosso R, Fotiadis D. CHRONIOUS: clinical validation of the CHRONIOUS wearable system in patients with chronic disease. 35th Annual International Conference of the IEEE EMBS; July 3-7, 2013; Osaka, Japan. 2013. pp. 7084–7. [PubMed]
25. Steele GC, Khan AI, Kuluski K, McKillop I, Sharpe S, Bierman AS, Lyons RF, Cott C. Improving patient experience and primary care quality for patients with complex chronic disease using the electronic patient-reported outcomes tool: adopting qualitative methods into a user-centered design approach. JMIR Res Protoc. 2016 Feb;5(1):e28. doi: 10.2196/resprot.5204. http://www.researchprotocols.org/2016/1/e28/ [PMC free article] [PubMed] [Cross Ref]
26. Sugiura H, Ohkusa Y, Akahane M, Sano T, Okabe N, Imamura T. Development of a web-based survey for monitoring daily health and its application in an epidemiological survey. J Med Internet Res. 2011;13(3):e66. doi: 10.2196/jmir.1872. http://www.jmir.org/2011/3/e66/ [PMC free article] [PubMed] [Cross Ref]
27. Dorsch MP, Farris KB, Bleske BE, Koelling TM. A web application for self-monitoring improves symptoms in chronic systolic heart failure. Telemed J E Health. 2015 Apr;21(4):267–70. doi: 10.1089/tmj.2014.0095. [PubMed] [Cross Ref]
28. Open A World of Cloud-Based Collaborative Care. Amsterdam, Netherlands: Koninklijke Philips N.V; 2015. [2015-09-16]. http://www.usa.philips.com/healthcare-innovation/about-health-suite/applications-health-suite webcite.
29. Lygidakis C, Wallace P, Tersar C, Marcatto F, Ferrante D, Della VR, Scafuri F, Scafato E, Struzzo P. Download your doctor: implementation of a digitally mediated personal physician presence to enhance patient engagement with a health-promoting Internet application. JMIR Res Protoc. 2016 Mar;5(1):e36. doi: 10.2196/resprot.5232. http://www.researchprotocols.org/2016/1/e36/ [PMC free article] [PubMed] [Cross Ref]
30. McLean G, Murray E, Band R, Saunderson K, Hanlon P, Little P, McManus RJ, Yardley L, Mair FS. Digital interventions to promote self-management in adults with hypertension: protocol for systematic review and meta-analysis. JMIR Res Protoc. 2015 Nov;4(4):e133. doi: 10.2196/resprot.4648. http://www.researchprotocols.org/2015/4/e133/ [PMC free article] [PubMed] [Cross Ref]
31. American Association for Public Opinion Research . Best Practices for Research. Oakbrook Terrace, IL: AAPOR; 2016. [2016-05-18]. http://www.aapor.org/Standards-Ethics/Best-Practices.aspx webcite.
32. Romano MF, Sardella MV, Alboni F, L'Abbate A, Mariotti R, Di Bello V. The informative contribution of the “virtual medical visit” in a new heart failure telemedicine integrated system. Telemed J E Health. 2014 Jun;20(6):508–21. doi: 10.1089/tmj.2013.0225. [PubMed] [Cross Ref]
33. McMurray JJ, Adamopoulos S, Anker SD, Auricchio A, Böhm M, Dickstein K, Falk V, Filippatos G, Fonseca C, Gomez-Sanchez MA, Jaarsma T, Køber L, Lip GY, Maggioni AP, Parkhomenko A, Pieske BM, Popescu BA, Rønnevik PK, Rutten FH, Schwitter J, Seferovic P, Stepinska J, Trindade PT, Voors AA, Zannad F, Zeiher A, ESC Committee for Practice Guidelines ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure 2012: The Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure 2012 of the European Society of Cardiology. Developed in collaboration with the Heart Failure Association (HFA) of the ESC. Eur Heart J. 2012 Jul;33(14):1787–847. doi: 10.1093/eurheartj/ehs104. http://eurheartj.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=22611136. [PubMed] [Cross Ref]
34. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) J Med Internet Res. 2004 Sep 29;6(3):e34. doi: 10.2196/jmir.6.3.e34. http://www.jmir.org/2004/3/e34/ [PMC free article] [PubMed] [Cross Ref]
35. Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11. doi: 10.2196/jmir.7.1.e11. http://www.jmir.org/2005/1/e11/ [PMC free article] [PubMed] [Cross Ref]
36. Axinn WG, Jennings EA, Couper MP. Response of sensitive behaviors to frequent measurement. Soc Sci Res. 2015 Jan;49:1–15. doi: 10.1016/j.ssresearch.2014.07.002. http://europepmc.org/abstract/MED/25432599. [PMC free article] [PubMed] [Cross Ref]
37. Callegaro M. Using paradata to improve the quality of web surveys: some examples and applications. Conference on Paradata From surveys research to practice; 2014; London, UK. London, UK: Royal Statistical Society; 2014. http://www.southampton.ac.uk/s3ri/news/events/2014/06/26_paradata_from_survey_research_to_practice.page webcite.
38. Gracia E, Herrero J. Internet use and self-rated health among older people: a national survey. J Med Internet Res. 2009 Dec;11(4):e49. doi: 10.2196/jmir.1311. http://www.jmir.org/2009/4/e49/ [PMC free article] [PubMed] [Cross Ref]
39. Romano MF, Sardella MV, Alboni F, Russo L, Mariotti R, Nicastro I, Barletta V, Di Bello V. Is the digital divide an obstacle to e-health? An analysis of the situation in Europe and in Italy. Telemed J E Health. 2015 Jan;21(1):24–35. doi: 10.1089/tmj.2014.0010. [PubMed] [Cross Ref]
40. Jones J, Cassie S, Thompson M, Atherton I, Leslie SJ. Delivering healthcare information via the internet: cardiac patients' access, usage, perceptions of usefulness, and web site content preferences. Telemed J E Health. 2014;20(3):223–8. doi: 10.1089/tmj.2013.0182. [PubMed] [Cross Ref]
41. Wang J, Wang Y, Wei C, Yao NA, Yuan A, Shan Y, Yuan C. Smartphone interventions for long-term health management of chronic diseases: an integrative review. Telemed J E Health. 2014 Jun;20(6):570–83. doi: 10.1089/tmj.2013.0243. [PubMed] [Cross Ref]
42. Bellos C, Papadopoulos A, Rosso R, Fotiadis D. CHRONIOUS: A wearable platform for monitoringmanagement of patients with chronic disease. 33rd Annual International Conference of the IEEE EMBS; August 30-September 3, 2011; Boston, MA. 2011. pp. 864–7. [PubMed]
43. Bostock Y, Hanley J, McGown D, Pinnock H, Padfield P, McKinstry B. The acceptability to patients and professionals of remote blood pressure monitoring using mobile phones. Primary Health Care. 2009 Jul 28;10(4):299–308. doi: 10.1017/S1463423609990107. [Cross Ref]
44. Cafazzo JA, Leonard K, Easty AC, Rossos PG, Chan CT. Patient perceptions of remote monitoring for nocturnal home hemodialysis. Hemodial Int. 2010 Oct;14(4):471–7. doi: 10.1111/j.1542-4758.2010.00473.x. [PubMed] [Cross Ref]
45. Matlock D. Big Brother is watching you: what do patients think about ICD home monitoring? Circulation. 2010;122(4):319–21. doi: 10.1161/CIRCULATIONAHA.110.966515. [PubMed] [Cross Ref]
46. Or CKL, Karsh B, Severtson DJ, Burke LJ, Brown RL, Brennan PF. Factors affecting home care patients' acceptance of a web-based interactive self-management technology. J Am Med Inform Assoc. 2011 Jan;18(1):51–9. doi: 10.1136/jamia.2010.007336. http://jamia.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=21131605. [PMC free article] [PubMed] [Cross Ref]
47. McBride DL, Levasseur SA, Li D. Development and validation of a web-based survey on the use of personal communication devices by hospital registered nurses: pilot study. JMIR Res Protoc. 2013;2(2):e50. doi: 10.2196/resprot.2774. http://www.researchprotocols.org/2013/2/e50/ [PMC free article] [PubMed] [Cross Ref]
48. Alkhaldi G, Hamilton FL, Lau R, Webster R, Michie S, Murray E. The effectiveness of technology-based strategies to promote engagement with digital interventions: a systematic review protocol. JMIR Res Protoc. 2015;4(2):e47. doi: 10.2196/resprot.3990. http://www.researchprotocols.org/2015/2/e47/ [PMC free article] [PubMed] [Cross Ref]
49. Antonelli D, Bellomo D, Bruno G, Villa A. Evaluating collaboration effectiveness of patient-to-doctor interaction in a healthcare territorial network. In: Camarinha-Matos LM, Xu L, Afsarmanesh H, editors. Collaborative Networks in the Internet of Services. Berlin, Germany: Springer; 2012. pp. 128–36.
50. Boulos MNK, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online. 2011;10:24. doi: 10.1186/1475-925X-10-24. http://www.biomedcentral.com/1475-925X/10/24. [PMC free article] [PubMed] [Cross Ref]
51. Lindberg B, Nilsson C, Zotterman D, Söderberg S, Skär L. Using information and communication technology in home care for communication between patients, family members, and healthcare professionals: a systematic review. Int J Telemed Appl. 2013;2013:461829. doi: 10.1155/2013/461829. http://dx.doi.org/10.1155/2013/461829. [PMC free article] [PubMed] [Cross Ref]
52. Bert F, Giacometti M, Gualano MR, Siliquini R. Smartphones and health promotion: a review of the evidence. J Med Syst. 2014 Jan;38(1):9995. doi: 10.1007/s10916-013-9995-7. [PubMed] [Cross Ref]
53. Mattsson S, Olsson EMG, Alfonsson S, Johansson B, Carlsson M. Measuring use of health-related support on the Internet: development of the Health Online Support Questionnaire (HOSQ) J Med Internet Res. 2015 Nov;17(11):e266. doi: 10.2196/jmir.4425. http://www.jmir.org/2015/11/e266/ [PMC free article] [PubMed] [Cross Ref]
54. Mahler SA, Burke GL, Duncan PW, Case LD, Herrington DM, Riley RF, Wells BJ, Hiestand BC, Miller CD. HEART pathway accelerated diagnostic protocol implementation: prospective pre-post interrupted time series design and methods. JMIR Res Protoc. 2016 Jan;5(1):e10. doi: 10.2196/resprot.4802. http://www.researchprotocols.org/2016/1/e10/ [PMC free article] [PubMed] [Cross Ref]
55. Eissens van der Laan MR, van Offenbeek MAG, Broekhuis H, Slaets JPJ. A person-centred segmentation study in elderly care: towards efficient demand-driven care. Soc Sci Med. 2014 Jul;113:68–76. doi: 10.1016/j.socscimed.2014.05.012. [PubMed] [Cross Ref]
56. Dixon RF, Rao L. Asynchronous virtual visits for the follow-up of chronic conditions. Telemed J E Health. 2014 Jul;20(7):669–72. doi: 10.1089/tmj.2013.0211. [PubMed] [Cross Ref]
57. Society for Participatory Medicine e-patients.net. 2015. [2015-09-28]. http://e-patients.net/ webcite.
58. PatientsLikeMe. 2015. [2016-05-18]. https://www.patientslikeme.com/ webcite.
59. Couper M. Is the sky falling? New technology, changing media, and the future of surveys. Survey Res Methods. 2013;7(3):145–56. doi: 10.18148/srm/2013.v7i3.5751. [Cross Ref]

Articles from JMIR Research Protocols are provided here courtesy of JMIR Publications Inc.