This is a study of web-logfiles generated from eviQ over a 26-month period. The study period coincides with the launch of the new platform in October 2009 and covers the first two years of operation until December 2011. The previous platform (CI-SCaT) was taken offline on March 31 2010 to allow sufficient opportunity for users to transition and register on the eviQ website.
In Australia, oncology protocols are delivered primarily in the ambulatory care (outpatient) setting at metropolitan hospitals (university-affiliated, tertiary referral centers covering geographic areas of around 75 square kilometers), regional centers (with catchments up to 1,200 square kilometers) and rural hospitals (with catchments up to 3,400 square kilometers).
eviQ is managed by the Cancer Institute NSW, a government funded agency established to improve cancer control in Australia’s largest state, NSW. eviQ primarily targets health professionals involved in implementing cancer care by providing detailed and extensive instructions on how to deliver evidence-based treatments safely and appropriately. Treatment information encompasses adolescent and young adult care, cancer genetics, haematology, haemapoietic progenitor cell transplants, medical oncology, nursing, primary health, palliative care and radiation oncology. The site comprises over 1,300 protocols, developed by a consensus process involving specialist physicians, nurses, pharmacists and allied health practitioners from across Australia. Each protocol undergoes a comprehensive review every 1 to 2 years. While the primary eviQ target audience is health professionals, the site also publishes information tailored specifically to cancer patients and their carers.
Data sources and analysis
For the purposes of this evaluation we used two data sources, both of which have different capabilities in terms of understanding eviQ use. Outputs from both data sources were converted to Microsoft excel 2010 format for analysis. We report demographic variables for all registrants at the end of our study period (December 31 2011) and patterns of use during a typical month based on eviQ logfiles. For the latter, we examined rates of use across the last three months of the study period (October through December, 2011). Rates of overall use were lowest in December (1,145 hits/100 registrants) most probably due to the holiday period, but were comparable in October (1,499 hits/100 registrants) and November (1,418 hits/100 registrants). We selected October 2011 to represent a ‘typical’ month and all subsequent analyses focused exclusively on this period.
The eviQ secretariat provided the research team with access to de-identified data from the eviQ platform. Demographic registrant and logfile data were obtained on-site at the Cancer Institute NSW in unit record format (stripped of personal identifiers such as usernames). The eviQ platform has the capacity to generate data on the characteristics of all registered users including registrant type (individual clinician or unit registration), health setting (primary care or hospital), health sector (public, private, or both), geographical location of practice, clinician group, years of oncology experience, and source of referral to the eviQ website. This information is reported by users upon registration and website registrants are prompted to update this information on an annual basis. Further, logfiles also monitor webhits, defined as one click anywhere on the eviQ website, that can be stratified by any of the aforementioned variables (e.g., clinician type, years of oncology experience) and the time at which the webhits occur (e.g., time of day, month, year). However, the current eviQ logfile reports are aggregated and do not have the capacity to determine the content accessed according to health professional groups.
As such, we report on the following:
We report the characteristics of individuals and units identifying themselves as Australian health professionals upon site registration. We report the number of new health professional registrations by month for the period October 2009 to December 2011 and the demographic characteristics of all registrants at the end of the study period.
Patterns of eviQ use
We examined webhits during standard clinic hours (Monday to Friday between 08:00–18:00) compared with use outside clinic hours. This approach has been used previously in logfile analyses as a proxy for point-of-care use in Australian clinical practice [13
]. Moreover, the majority of chemotherapy and radiotherapy cancer treatments are delivered during these times.
We stratified our analysis by registrant type (individual or unit registration), individual clinician group (medical, nursing, pharmacy and radiation therapists) and years of oncology experience. As is standard in logfile analysis, we report both the volume of webhits and hits/100 website registrants [14
The eviQ secretariat provided the research team with access to their Google Analytics profile from which we extracted data of interest. Google Analytics’ reports provide data on website traffic and allows for the examination of the intensity of eviQ use for all registrants including number of visits and unique visits to eviQ. A visit is defined as a registrant logging on to eviQ for up to four hours, unless the registrant terminates their visit by logging off or leaving the website. Google Analytics produces aggregated data relating to typical user sessions including number of pages accessed and time spent on eviQ during a typical visit for a defined period (e.g., visits in a typical week or month). Google Analytics also has the capacity to provide data on content accessed and time spent on specific eviQ pages (e.g., time spent on a particular chemotherapy protocol page).
Data from Google Analytics does not distinguish between different user groups (e.g., health professionals’ vs. consumers), limiting the capacity to undertake analyses on how different groups access eviQ. However, given that health professionals comprise 92% (n
14,800) of eviQ registrants, Google Analytics output will be heavily influenced by patterns of use by health professionals.
Using this data source, we report on the following during a typical month (October 2011):
Typical user sessions
We report the total number of eviQ visits, average number of pages/visit, average visit duration, and mode of access (mobile device or computer). Google Analytics output is averaged across all users such that we cannot calculate descriptive statistics such as median and range for these data.
We report the top 100 pages accessed and time spent on these pages. The top 100 pages were grouped according to their content into the following categories: login/registration; transition pages (such as tabs directing users to specific content); and cancer treatment content pages (chemotherapy protocols or supportive treatment information). We used the time spent on each of the 100 pages to calculate the range and median time spent on pages according to the abovementioned categories. To better understand the range of clinical content accessed by eviQ users, we also conducted an analysis of the top 100 cancer treatment content pages (excluding all login, registration, and transition pages). We present total, median and range of page views according to the following categories: medical oncology, haematology, radiation oncology, and supportive treatment information.
Ethics approval to monitor eviQ utilization using eviQ web-logfiles and Google Analytics was obtained from the NSW Population and Health Services Research Ethics committee (approval number HREC/10/CIPHS/70).