|Home | About | Journals | Submit | Contact Us | Français|
Statistical measurements alone are insufficient to ensure robust data for point prevalence surveys (PPS) of healthcare-associated infections (HAI). Data quality is determined by the type of data, data collection methods and available resources. Data collectors’ views regarding the acceptability of data collection process for validation studies are also important to consider.
To explore data collectors’ views on the acceptability of data collection processes used for a European validation PPS of HAI and antimicrobial use (AMU).
An anonymous online survey was conducted with 67 data collectors from 10 European countries involved in the study.
Twenty-five (64.1%) participants viewed AMU data collection as easy/quite easy whereas only five (12.8%) thought HAI data collection was easy/quite easy. Six (17%) participants indicated that incentives and 21 (56.8%) that disincentives were possibly/definitely present for reporting cases of HAI. Engagement of staff was not thought to have adversely affected data collection as only one (2.6%) and five (15.4%) participants thought involvement of hospital PPS teams and administration was low/very low, respectively.
Participants believed the approaches used were appropriate but that more training was required prior to data collection, some case definitions should be reviewed and the number of variables reduced.
The validity of approaches to the surveillance of healthcare-associated infection (HAI) is often assessed statistically to produce measures of sensitivity, specificity, positive predictive value, negative predictive value and inter-rater reliability (Gastmeier et al., 1998; McCoubrey et al., 2005; Sherman et al., 2006; Zuschneid et al., 2007). It is argued that the views of data collectors regarding the acceptability of the data collection process for validation studies are also important to consider (Zuschneid et al., 2007).
A validation study was conducted of a European point prevalence survey (PPS) of HAI and antimicrobial use (AMU) (European Centre for Disease Prevention and Control [ECDC], 2013). The objectives of the study were to test the usual measures of the sensitivity and specificity of reporting HAIs and AMU and inter-rater reliability in the European PPS, but in addition the views of data collectors on the acceptability of the data collection processes were sought to provide a comprehensive assessment of how data collection for PPS could be improved for future PPS. Findings of the validation study related to sensitivity, specificity and inter-rater reliability are published elsewhere (Reilly, 2015). This paper presents the findings related to data collectors’ views on the acceptability of the data collection processes.
The ECDC PPS collected data on the presence of HAI, use of antimicrobials, and patient and hospital denominator data from 29 countries using a standardised methodology (ECDC, 2013). Data collection was coordinated by national coordinating teams who were trained centrally in the methodology and who then cascaded training to local data collectors. Concurrently with data collection for the PPS, and using the same standardised methodology, 10 countries collected data for the validation study. Validation of the data was performed in each country by either a member of the National Coordinating Centre or a second local data collector. Following completion of data collection for the validation study, an invitation to participate in an anonymous online survey, using SurveyMonkey©, was sent by the National Coordinating Centres to all English-speaking data collectors who had taken part in the validation study. The questionnaire asked what had gone well and what did not go so well. The questions were developed by the project management group following review of the literature (Duerink et al., 2006; Gastmeier et al., 1998; Liata et al., 2009; McCoubrey et al., 2005; Sherman et al., 2006) (Table 1). The survey was approved by the Glasgow Caledonian University Ethics Committee (HLS id: A11/40) and adhered to the principle of informed consent by incorporating information about the survey and a consent form at the beginning of questionnaire.
Responses to the questions were of two types: participants were required to either choose from a number of categorical variables on a Likert-type scale or to answer by adding free text comments. The categorical responses were analysed using descriptive statistics and the free text by content analysis (Hsieh and Shannon, 2005). The data from the free text responses were analysed question by question. The responses were read repeatedly, then word by word to identify words or phrases that explained the related categorical responses.
A total of 67 data collectors from the 10 participating countries were invited to complete the survey. Of these, 40 responded (60% response rate). The results are presented under the four topic areas of the questionnaire.
Collection of the AMU data for the PPS validation study was perceived to be easier than the collection of HAI data (Table 2). Twenty-five out of 39 (64.1%) participants viewed AMU data collection as easy or quite easy whereas five out of 39 (12.8%) thought HAI data collection was easy or quite easy. This was a significant difference with the test for trend in Table 2 being χ²(1) = 22.0, 2p = 0.000003.
Participants’ comments related to the difficulty of collecting HAI data suggested that the reasons for the difficulty were the inability to meet the strict case definitions either because of lack of patient information or lack of diagnostic testing. Patient information was lacking because of inadequate documentation in the patients’ clinical notes or lack of access to the patients’ doctors to give this information verbally. Diagnostic tests were not always available because of differences in local policy that determined what diagnostic tests could be conducted.
Survey participants thought that there were little incentives present, and more disincentives, for both reporting cases of HAI and performing diagnostic test for HAI. Only six out of 35 (17%) indicated that incentives were possibly or definitely present for reporting cases of HAI and similarly only five out of 35 (14%) thought incentives for diagnostic testing were possibly or definitely present. Conversely, just over 50% of participants thought that disincentives for both reporting cases of HAI and conducting diagnostic testing for HAI were present (Table 3). Those reporting disincentives suggested that these were national targets with financial penalties and the fear of creating a negative image of the hospital.
Engagement of staff was not thought to be a factor adversely affecting data collection for the PPS validation study as only one (2.6%) and five (15.4%) participants thought involvement of hospital PPS teams and hospital administration was low or very low, respectively. However, hospital PPS team involvement was rated higher than hospital administration involvement. Thirty-one (82%) participants thought involvement of hospital PPS teams was high / very high whereas only six (15%) participants rated hospital administration involvement this high. Some improvements were suggested. Participants felt that if the PPS lead within each hospital was appropriately experienced, financial incentives were offered and ward staff were involved early in the process staff engagement would improve.
Suggestions for improvement of the data collection process from participants included comments on the training, variables and case definitions, and that software could have been more user-friendly. They suggested that the amount of training should be increased and that training could have included: recommendation on how to prepare for the survey; more examples of how to interpret the diagnostic criteria; and examples of completed data collection forms.
Pertaining to the variables, survey participants thought there were some unnecessary complexities in the study design, and that there could be fewer variables in general and in particular that the McCabe score could be removed. With respect to case definitions survey participants suggested three changes. First, that some laparoscopic procedures should be considered minimally invasive; second, that the classification of surgery be reduced to just two criteria – invasive and minimally invasive; and finally, that the criteria for the diagnosis of pneumonia be reviewed as the requirement for two abnormal X-rays was difficult outside intensive care units.
‘Validation of surveillance data is necessary to ensure its scientific credibility, to identify methodological problems within the surveillance programme, to help increase compliance and participation in the surveillance programme, and to identify data quality issues at local level’ (McCoubrey et al., 2005: 194). This survey allowed data collectors the opportunity to share their views on the feasibility and appropriateness of the data collection process for the European PPS validation study hence identifying data quality issues at the local level. They expressed the views that HAI data collection was the most difficult This is consistent with the findings of other PPS and validation studies (Duerink et al., 2006; Gastmeier et al., 1998; McCoubrey et al., 2005) where data collectors highlighted that HAI data were difficult to collect because they required the application of complex case definitions in circumstances where clinical information may be missing from patient notes (Gastmeier et al., 1998; McCoubrey et al., 2005) and where the required diagnostic tests had not been performed (Duerink et al., 2006; Gastmeier et al., 1998). However, participants in this survey also provided recommendation on how to reduce the difficulty of collecting HAI data. They recommended reviewing three case definitions, adding further content to the data collectors’ training programme and reducing the number of variables to be collected as ways of reducing the complexity of HAI data collection. The necessity for clear definition and training of data collectors to ensure reliable application of the criteria is also well documented in previous studies (Duerink et al., 2006; Fabry et al., 2007; Gastmeier et al., 1998; Liata et al., 2009; Sherman et al., 2006; Stewart et al., 2006).
Statistical tests performed during validation studies will demonstrate the validity and reliability of the data collected. Decisions about what data to collect should not rely solely on these measures. This survey had demonstrated that the practicalities of the availability and accessibility of the data are worthy of consideration. In addition, PPS validation studies and PPS studies occur in busy clinical environments while patient care is being delivered, therefore, measures to improve the efficiency of the process and reduce the time required for completion should be considered and the views expressed here by data collectors can contribute to this debate.
For their participating in the data collection for this study: Bulgaria: Rossitza Vatcheva-Dobrevska, Ivan Ivanov; Finland: Tommi Kärki; Germany: Petra Gastmeier; Hungary: Karolina Böröcz, Ágnes Hajdu; Italy: Silvio Brusaferro, Maria Luisa Moro, Luca Arnoldo; Latvia: Elina Dimina, Uga Dumpis; Lithuania: Jolanta Ašembergienė, Rolanda Valintėlienė; Poland: Aleksander Deptula, Waleria Hryniewicz; Spain: Jose Angel Rodrigo Pendas, Josep Rafart Vaqué.
Declaration of conflicting interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This survey was funded by the European Centre for Disease Prevention and Control [EDC. 2971].
Peer review statement: Not commissioned; blind peer-reviewed.