|Home | About | Journals | Submit | Contact Us | Français|
Previous evaluation showed insufficient use of a national guideline for integrated local health policy by Regional Health Services (RHS) in the Netherlands. The guideline focuses on five health topics and includes five checklists to support integrated municipal health policies. This study explores the determinants of guideline use by regional Dutch health professionals.
A web survey was send to 304 RHS health professionals. The questionnaire was based on a theory- and research-based framework of determinants of public health innovations. Main outcomes were guideline use and completeness of use, defined as the number of health topics and checklists used. Associations between determinants and (completeness of) guideline use were explored by multivariate regression models.
The survey was started by 120 professionals (39%). Finally, results from 73 respondents (24%) were eligible for analyses. All 28 Dutch RHS organizations were represented in the final dataset. About half of the respondents (48%) used the guideline. The average score for completeness of use (potential range 1–10) was 2.37 (sd = 1.78; range 1–7). Knowledge, perceived task responsibility and usability were significantly related to guideline use in univariate analyses. Only usability remained significant in the multivariate model on guideline use. Only self-efficacy accounted for significant proportions of variance in completeness of use.
The results imply that strategies to improve guideline use by RHSs should primarily target perceived usability. Self-efficacy appeared the primary target for improving completeness of guideline use. Methods for targeting these determinants in RHSs are discussed.
The online version of this article (doi:10.1186/s12913-017-2499-2) contains supplementary material, which is available to authorized users.
The development and implementation of public health policies in the Netherlands is largely delegated to local authorities. In this process, municipalities are supported by Regional Health Services (RHSs). RHSs focus on three main prevention areas of Infectious Disease, Youth Health, and lifestyle related Health Promotion. Typical RHSs professions are doctors, nurses, health promoters, health scientists, policy officers, and epidemiologists. Their work consists of directly client-oriented activities (e.g. information on sexually transmitted diseases; intervention for obesity prevention, health education in schools), advisory for policy development and of collecting statistical information (monitoring of regional - and local trends in health and disease) to provide input for regional - and local policy advice. RHSs can either have a regional scope and serve multiple municipalities or serve a single (urban) municipality.
Since 2006, the Dutch Ministry of Health has equipped municipalities and RHSs with national guidelines for the planning and implementation of their public health policies . Four different guidelines, incorporating recommended interventions to address smoking , obesity , alcohol abuse , and depression  were issued separately and were published sequentially within a period of two years. Preliminary evaluation indicated unsatisfactory results concerning the guidelines’ implementation and led to a revised, more extended and comprehensive Guideline for Local Health Policy (hereafter ‘guideline') . The new guideline integrated the four separate guidelines, and added the topic ‘sexual health’, and new tools (checklists) for developing cross-sectoral public health policies. The overall purpose of the guideline is to stimulate the use of evidence in this planning process . The guidelines’ health topics and checklists serve different, though related purposes. The health topics are about selection and application of exemplary interventions, while the checklists provide leads for improving collaboration and commitment among those participating in the planning of local health policies. RHS policy officers are called upon to use the checklists containing leads for health policy planning, and evaluation. They may also use practical examples of support based collaboration between municipal departments and partner organizations, which describe do’s and don’ts for reaching commitment and shared priority setting among public health parties. The guideline supports health promoters with evidence based interventions for (e.g.) obesity and depression, and offers formats for setting up regional programs for specific health risks, such as the prevention of alcohol abuse. Successful implementation of the guideline can be defined as ‘improved local collaboration in projects and programs for integrated health’ (e.g. environmental planning and stimulating physical exercise). In addition, the target population will be better reached by interventions based on evidence. The Guideline offers RHS organizations new methods that challenge their professionals to practice specific (partly new) competencies and skills.
This research aims to answer two questions:
This study wants to gain leads for improving local public health policies that fit within the structures and workflow of local health organizations and their cross sectoral networks . International research provides an extensive range of knowledge when it comes to barriers and facilitators for the implementation of clinical guidelines. Determinants have been found such as professionals’ views and beliefs , outcome expectancies of an innovation , knowledge and attitudes towards change of practice , self-efficacy beliefs , and social- and organizational support . Less has been written about the determinants for guideline implementation within the political-administrative context of public health [14, 15].
Clinical guidelines often target rather homogeneous professional groups (e.g. doctors, paramedics) which is different from the implementation of guidelines for local health policy by a network of organizations. Besides, clinical guidelines predominantly prescribe a step-by-step treatment of a patients’ specific disorder with a specified outcome. The adoption process usually takes place in a hierarchical organizational context that often leaves little or no room for personal interpretation and flexibility. In contrast, the implementation of policy guidelines for local health needs to build coalitions among various organizations, each having their own interests, priorities and perceptions about the means by which public health goals are best achieved. Compared to a clinical setting, the process of adoption of policy guidelines in public health requires more negotiations among network partners for reaching consensus about shared goals and their investment for reaching these goals . As such, cross-sectoral collaboration in public health policy making requires a more horizontal basis with input from the participants’ calculations of their own costs-benefit ratio [17, 18].
Overall, the implementation literature since Matland (1995)  has come to a consensus about the need for combining both top-down and bottom-up strategies in order to account for the local conditions for guideline implementation, such as available resources, specific interests of coalitions, partners, activities already implemented and the distribution of power .
This research focuses on the guideline’s implementation by RHS policy advisors and health promoters. In order to assess implementation barriers and facilitators, we constructed a research framework of potentially relevant determinants of guideline use. The theory- and research-based framework MIDI (Measurement Instrument for Determinants of Innovations) was used as a point of reference for the framework for this study [11, 20]. MIDI offers an overview of potentially relevant determinants of public health innovations and leads for assessment. The framework for this study consisted of proximal determinants (e.g. task responsibility), which are expected to impact guideline use directly. The selected, distal determinants (e.g. years of working experience) are expected to be mediated by the proximal factors. The research framework was further refined by premises from Rogers’ diffusion of innovations theory , Bandura’s social cognitive theory (i.e. Self-efficacy theory) [12, 21], policy theory [22–24], organization theory [25, 26], and by the results of a recent Dutch study among key informants about local public health implementation processes (i.e. RHS professionals, RHS managers, public health experts, municipal policy officers, and guideline developers) . The framework applied in this study is presented in Fig. Fig.11.
Determinants, as described in the research framework, were assessed by one or more questionnaire items. They were assessed as either dichotomous, continuous, counts, or by Likert-type scaled items). Composite scores were computed when acceptable levels of internal consistency (Cronbach’s alpha ≥0.60) were reached. Composite scores were computed by dividing the sum score by the number of items. Negatively formulated items were flipped first, whenever necessary. Table Table11 provides an overview of the Alpha scores.
‘Guideline use’ was defined as whether or not the respondent used any of the health topics and/or checklists. ‘Completeness of use’ was defined as the number of the health topics (range 1–5) and/or checklists (range 1–5) included in the guideline, that were put into practice. Because ‘guideline use’, and ‘completeness of use’ are quite distinct phenomena, it was expected that they could be affected by different factors.
‘Outcome expectancy’ was assessed as the product of perceived importance and feasibility of the guidelines’ key objectives. We used 14 5-point Likert scaled items. ‘Self-efficacy’ was assessed by two five-point scaled items. ‘Task responsibility’ contained three items, such as ‘The guideline contains activities that fit my job within the RHS’. For all Likert scaled items, response options ranged from 1 = strongly agree to 5 = strongly disagree. We used 19 items to assess ‘usability’.
Measurement of ‘organizational readiness’ contained six 5-point scaled items. We used four 5-point scaled items for measuring social influences. ‘Peer interaction’ referred to the number (1–8) of different meetings in which the guideline’s implementation was discussed. (e.g. a section meeting of RHS policy officers, or a meeting between RHS manager and municipal city councilor). ‘Participative decision making’ about the guideline’s adoption was assessed dichotomously by two items, such as ‘Were executive professionals involved in the decision whether or not to use the guideline within the RHS?’).
Perceived ‘social and administrative legitimacy’ was measured by nine 5-point scaled items, such as ‘Our municipalities encourage to work in accordance with the recommendations of the guideline’.
‘Organizational background characteristics’ referred to RHS type (urban vs. regional). ‘RHS’s imago’ was assessed by two 5-point scaled items such as ‘My RHS has a positive image among the municipalities’. To assess the policy support orientation of the RHS, we used one 5-point scaled statement. ‘Professional occupation’ required one answer (tick box: executive health promoter, health policy officer, team leader, manager, and other). We used an open gray box for ‘working experience’ (in years), and five options for ‘perceived work pressure’ (1 = too low, 2 = low, 3 = moderate, 4 = high, 5 = too high). For measuring ‘subjective knowledge of the guideline’, we used three 5-point scaled questions (e.g. ‘I know where to find the guideline’). ‘Objective knowledge’ (of guideline content) was assessed by five 3-point scaled items, like ‘The guideline contains methods for social marketing to enhance political - and administrative base of support’ (1 = yes, 2 = no, 3 = don’t know).
(See Additional file 1: Appendix 1 with questionnaire variables and questions)
First we assessed the univariate associations between guideline use and the determinants in our framework by means of Chi-squared tests for nominal variables, non-parametric Mann-Whitney U Test for ordinal variables, and T-tests for continuous variables. Only determinants associated with guideline use (p < 0.1, two sided) were entered in the multivariate logistic analysis (forward selection). Finally, mean-score differences between users versus non-users of the guideline were computed for each individual item of the determinant(s) in the final multivariate model.
A similar stepwise multivariate linear regression approach was used for analyzing the determinants of completeness of use.
The questionnaire was sent to 304 regional public health professionals, and was returned by 120. For identification of potential respondents we used the national RHS department address files. These files contained addresses of RHS functions (such as managers and former employees) who were not meant to (and actually did not) respond to the questionnaire. A check with all RHSs revealed that RHS organizations had insufficient insight into the exact number of policy advisors. Respondents with functions other than policy advisor or health promoter, 14 in total, were excluded from analysis. Also excluded were 33 respondents who returned incomplete questionnaires (i.e. without information on primary outcomes). Our final dataset included 73 complete cases and all (28) Dutch RHS organizations were represented by at least one professional.
Of all respondents, 35 used the guideline, 38 did not. Among the 35 respondents who used the guideline, thirty-one respondents had used at least one health topic. The topic most often used was obesity (n = 19), followed by alcohol (n = 11), smoking (n = 6), sexual health (n = 6) and depression (n = 4). The checklists for policy planning were used by 14 respondents. Use of checklists was highest for checklist B (policy preparation; n = 10), followed by C (policy formulation; n = 9), E (policy preconditions; n = 7), D (policy execution and evaluation; n = 6) and A (policy context; n = 5). These results showed that, except for the topic ‘obesity’, guideline use by RHS professionals was rather moderate.
Of the user group, 18 used only one public health topic or one checklist and 17 used 2 to 7 topics or checklists. The average score for completeness of guideline use was 2.37 (sd = 1.78; range 1–7).
Cronbach’s alpha ≥0.60 was used as cut off point for internal consistency of the composite variables. These internal consistency checks led to two adaptations: 1) the two self-efficacy items didn’t correlate well enough (alpha 0.46) and were therefore analyzed as separate factors (Self-efficacy A: ‘The guideline contains methods and tasks which I can actually perform’; Self-efficacy B: ‘I don’t think I can exchange my own routines with the new methods prescribed by the guideline), and 2) ‘Organizational readiness’, (9 items, alpha = 0.50) was split into two subscales, labeled as ‘Encouragement’ (referring to the presence of deliberate activities to promote guideline use) and ‘Organizational readiness’ (referring to the presence of interdisciplinary communication and sharing of knowledge and aspirations on integrated health targets in the RHS’s hierarchy). The alpha scores for the final constructs ranged from 0.61 to 1.0 (Table (Table11).
The univariate associations found between determinants and guideline use are presented in Table Table2.2. Of the background characteristics, only subjective knowledge and objective knowledge appeared associated with guideline use (p < 0.10). Of the proximal and distal determinants, only perceived task responsibility and usability were significantly related to guideline use (p < 0.05). The inter-correlation between these two determinants appeared to be moderately high: r = 0.65 (p < 0.001).
When objective knowledge, subjective knowledge, task responsibility, and usability were entered in the multivariate logistic model according to their theoretically expected order (forward selection), only ‘usability’ remained in the final model with OR 5.86 (1.68–20.5). The model fit (proportion of explained variance) appeared rather weak (Nagelkerke R Square 0.17).
For more in-depth insight into usability, as determinant of guideline use, we assessed the mean score differences of the 19 usability items between users versus non-users (Mann-Whitney U-tests). Table Table33 only shows the mean scores differences that were statistically significant at p < 0.05.
Beliefs showing relatively high scores among both users and non-users referred to effective collaboration with other sectoral policies on the health topics covered by the guideline (mean = 4.41), and perceived fit with current national policies, regulations and laws (mean = 4.14). We found relatively low scores on perceived procedural clarity of the guideline (mean = 3.67) and on the number of examples to work on your own with the guideline (mean = 3.71).
The largest differences between users and non-users were found in their perception of how well the guideline is based in science, and their perceived clarity of the leads offered by the guideline for developing local health policy.
Univariate analyses showed that completeness of use was only significantly associated with Self efficacy A (‘The guideline contains methods and tasks which I can actually perform’; Spearman’s rho = 0.36; p < 0.05) and Self efficacy B (‘I don’t think I can exchange my own routines with the new methods prescribed by the guideline’; Spearman’s rho = 0.44; p < 0.01). The inter-correlation of the two self-efficacy items was rather strong (Spearman’s rho = 0.54; p < 0.001). The multivariate linear regression on completeness of use (forward selection) showed that only Self efficacy B entered the model (β = 0.43, 95% CI: 0.24–1.70). The model fit (proportion of explained variance) was weak (R Square 0.18).
The main goal of this study was to explore the determinants of implementation of a public health policy guideline within Dutch RHSs, since these should be the primary target for strategies aiming to improve implementation.
The questionnaires of 73 respondents (24% out of 304 health professionals approached) appeared eligible for analysis.
About half of these respondents reported to use the guideline. The guideline was most often used within the context of the prevention of obesity. This corresponds to the relatively high priority of obesity prevention in both national and local public health policies in the Netherlands .
In the univariate analysis of guideline use, we found ‘knowledge’, ‘task responsibility’ and ‘usability’ (procedural clarity), and Self-efficacy to be related to the use versus non-use of the guideline. The analysis of determinants of guideline use also showed subjective and objective knowledge to be associated with guideline use (Table (Table11).
To improve implementation of the guideline, dissemination of knowledge about the guideline should be improved in municipalities and in regional health services. This was also confirmed by the interview results indicating that not all professionals and managers were aware about both the availability and the guideline’s core objectives. Besides media exposure, such as articles in professional journals, presentations online or at conferences, awareness can be increased by interpersonal communication. The latter provides the opportunity for exploring alternative plans for implementation that are tailored to the characteristics of the local setting in which the municipality and regional health service operate. The planning process should account for shared decision making by professionals and management within and between the local municipality and regional health service . The implementation plan should clarify how application of the guideline fits with the current organization perspectives, vision and still existing methods and tools. Besides, the planning should account for feedback on progress, technical support, and training . Internal communication and collegial interactions can be further initiated via online news channels and the organizations intranet.
The results showed differential perceptions among the professionals concerning their ‘task responsibility’ with regard to using the guideline. These differences reflect insufficient correspondence between the guideline-related tasks and objectives and their own perception of their professional task-obligations. If not, the outcome might as well express some sort of defensive response of those experiencing uncertainty about their competence relative to the execution of particular guideline- related tasks. Nevertheless, implementation of the guideline can be expected to improve by: 1) maximizing procedural clarity about the professionals’ core tasks and responsibilities within the context of the guideline ; 2) aiming at consensus among the professionals and managers within the RHS on tasks for which both disciplines are to be held responsible; 3) (individual) coaching and feedback on progress during the stage that the guideline is put into practice .
As yet, the results for the determinant ‘usability’ (including ‘procedural clarity’) indicate that the guideline does not provide the professionals with enough clarity about guideline-related tasks and responsibilities. Besides, the respondents differed in the extent to which they expressed their need for more explicit guidance and clarity, irrespective of their perceived importance of implementing the guideline.
This may reflect differences in perceived mastery of the professional skills involved when implementing the guideline as intended. This provisional conclusion is congruent with the overall low ‘self-efficacy’ scores we also found. Self-efficacy beliefs can be increased by ‘vicarious learning’: watching role models practicing the intended course of action . This can be accomplished virtually, for example within a training session or by watching a video, and in practice, for example when junior professionals watch seniors performing the intended task. In addition, coaching can help to ensure that professionals gradually gain confidence in executing new tasks. This is also supported by literature on improving self-efficacy beliefs within the context of implementation of guidelines .
Our conclusions are only tentative, because they are based in cross-sectional data and a relatively low number of cases. The response was lower than expected and 47 respondents could not be included in the analyses. Non-response was partly due to the timing of the survey, which was conducted fairly short (nine months) after publication of the (renewed) guideline. For some Regional Health Services, there was no or at least limited opportunity to incorporate the guideline because of the 4-year life cycle of the planning of regional public health policy. So, some regions were in the mid-term of executing their previously planned strategy and were not yet ready for preparation of the proceeding strategy period. This would have been compensated, at least partially, if we had assessed intentional use in the near future.
Selection bias may have affected some of the outcomes if respondents who had used the guideline would have been more willing to complete the questionnaire. In that case the descriptive statistics (percentages and averages) could be biased. However, the main question of this research was to explore associations between variables which are known to be less vulnerable for selection bias .
The amount of explained variance may have been affected by the low number of items used for the assessment of particular constructs. Our intention to develop a questionnaire (based on our research framework) that was feasible to complete within a restricted timeframe, may have been at the expense of the stability of some assessments, especially those based in a single item. Also the scope of the criterion ‘completeness’ (of use) is not the optimum when thinking about guideline implementation as intended by the developers. Implementation is more than just the number of themes and/or checklists used in practice, for example it does not account for the number of relevant others (in or outside their own organization) also using the guideline neither for the quality of implementation.
The results of our analyses indicate that knowledge, perceived task responsibility and beliefs about the guideline’s ‘usability’ are best discriminating professionals who use and not use the guideline. Hence, these are primary targets for improving the implementation of the Guideline for Local Health Policy. For improving completeness of guideline use, attention should be given to the RHS professionals’ self-efficacy.
We thank all participants from RHSs for their collaboration and contributions to the survey results. We thank the Dutch RHS Association for practical support for distribution of the questionnaire. We thank the Netherlands Organization for Health Research and Development (ZonMw) for financially supporting this project.
This study was funded by the Netherlands Organization for Health Research and Development (ZonMw, grant number 201000001).
Web survey and data copy of our analysis (SPSS syntax) are available on demand and will be provided via the corresponding author of this manuscript.
Additional file 1:(50K, docx) BMC HSR Appendix 1′. Appendix 1 Determinants, outcomes and corresponding questionnaire items. The file provides determinants and outcomes with the questionnaire items on which they are based. (DOCX 50 kb)
BMC HSR Appendix 1′. Appendix 1 Determinants, outcomes and corresponding questionnaire items. The file provides determinants and outcomes with the questionnaire items on which they are based. (DOCX 50 kb)
TP, TK and MJ contributed to the design, execution and data analyses of the study. TK, MJ, and TP wrote the manuscript and provided comments during the analysis and the writing process of the manuscript. LvdG., HvO, and MvB revised and commented on the manuscript. All authors read and approved the final manuscript.
Theo J.M. Kuunders works as a researcher and science practitioner for the Department Tranzo, Scientific Center for Care and Welfare, Tilburg University. As a policy officer for a Regional Health Service, he is specialized in network policy development for RHS, municipalities and primary healthcare. His current research activities concern the implementation of national guidelines on health promotion at the local level.
Monique A.M. Jacobs - van der Bruggen (PhD) is an epidemiologist at the RHS ‘GGD Hart voor Brabant’ and works as a science practitioner at the Department Tranzo, Scientific Center for Care and Welfare, Tilburg University. From 2005 to 2010 most of her research focused on (cost) effectiveness of diabetes prevention. In 2010 she finished her thesis: Opportunities for diabetes prevention. She is currently involved in the evaluation of overweight prevention projects for children and monitoring of new tasks for municipalities in youth care.
Ien A.M. van de Goor is professor of Public Health at Tilburg University. She is senior researcher and program leader for the “Effectiveness of Individual Prevention-program.” She has expertise in initiating and supervising research projects in healthcare and welfare. She is specialized in studying prevention and lifestyle interventions, needs for care and support in clients and patients; evaluation research of health care and health promotion interventions; and studies in addiction, addiction care and treatment.
Marja J.H. van Bon-Martens (PhD), epidemiologist, works as senior scientific staff member at the Trimbos Institute, Netherlands Institute of Mental Health and Addiction, and as senior researcher at the Department Tranzo, Scientific Center for Care and Welfare, Tilburg University. She has wide managerial expertise in bringing evidence into policy and practice through implementation of epidemiological research, and in supporting health policy making.
Hans A.M. van Oers (Professor) is trained as a public health epidemiologist. Currently he is Chief Science Officer of System Assessment for Policy Support at the Dutch National Institute of Public Health and the Environment, and professor of Public Health at Tilburg University. His main research interests are in the field of public health epidemiology and public health policy making.
Theo G.W.M. Paulussen (PhD) is a health scientist with specific expertise in health promotion and education. He graduated from Maastricht University and his PhD thesis was about implementation of health education innovations. He is affiliated with TNO Innovation for Life, and Expertise Group Life Style in Leiden, Netherlands.
The respondents of all Regional Health Services voluntarily took part in the survey. Prior to their participation, respondents were informed that their contributions would be made anonymous and would not be reducible to individuals or individual RHS organizations. On the basis of these conditions and prior to the execution of the survey, respondents agreed to take part and gave informed consent to use the survey results in publications. Approval by the ‘Dutch Medical Research Involving Human Subjects Act’ was therefore not applicable.
The Guideline for Local Health Policy is designed by collaborating national health institutes and the National Institute for Public Health and the Environment. All authors declare that they have no (financial or non-financial) competing interests in relation to this manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
The online version of this article (doi:10.1186/s12913-017-2499-2) contains supplementary material, which is available to authorized users.
Theo J.M. Kuunders, Phone: +31 13 466 3476, Email: firstname.lastname@example.org.
Monique A.M. Jacobs, Email: email@example.com.
Ien A.M. van de Goor, Email: firstname.lastname@example.org.
Marja J.H. van Bon-Martens, Email: ln.sobmirt@nobm.
Hans A.M. van Oers, Email: email@example.com.
Theo G.W.M. Paulussen, Email: firstname.lastname@example.org.