Background

Despite growing support for mixed methods approaches we still have little systematic knowledge about the consequences of combining surveys and focus groups. While the methodological aspects of questionnaire surveys have been researched extensively, the characteristics of focus group methodology are understudied. We suggest and discuss whether the focus group setting, as compared to questionnaire surveys, encourages participants to exaggerate views in a negative direction.

Discussion

Based on an example from our own research, where we conducted a survey as a follow up of a focus group study, and with reference to theoretical approaches and empirical evidence from the literature concerning survey respondent behaviour and small group dynamics, we discuss the possibility that a discrepancy in findings between the focus groups and the questionnaire reflects characteristics of the two different research methods. In contrast to the survey, the focus group study indicated that doctors were generally negative to clinical guidelines. We were not convinced that this difference in results was due to methodological flaws in either of the studies, and discuss instead how this difference may have been the result of a general methodological phenomenon.

Summary

Based on studies of how survey questionnaires influence responses, it appears reasonable to claim that surveys are more likely to find exaggerated positive views. Conversely, there are some indications in the literature that focus groups may result in complaints and overly negative attitudes, but this is still an open question. We suggest that while problematic issues tend to be under-communicated in questionnaire surveys, they may be overstated in focus groups.

We argue for the importance of increasing our understanding of focus group methodology, for example by reporting interesting discrepancies in mixed methods studies. In addition, more experimental research on focus groups should be conducted to advance the methodology and to test our hypothesis.

Background

Effective provider-parent communication can improve childhood vaccination uptake and strengthen immunisation services in low- and middle-income countries (LMICs). Building capacity to improve communication strategies has been neglected. Rigorous research exists but is not readily found or applicable to LMICs, making it difficult for policy makers to use it to inform vaccination policies and practice.

The aim of this project is to build research knowledge and capacity to use evidence-based strategies for improving communication about childhood vaccinations with parents and communities in LMICs.

Methods and design

This project is a mixed methods study with six sub-studies. In sub-study one, we will develop a systematic map of provider-parent communication interventions for childhood vaccinations by screening and extracting data from relevant literature. This map will inform sub-study two, in which we will develop a taxonomy of interventions to improve provider-parent communication around childhood vaccination. In sub-study three, the taxonomy will be populated with trial citations to create an evidence map, which will also identify how evidence is linked to communication barriers regarding vaccination.

In the project's fourth sub-study, we will present the interventions map, taxonomy, and evidence map to international stakeholders to identify high-priority topics for systematic reviews of interventions to improve parent-provider communication for childhood vaccination. We will produce systematic reviews of the effects of high-priority interventions in the fifth sub-study. In the sixth and final sub-study of the project, evidence from the systematic reviews will be translated into accessible formats and messages for dissemination to LMICs.

Discussion

This project combines evidence mapping, conceptual and taxonomy development, priority setting, systematic reviews, and knowledge transfer. It will build and share concepts, terms, evidence, and resources to aid the development of communication strategies for effective vaccination programmes in LMICs.

Background

Qualitative research is used increasingly alongside trials of complex interventions to explore processes, contextual factors, or intervention characteristics that may have influenced trial outcomes. Qualitative research conducted alongside trials can also be used to shed light on the results of systematic reviews of effectiveness by looking for factors that can help explain heterogeneous results across trials. In a Cochrane review on the effects of using lay health workers on maternal and child health and infectious disease control, we identified 82 trials. These trials showed promising benefits but results were heterogeneous.

Objective

To use qualitative studies conducted alongside these trials to explore factors and processes that might have influenced intervention outcomes.

Methods

We attempted to identify qualitative research carried out alongside the trials by contacting trial authors, checking papers for references to qualitative research, searching Pubmed for related studies, and carrying out citation searches. For those qualitative studies that we included, we extracted information regarding study objective, data collection and analysis methods, and key themes and categories.

Results

For 52 (63%) of the trials, we found no qualitative research that had been conducted alongside the trials. For 16 (20%) trials, some form of qualitative data collection had been done but was unavailable or had been done before the trial. For 14 (17%) trials, qualitative research had been done during or shortly after the trial, although descriptions of qualitative methods and results were often sparse. Most of these 14 studies aimed to elicit trial participants' perspectives and experiences of the intervention. A common theme was participants' appreciation of the lay health workers' shared circumstances, for instance with regard to social background or experience of the health condition. In six studies, researchers explored the experiences of the lay health workers themselves. Issues included the importance of regular supervision and health professionals' support or lack of support.

Conclusions

Qualitative studies carried out alongside trials of complex interventions could offer opportunities to authors of systematic reviews of effectiveness wishing to understand the heterogeneity of trial results. For interventions of lay health worker programmes at least, too few such studies exist at present for these opportunities to be realised.

Background

Focus group studies are increasingly published in health related journals, but we know little about how researchers use this method, particularly how they determine the number of focus groups to conduct. The methodological literature commonly advises researchers to follow principles of data saturation, although practical advise on how to do this is lacking. Our objectives were firstly, to describe the current status of sample size in focus group studies reported in health journals. Secondly, to assess whether and how researchers explain the number of focus groups they carry out.

Methods

We searched PubMed for studies that had used focus groups and that had been published in open access journals during 2008, and extracted data on the number of focus groups and on any explanation authors gave for this number. We also did a qualitative assessment of the papers with regard to how number of groups was explained and discussed.

Results

We identified 220 papers published in 117 journals. In these papers insufficient reporting of sample sizes was common. The number of focus groups conducted varied greatly (mean 8.4, median 5, range 1 to 96). Thirty seven (17%) studies attempted to explain the number of groups. Six studies referred to rules of thumb in the literature, three stated that they were unable to organize more groups for practical reasons, while 28 studies stated that they had reached a point of saturation. Among those stating that they had reached a point of saturation, several appeared not to have followed principles from grounded theory where data collection and analysis is an iterative process until saturation is reached. Studies with high numbers of focus groups did not offer explanations for number of groups. Too much data as a study weakness was not an issue discussed in any of the reviewed papers.

Conclusions

Based on these findings we suggest that journals adopt more stringent requirements for focus group method reporting. The often poor and inconsistent reporting seen in these studies may also reflect the lack of clear, evidence-based guidance about deciding on sample size. More empirical research is needed to develop focus group methodology.

Background

A recently updated Cochrane systematic review on the effects of lay or community health workers (LHWs) in primary and community health care concluded that LHW interventions could lead to promising benefits in the promotion of childhood vaccination uptake. However, understanding of the costs and cost-effectiveness of involving LHWs in vaccination programmes remains poor. This paper reviews the costs and cost-effectiveness of vaccination programme interventions involving LHWs.

Methods

Articles were retrieved if the title, keywords or abstract included terms related to 'lay health workers', 'vaccination' and 'economics'. Reference lists of studies assessed for inclusion were also searched and attempts were made to contact authors of all studies included in the Cochrane review. Studies were included after assessing eligibility of the full-text article. The included studies were then reviewed against a set of background and technical characteristics.

Results

Of the 2616 records identified, only three studies fully met the inclusion criteria, while an additional 11 were retained as they included some cost data. Methodologically, the studies were strong but did not adequately address affordability and sustainability and were also highly heterogeneous in terms of settings and LHW outcomes, limiting their comparability. There were insufficient data to allow any conclusions to be drawn regarding the cost-effectiveness of LHW interventions to promote vaccination uptake. Studies focused largely on health outcomes and did illustrate to some extent how the institutional characteristics of communities, such as governance and sources of financial support, influence sustainability.

Conclusion

The included studies suggest that conventional economic evaluations, particularly cost-effectiveness analyses, generally focus too narrowly on health outcomes, especially in the context of vaccination promotion and delivery at the primary health care level by LHWs. Further studies on the costs and cost-effectiveness of vaccination programmes involving LHWs should be conducted, and these studies should adopt a broader and more holistic approach.

Objective To examine the use of qualitative approaches alongside randomised trials of complex healthcare interventions.

Design Review of randomised controlled trials of interventions to change professional practice or the organisation of care.

Data sources Systematic sample of 100 trials published in English from the register of the Cochrane Effective Practice and Organisation of Care Review Group.

Methods Published and unpublished qualitative studies linked to the randomised controlled trials were identified through database searches and contact with authors. Data were extracted from each study by two reviewers using a standard form. We extracted data describing the randomised controlled trials and qualitative studies, the quality of these studies, and how, if at all, the qualitative and quantitative findings were combined. A narrative synthesis of the findings was done.

Results 30 of the 100 trials had associated qualitative work and 19 of these were published studies. 14 qualitative studies were done before the trial, nine during the trial, and four after the trial. 13 studies reported an explicit theoretical basis and 11 specified their methodological approach. Approaches to sampling and data analysis were poorly described. For most cases (n=20) we found no indication of integration of qualitative and quantitative findings at the level of either analysis or interpretation. The quality of the qualitative studies was highly variable.

Conclusions Qualitative studies alongside randomised controlled trials remain uncommon, even where relatively complex interventions are being evaluated. Most of the qualitative studies were carried out before or during the trials with few studies used to explain trial results. The findings of the qualitative studies seemed to be poorly integrated with those of the trials and often had major methodological shortcomings.

Background

GPs' adherence to clinical practice guidelines is variable. Barriers to guideline implementation have been identified but qualitative studies have not been synthesised to explore what underpins these attitudes.

Aim

To explore and synthesise qualitative research on GPs' attitudes to and experiences with clinical practice guidelines.

Design of study

Systematic review and meta-synthesis of qualitative studies.

Method

PubMed, CINAHL, EMBASE, Social Science Citation Index, and Science Citation Index were used as data sources, and independent data extraction was carried out. Discrepancies were resolved by consensus. Initial thematic analysis was conducted, followed by interpretative synthesis.

Results

Seventeen studies met the inclusion criteria. Five were excluded following quality appraisal. Twelve papers were synthesised which reported research in the UK, US, Canada, and the Netherlands, and covered different clinical guideline topics. Six themes were identified: questioning the guidelines, GPs' experience, preserving the doctor–patient relationship, professional responsibility, practical issues, and guideline format. Comparative analysis and synthesis revealed that GPs' reasons for not following guidelines differed according to whether the guideline in question was prescriptive, in that it encouraged a certain type of behaviour or treatment, or proscriptive, in that it discouraged certain treatments or behaviours.

Conclusion

Previous analyses of guidelines have focused on professional attitudes and organisational barriers to adherence. This synthesis suggests that the purpose of the guideline, whether its aims are prescriptive or proscriptive, may influence if and how guidelines are received and implemented.

Background

Evidence-based decision making relies on easy access to trustworthy research results. The Cochrane Library is a key source of evidence about the effect of interventions and aims to "promote the accessibility of systematic reviews to anyone wanting to make a decision about health care". We explored how health professionals found, used and experienced The Library, looking at facets of user experience including findability, usability, usefulness, credibility, desirability and value.

Methods

We carried out 32 one-hour usability tests on participants from Norway and the UK. Participants both browsed freely and attempted to perform individually tailored tasks while "thinking aloud". Sessions were recorded and viewed in real time by researchers. Transcriptions and videos were reviewed by one researcher and one designer. Findings reported here reflect issues receiving a high degree of saturation and that we judge to be critical to the user experience of evidence-based web sites, based on principles for usability heuristics, web guidelines and evidence-based practice.

Results

Participants had much difficulty locating both the site and its contents. Non-native English speakers were at an extra disadvantage when retrieving relevant documents despite high levels of English-language skills. Many participants displayed feelings of ineptitude, alienation and frustration. Some made serious mistakes in correctly distinguishing between different information types, for instance reviews, review protocols, and individual studies. Although most expressed a high regard for the site's credibility, some later displayed a mistrust of the independence of the information. Others were overconfident, thinking everything on The Cochrane Library site shared the same level of quality approval.

Conclusion

Paradoxically, The Cochrane Library, established to support easy access to research evidence, has its own problems of accessibility. Health professionals' experiences of this and other evidence-based online resources can be improved by applying existing principles for web usability, prioritizing the development of simple search functionality, emitting "researcher" jargon, consistent marking of site ownership, and clear signposting of different document types and different content quality.

Background

In an evidence-informed patient choice the patient has access to research-based information about the effectiveness of health care options and is encouraged to use this information in treatment decisions. This concept has seen growing popularity in recent years. However, we still know relatively little about users' attitudes to the use of research-based information, possibly because people have been unexposed to this type of information. After developing the BackInfo website where the results of Cochrane systematic reviews on the effects of low back pain were adapted and presented to lay users we evaluated how users responded to this information.

Methods

Focus group meetings were held with 18 chronic back pain sufferers, after they had been sent a link to the website before the meetings.

Results

The focus groups suggest that the most important challenges to the use of BackInfo's research-based information are not primarily tied to the comprehension or presentation of the information, but are mainly associated with participants' attitudes towards the credibility of research and researchers, and the applicability of research results to themselves as individuals. Possible explanations for participants' lack of trust in research and their apparent difficulties in applying this research to their own situations include aspects that may be typical for the general public including the media's presentation of research, and a lack of familiarity with and feelings of distance to research evidence. Other aspects may be typical for patient groups with chronic and unclear medical conditions, such as a lack of trust in the health care establishment in general.

Conclusion

In order to enhance the credibility and applicability of research evidence, providers of research-based information could explore a number of possibilities including the use of including personal stories to illustrate the research outcomes.

Background

The Internet offers a seemingly endless amount of health information of varying quality. Health portals, which provide entry points to quality-controlled collections of websites, have been hailed as a solution to this problem. The objective of this study is to assess the extent to which government-run health portals provide access to relevant, valid and understandable information about the effects of health care.

Methods

We selected eight clinically relevant questions for which there was a systematic review, searched four portals for answers, and compared the answers we found to the results of the systematic reviews.

Results

Our searches resulted in 3400 hits, 155 of which mentioned both the condition and the intervention in one of the eight questions. Sixty-three of the 155 web pages did not give any information about the effect of the intervention. Seventy-seven qualitatively described the effects of the intervention. Twenty-six of these had information that was too unclear to be categorised; 15 were not consistent with the systematic review; and 36 were consistent with the review, but usually did not mention what happens without the intervention, what outcomes have been measured or when they were measured. Fifteen web pages quantitatively described effects. Four of these were abstracts from the systematic review, nine had information that was incomplete and potentially misleading because of a lack of information about people not receiving the intervention and the length of follow-up; one had information that was consistent with the review, but only referred to three trials whereas the review included six; and one was consistent with the review.

Conclusion

Information accessible through health portals is unlikely to be based on systematic reviews and is often unclear, incomplete and misleading. Portals are only as good as the websites they lead to. Investments in national health portals are unlikely to benefit consumers without investments in the production and maintenance of relevant, valid and understandable information to which the portals lead.

A systematic search of seven electronic databases was done to identify randomized controlled trials that assessed the effect of computer-generated patient education material (PEM) on professional practice. Three studies met the authors' criteria.

All three studies involved preventive care. All used a complex intervention of which computer-generated PEM was a major component. Improvements in practice were seen in all studies, although these gains were generally modest. One study showed improvement in patient outcomes. Mann-Whitney statistics calculated for the studies' outcome measures ranged from 0.48 to 0.66, equivalent to risk differences of −4 to 32 percent.

Computer-generated PEM seems to have a small, positive effect on professional practice. The small number of included studies and the complex nature of the interventions makes it difficult to draw conclusions about the ability of computer-generated PEM to change professional practice. Future work should involve well-defined interventions that can be clearly evaluated in terms of effect and cost.