After several years of conducting qualitative research as well as mixed methods research, combining broad surveys with in depth focus group studies, we have started reflecting upon an apparent tendency for focus groups to convey a more negative view of the topics in question than the views conveyed through surveys. Revisiting the material from our earlier focus group studies, we note that some participants have had similar thoughts when they have been asked to assess the focus group experience. Below is an extract from a focus group discussion among general practitioners where they comment on their own presentation of their relationship with hospital psychiatrists (study described in [1
"K: Well, we are painting a rather gloomy picture here."
"G: I think that we are kind of coming up with all the bad stuff. There are a lot of people who manage fine."
"Several participants: Mmmm. Yes."
"K: And not everyone we meet in the hospital sector is that difficult."
"G: Yes, some of them are really nice."
These kinds of statements, combined with a comparison of survey results, have made us speculate: Could the focus group as a method be biased towards negative findings?
Quantitative and qualitative methods have complementary strengths and weaknesses, and researchers are therefore increasingly encouraged to mix these different approaches [2
]. The manner in which (qualitative) focus group studies can enhance the validity and value of (quantitative) surveys has been described with particular enthusiasm [7
]. Focus groups allow the researcher to get a more complex and complete picture of a phenomenon and is seen as a useful basis when developing valid questionnaires for surveys [9
]. Surveys, on the other hand, can serve to map the distribution of findings from focus group studies.
Despite growing support for mixed methods approaches we still have little systematic knowledge about the consequences of combining research methods in general, and focus groups and surveys specifically [10
]. Because of a lack of empirical research on focus group methodology, we know far less about the mechanisms that characterise focus group discussions than we know about respondents’ reactions to questionnaire surveys. Psychology and the social sciences have a long tradition of survey methodology research [11
], and already in the early 1970s, Sudman and Bradburn [14
] were able to include more than 800 sources in their review of studies of survey response effects. However, with a few exceptions [15
], no such interest has been given to the advancement of focus group methodology [10
]. Some researchers have described how focus group studies utilize small group dynamics to extract other types of and additional information to surveys, but authors disagree about the kind of information this involves and how to best combine methods [3
]. As Morgan and others have repeatedly pointed out, assertions about how focus groups work best and their limitations are usually based on intuition and personal experience rather than empirical evidence [3
As a basis for our discussion, we briefly present an example from our own research where our use of both focus groups and a questionnaire survey led to results that appeared to be pointing in different directions and where the focus group findings conveyed a much more negative view than the survey. The findings, which were reported in separate publications [21
], appeared plausible and were supported by other studies on the same topic and using the same methodology. In addition, important issues that were raised in the focus group study seemed to be confirmed as valid by the survey. It was not until we compared the main messages from our two studies that we were struck by the discrepancy, and even conflict between them. With reference to existing research on survey respondent behaviour and small group dynamics we discuss these differences; and we suggest how more research on focus group methodology is needed to further our understanding.
Methods and results of the mixed methods study
In our first study we conducted focus groups with Norwegian general practitioners (GPs). The aim of the focus group study was to elicit GPs’ attitudes to clinical guidelines and guideline adherence. In the second study, we conducted a survey where key postulations were directly drawn from the focus group discussions. The survey aimed to explore the importance and distribution of attitudes to guidelines that we had identified in the focus group study. Further details of both parts of the study have been reported elsewhere [21
The focus groups
We conducted six focus groups with a total of 27 GPs in 2007. Our sampling strategy was a mixture of convenience and purposive sampling. We mailed a general invitation to participate in the study to the leaders of 93 doctors' educational groups. A majority of Norwegian GPs participate in these groups during their careers, either to obtain a specialist certification in general practice or to maintain the specialist competence. Eleven educational groups responded to the invitation and we selected six of these groups with a total of 27 members. The six groups were chosen in order to achieve a sample that was fairly similar to the population of GPs with regard to age, gender, professional experience, list size, urbanisation, and patient populations of different socioeconomic levels. Interviews were carried out until no new themes occurred according to a continuous and preliminary analysis.
The participants in each group knew each other from regular group meetings. The focus groups were moderated by one of the authors (BC), who presented herself at the start of each group session as a social scientist funded by the Research Council of Norway, and informed participants about the motivation of the study and anonymity issues. The interview guide specified some overarching themes, including participants’ confidence in and adherence to clinical guidelines, and how such guidelines influence professional autonomy and shared decision making. The moderator asked the participants initially to define what they meant by guidelines, encouraged participants to discuss freely, and only probed to clarify and to secure that all the predefined themes were discussed. The moderator aimed at conveying a neutral stance towards the role of clinical guidelines in general practice. Participants were asked specifically for both positive and negative views and examples, but the moderator consistently began by asking participants to discuss what they perceived to be positive aspects of guidelines.
We applied thematic content analysis [25
] to identify common themes and arguments. Two researchers read the transcripts from the discussions thoroughly and then discussed emergent themes and possible codes until agreement was reached.
A total of 27 GPs, 18 men and 9 women with a mean age of 45
years, participated in the focus groups. In comparison, the mean age in the national population of GPs is 47. The sex ratio and size of patient lists (work load) were similar to those of the study sample [26
We registered lively discussions in the groups; the moderator did not dominate the discussions and talked less than any single participant. The GPs referred primarily to a limited number of well-known guidelines, including guidelines for primary prevention of cardiovascular disease, diabetes, antenatal care, COPD/asthma and mammography screening. In line with earlier international studies, several barriers to following guidelines emerged, including a lack of trust in the evidence behind the guidelines, a desire to adjust treatment to the needs of the individual patient, and different practical challenges. The most striking and consistent message focus group participants gave was one of general scepticism towards guidelines and their authors. The doctors were particularly suspicious of health authorities’ possible economic motives behind their guidelines, and this was presented as a reason for non-adherence.
Our main impression from the focus group interviews was that negative views dominated, and participants’ negative statements were both longer and more emotional than their positive remarks. This impression was based on our observations during the interviews, on further interpretations while listening to the recordings of the interviews and on our analyses of the transcriptions. In addition, we coded and quantified what we perceived to be positive and negative remarks about guidelines, and counted 69 positive and 46 negative remarks. While the strength of participants’ scepticism is difficult to convey, the statement below, where a doctor describes his relationship to the health authorities, captures some of the atmosphere:
"Dr N: To me, it’s essentially about feeling part of a team. I don’t feel we are part of a team, we are opponents. We each have our own team and we are doing the best we can, at least I am. I’m not so sure about them. But I don’t feel they are very interested in working with me or with us as a group. That’s very negative of me, but it’s how I feel."
The survey was carried out a year later, in 2008. Using the key findings from the focus group study, we developed a questionnaire to explore how the attitudes we had identified were distributed among Norwegian GPs. We asked about adherence to and confidence in clinical guidelines in general, presenting a broad definition of guidelines, and did not mention any particular guideline as an example. We also asked about confidence in guideline authors, such as the health authorities. Key postulations that the respondents were asked about their agreement with were:
·I have good knowledge of guidelines in my specialty
·Generally, I follow the guidelines
·I have confidence in guidelines from the health authorities
·I have confidence in guidelines from the Medical Association
·There are too many guidelines in my specialty
·The distinction between information, guidelines and regulations is unclear
·Guidelines are frequently difficult to access
We also asked the respondents to rate the importance of the following obstacles when guidelines are ignored:
·I lack a comprehensive, definitive source of guidelines
·There are several competing guidelines in my specialty
·Inconsistent guidelines are confusing
·The guideline does not fit the individual patient
·Guidelines are only suggestions, clinical judgement should be applied
·Economic concerns overshadow clinical concerns
·I am sceptical about the evidence
·The recommendation is contrary to the patient’s preferences
We distributed the questionnaire to a representative sample of 1600 Norwegian medical doctors, including 400 GPs. The questionnaire was part of a more extensive panel survey administered by The Research Institute of the Norwegian Medical Association every other year. The response rate was 60
% among the GPs in the survey. The respondents were informed about anonymity issues when they originally entered the panel. We also had access to anonymous background data about the respondents and their practices.
The survey results showed that almost all of the respondents (98
%) claimed that they generally followed guidelines, while 88
% said that they had confidence in guidelines issued by the health authorities. Hence, the survey, in contrast to the focus group study, indicated that almost all the GPs were generally positive to clinical guidelines issued by the authorities. However, the barriers to guideline adherence that emerged in the focus group study were confirmed to be of importance and the survey also confirmed the relative importance of the different barriers. Thus, the barriers identified in the focus group study were confirmed as relevant by survey participants, but appeared to have little impact on participants’ confidence in the guidelines and the health authorities.