All but one of the trainees invited (n = 18/19) and all of the GP trainers invited (n = 11/11) agreed to participate in the focus group meetings. One trainee did not attend the second meeting, but the others were present at all 3 meetings.
Can ill-structured problems be used to elicit general practice trainees' and trainers' EB?
We used King and Kitchener's framework to analyze the data, and found that it did allow us to code a considerable number of statements made by participants. Figures and illustrate the number of statements expressing EB and demonstrate that both trainees and trainers did express a wide range of EB at different levels. The coding and recoding of these resulted in an interrater reliability of 0.851 (Spearman's rank order correlation coefficient with p < 0.01), an interrater agreement of 81.4% and a Weighted Kappa of 0.740.
Figure 1 Number of EB's expressed per focus group meeting with the trainees. FG 1 1: refers to the first meeting of the focusgroup 1. FG 2 1: refers to the first meeting of the focusgroup 2. K&K1-7: refers to the expressed level of EB as defined in the (more ...)
Figure 2 Number of EB's expressed per focus group meeting with the medical trainers. AOP 1: refers to the meeting with the first group of GP trainers. AOP 2: refers to the meeting with the second group of GP trainers. K&K1-7: refers to the expressed level (more ...)
What were participants' views on knowledge?
Participants expressed EB reflecting different stages of development.
One of the vignettes argues that depression should be considered as a disease in view of its association with an imbalance in the levels of neurotransmitters in the brain.
One of the trainers answered:
"That is absolutely right. When we think about the serotonin involvement, our daily experience has taught us that we get very good results when we give serotonin reuptake inhibitors. We all agree on that."
When asked why there are different opinions on the same issue, one trainee answered that:
'I think that there is still a lot that is unknown in that field....'
According to King & Kitchener's staging of EB, these two statements indicate EB at the first stage of pre-reflective thinking: the belief that knowledge is certain and available through direct observation. In this case, the trainee assumed knowledge to exist absolutely, and that good and thorough observation would lead to the correct information. Neither of them seemed to consider alternative points of view at this stage.
Another trainee saw the different points of view as more context dependent and subjective (as filtered through a person's perceptions). This is what King and Kitchener describe as the quasi-reflective phase (see table ).
"It just depends on how you look at it. When we, for example, look at the domain of psychology from the medical perspective, we look at it differently than people who look at it from another field of interest."
This trainee had not quite reached the reflective phase of EB, because he did not seem able to select a single point of view as more plausible. Rather, he was convinced that different opinions result from different but equally legitimate interpretations of the same observation.
Another trainee translated these beliefs into the idea that knowledge is uncertain and that it is impossible to know for certain. He found this thought deeply disturbing:
'You don't know, and we will probably never really know. It (the construction of Egyptian pyramids) happened too long ago. But if that's what we have to tell the patients...
The shoulder vignette states that in order to select an appropriate management plan for patients with acute shoulder pain, only a minimal number of clinical tests are required. In discussing this vignette, trainers pondered the context-dependency of knowledge (evidence), its evolutive nature and the need to select the most plausible interpretation of knowledge in a given situation. These are arguments suggesting reflective levels of EB. For example:
"That position is taken by the Dutch Guidelines because their consultations last 10 minutes per patient... Those are indeed the most important tests given that timeframe."
"How do I explain that experts differ on this matter? Firstly, the way you decide to write guidelines. Secondly, the strategy the Dutch GP uses to make the guidelines. They only keep data of which they are sure there is evidence... Maybe we shouldn't forget that the guidelines are 6 years old. I don't know whether the guidelines would be the same if they were rewritten today."
How did the participants justify knowing?
In the pre-reflective phase, knowledge requires no justification (stage 1) or is justified through the opinion of experts or authorities (stages 2 and 3). In our study, trainees and trainers very often justified knowledge through the views of authorities:
'I think I would first prescribe an ACE-inhibitor, because that's what I learnt during the endocrinology course and during my training.'
'Even if it no longer recommended, my training experience means that I still think it's important'
'From the courses, particularly in the 4th year. From cases and pharmacology etc.'
The subjects' own experiences were also often dominant in the justification of choices they made.
"I have worked on the urology ward, and that is where I have seen that..."
"In general practice, decisions are more based on one's own experience"
"During my practical training, I saw that the GP prescribed antibiotics straight away, because he knows the situation from his experience. So at first I will stick to the guidelines, but later on my experience will play a more important role."
Justification in the reflective stages is a result of a variety of considerations: weight of evidence, explanatory value of interpretations etc.
A GP trainer referred to the weight of evidence which is different for diagnostic research and for RCTs (Randomized Clinical Trials):
"I am sometimes irritated, especially with these studies (the guideline on acute shoulder pain), because absence of evidence does not necessarily mean that these tests are clinically irrelevant. The major problem with diagnostic tests is that it is very difficult to standardize the clinical examination. It is much easier to standardize RCTs. But if you decide to be guided by existing evidence, one indeed has to agree with this statement".
Similarly, in one of the groups, the conversation was repeatedly focused on the difficulty of applying statistical group results to individual patients:
"If you look at studies which start with 1000 subjects, the ones who react to the placebo and the ones with co-morbidity are excluded. ... And how well do, let's say, 100 'ideal' patients that are included represent the rest of the population? They are used in the guidelines that we have to interpret, but three quarters of the population doesn't fit in this model !