We asked a convenience sample of fourteen acquainted OPs working scattered over four different regions of the Netherlands to collect data for us. Our main criteria to ask a physician to participate were that he or she had to be professionally sufficiently experienced. Next, we took care that there was variation in location to avoid the situation that the same professional expert would be asked about the same case vignette by different OPs. Even though we tried to vary age, gender and professional experience, the majority was over 40, male and had a long standing professional experience and three OPs had achieved a doctor's degree. (Table ) All OPs were considered experienced and professionally motivated, and agreed to participate. The OPs were requested to obtain two professional advices on each of three case vignettes which would lead to a maximum of 84 cases. To be able to show that a relevant 15% of the answers would not be in line with the literature with α = 0.05 and β = 80% we would need about 53 cases. A professional advice was defined as an advice from a person who was considered by the OP to be an expert on the subject and who would also be consulted in the normal course of daily routine.
Personal characteristics of occupational physicians (N = 14) involved in the study
Twelve cases were selected on the basis of a clear occupational health problem, resemblance to daily practice for an OP and assumption that there would be sufficient literature (Table , See Additional file 1
). The cases represent a broad variety of occupational health practice ranging from return to work interventions in workers with musculoskeletal disorders to the causality of stress in case of a myocardial infarction. The case vignettes ended in a clear clinical question that could be answered by a simple yes or no. For example, 'does continuous years of work stress increase the risk of a myocardial infarction?' and 'is it useful to take melatonin to prevent jetlag?'
Summary of the case vignettes and correct evidence-based answer
The OPs were asked to draw their own conclusion on the case vignettes and to provide the professional advice of all the experts that were consulted. The OP could decide for himself whether or not to rely on the advice received. All cases had to be advised on by the experts with yes or no accompanied by a motivation for the answer. The experts were kept unaware by the consulting OP that the cases presented were fictive.
These professional advices were compared to evidence from the literature in the form of a critically appraised topic (CAT). Critically appraised topics are considered as the best way to retrieve an answer to a question arising from practice from the literature. We followed the guidelines for making critically appraised topics as formulated by Sacket et al.[6
] We used Medline, the Cochrane Library and the Dutch clinical guideline database (CBO) to search for relevant evidence to the clinical questions. We used the best available evidence that we could find on a certain topic. In three cases we could use a Cochrane systematic review, in four cases we could use a systematic review and in 5 cases we relied on original studies as the best evidence because no systematic review was available. We felt that for none of the cases the evidence was novel or surprising, but that the available recent literature all pointed in the same direction. All CATs are described in the appendix together with the search strategy and the evidence that was used to answer the clinical question. [See Additional file 1
A professional advice was considered correct if both the 'yes or no answer' and the motivation were in line with the evidence from the literature as summarised in the CAT. The conclusions of the OPs were assessed only by their 'yes or no answer'.
The first two authors (FS and JV) checked and evaluated both the professional advices and the answers from the OPs separately. We measured the proportion of advices and answers that were not correct.