Appraisal of the guideline
Eight of the 10 participants appraised all 36 recommendations with 24 questions. One appraiser assessed 25 recommendations, and one participant appraised only 7 recommendations because of lack of time.
In the final report (Additional File 4
, Appendix 4), 69 questions are marked as barriers (in red) and 501 as facilitators (in green). Twelve borderline barriers (doubtful items with only one-point difference) were marked separately (in orange) with inside in the table written the tendency toward which answer.
The guideline included five 'recommendations' (R14, R26, R31, R34, and R36) that did not have a described condition or action, the so-called nonrecommendations. These nonrecommendations were statements or observations that could not be appraised with the eGLIA instrument. Therefore, we excluded them from further analyses.
When we analysed the global dimension, we found three barriers to implementation (Q3-Q5). First, the guideline did not address strategies for implementation (Q3), although it seemed that dissemination of the guideline had been undertaken with an online version. Second, there was no tool for application (Q4) available, such as a summary document. The electronic version on the ESHRE website http://www.guidelines.endometriosis.org/
provided access to a concise summary and supporting documentation, but the paper version of the guideline did not refer to this. Third, the differences in the importance of the recommendations (Q5) were only described at the level of evidence. A clear presentation or formatting reflecting the differences was lacking.
Regarding the individual recommendations, two were straightforward to implement (R1 and R12). The remaining 29 recommendations contained one or more barriers.
The guideline scored very well on four dimensions, which can be considered implementation facilitators. First, the dimension of decidability (Q8-Q10) had positive scores for almost all recommendations. The description of the conditions and their mutual relations were very clear. All recommendations were easily identifiable because they were summarised in frames. Only two recommendations (R19 and R32) had a vague definition of the stated condition. For instance, the phrase 'depending on the severity of the disease' would need further specification (R19).
Second, the recommendations were as concise as possible and their presentation and formatting (Q15 and Q16) provided good visibility.
Third, the apparent validity (Q19 and Q20) was scored as a facilitator due to the structured reporting of the evidence and its quality linked to the individual recommendations.
Fourth, in the dimension of novelty/innovation (Q21-Q23), almost all recommendations were feasible without the need of new skills or knowledge (Q21). Moreover, the guideline considered the existing attitudes and beliefs of the intended users of the guideline (Q22 and Q23). However, R35 appeared incompatible with existing attitudes and beliefs of the guideline's intended users because it favoured complementary medicine.
Four barriers to implementation were identified. First, the appraisers found that executability (Q11 and Q12) was a barrier in various recommendations (R7, R9, R10, R13, R15, and R33) because they were vague in their descriptions of the recommended actions. Formulations such as 'consideration should be given' did not make clear whether the action should be carried out or not. In addition, information about how a certain action should be performed was missing. Measuring adherence to such recommendations is difficult.
Second, the effect on the process of care (Q13 and Q14) was identified as a barrier. Four recommendations (R9, R11, R21, and R30) included actions that needed extra equipment, staff, or provider time to make them implementable. For example, not all hospitals have magnetic resonance imaging or facilities for in vitro fertilisation available.
Third, the lack of clear measures (Q17 and Q18) was a barrier in seven recommendations (R5, R8, R10, R19, R23, R24, and R35). There were no criteria for measuring adherence to these recommendations, which could complicate the monitoring of endometriosis care.
Fourth, the flexibility (Q24-Q27) was found to be a barrier. Some recommendations (R7, R8, R10, R11, R15-R17, R19, R33, and R35) lacked specific patient or practice characteristics to enable individualisation of care (Q24). Most recommendations (R2-R10, R13, R15-R25, R27-R30, R32, R33, and R35) did not consider coincident drug therapy and common comorbid conditions (Q25). Furthermore, the incorporation of patient preference (Q27) formed a barrier. R33 and R35 considered this preference but did not propose any mechanisms to implement the preference in practice. An exception to flexibility as a barrier was the strength of the recommendations (Q26), which the guideline developers stated explicitly with the classification of the recommendations.
We excluded the four optional items from the dimension computability (Q28-Q31) from further analysis because no electronic implementation was planned. At the time of our study, information technology support systems were not available to implement the guideline.
Of the 36 recommendations, 15 were graded with evidence strength A. These recommendations (R6-R8, R16-R18, R20, R22, R24, R25, R27-R29, R31, and R33) had significantly fewer individual barriers for implementation than the remaining recommendations did (Table ). Recommendations graded A had 26 barriers in 260 items (10%) versus 52 barriers in 315 items (16.5%) at levels B, C, D and the good practice points (p = .02; odds ratio = 0.5 [95% confidence interval, 0.3-0.9]).
Barriers related to the grade strength of the evidence
General implementability of the guideline
The median score for the additional question assessing the implementability of the guideline was 4, ranging from 2 (probably not implementable) to 5 (definitely implementable). Six appraisers (60%) thought that the guideline was probably implementable (with some adjustments) or definitely implementable. One participant considered the guideline as probably not implementable.
Process evaluation of eGLIA
On average, the time the participants spent completing the appraisal (response 8 of 10) was four hours (range: three to eight hours). The average time needed to complete one recommendation was 10 minutes (range: 5 to 24 minutes). The answering became easier and quicker as more recommendations were appraised.
Most participants (60%) found the explanation of the GLIA dimensions and the use of the eGLIA tool clear (Table ). However, they commented that more scoring examples would have been helpful. The general opinion was that the eGLIA tool was easy to use (70%) and functional for its purpose. Most questions were appraised with an agreement of more than 60% (for the answers 'agree' or 'strongly agree'). Identifying obstacles to implementation and judging the recommendations systematically were consistently appraised with close agreement (80% and 90%, respectively). There was wide variation in the understanding and application of the tool questions. Appraisers reported that several questions in the eGLIA instrument were not very clear or that they had to read them several times to understand the meaning. In addition, the participants stated that appraising a large number of questions was boring and too time consuming.