6.1 Define topic
Guidelines can be developed for a wide range of topics, including conditions (sinusitis, ear infections), procedures (tonsillectomy, tympanostomy tubes), and signs or symptoms (cough, hoarseness). Topics selected for guideline development should be high-priority and feasible.
topics have the potential for evidence-based practice to improve health outcomes, minimize undesirable variations in care, and reduce the burden of disease and health disparities. The Institute of Medicine has identified the following priority setting criteria as common to most international guideline development groups:8
- DISEASE BURDEN. Extent of disability, morbidity, or mortality imposed by a condition, including effects on patients, families, communities, and society overall.
- CONTROVERSY. Controversy or uncertainty around the topic and supporting data.
- COST. Economic cost associated with the condition, procedure, treatment, or technology related to the number of people needing care, unit cost of care, or indirect costs.
- NEW EVIDENCE. New evidence with the potential to change conclusions from prior assessments.
- POTENTIAL IMPACT. Potential to improve health outcomes and quality of life; improve decision making for patient or provider
- PUBLIC OR PROVIDER INTEREST. Consumers, patients, clinicians, payers, and others want an assessment to inform decision making.
- VARIATIONS IN CARE. Potential to reduce unexplained variations in prevention, diagnosis, or treatment; the current use is outside the parameters of clinical evidence.
Feasible topics have a sufficient base of high quality published evidence (ideally randomized, controlled trials) to drive guideline development, have one or more existing systematic reviews or meta-analyses already published on relevant issues, and have relatively clear definitions of the condition or procedure under consideration.
A steering committee that includes organizational leadership and broad stakeholder representation can help identify, prioritize, and refine guideline topics. Diversity of expertise and perspective helps minimize bias caused by conflicts of interests.
The AAO-HNS convened the Guideline Development Task Force as a steering committee for developing evidence-based guidelines and related knowledge products.19
The task force includes representatives of all sub-specialty groups within otolaryngology and of all relevant internal Academy groups, including research, patient safety, quality improvement, board of governors, and evidence-based medicine. Topics are solicited with a standardized form, based on principles outlined above, then presented to the task force for ranking and prioritization.
6.2 Convene the guideline working group
Perhaps the most important decision in creating a successful guideline relates to composition of the working group. A group size of 15–20 members encourages diversity and efficiency yet is small enough to avoid delays and redundancy.
The group should consist of the (1) chair and two assistant chairs, (2) staff lead and assistant, (3) technical consultant, (4) content experts, (5) stakeholders from all relevant disciplines, including nursing, primary care, and allied health, and (6) a consumer representative. The roles and responsibilities of group members are outlined in the sections that follow.
6.3 Identify organizational leadership
A staff lead is assigned as the primary liaison for the group, with one or more assistants who have the dual responsibility of supporting the lead and learning the process so they may serve as a future lead. Qualifications for staff lead include service as an assistant staff lead on a prior guideline panel, experience conducting literature searches and using a citation database, and a basic understanding of study design, medical terminology, and levels of evidence.
Specific responsibilities of the staff lead and assistants include:
- Conducting a preliminary search to assess topic feasibility
- Identifying guideline group members by working with internal leadership and relevant external organizations
- Scheduling and handling logistics for all conference calls and group meetings
- Working with the chair to create agendas and pre-distribute supporting materials
- Coordinating literature searches, organizing search results, and obtaining full-text articles
- Appraising the guideline for implementability using predetermined methods
- Identifying external peer reviewers and collating comments for distribution to the chair
- Assisting the chair in developing and obtaining permissions for tables and figures
- Proofreading the guideline final draft, including checks for grammar and spelling
- Submitting a summary of key action statements and supporting evidence profiles for review and approval by the organizational board of directors
- Assisting the chair in formatting the final document for publication submission
- Obtaining copyright transfer and financial disclosure forms from working group members
A technical consultant is assigned to ensure that the working group adheres to methodologic standards and protocols endorsed by the organization, and to serve as a facilitator who supports the chair during conference calls and meetings. The technical consultant should be fluent with guideline methodology, understand the process of systematic review, and have direct experience prior guidelines developed by the organization.
Developing valid guidelines is not intuitive, but is an acquired skill that is independent from clinical expertise and accomplishment. Whereas an explicit and comprehensive manual aids the process, it cannot substitute for hands-on experience.
6.4 Identify clinical leadership
A chair should be identified to lead the group in developing the guideline and to work with the technical consultant and staff lead to ensure adherence to methodologic standards. The chair also facilitates the interpersonal aspects of the group processes, so the members work in a spirit of collaboration with balanced contribution from all members.
Specific responsibilities of the chair include:20–21
- Assisting the staff in planning conference calls and meetings
- Steering discussions according to the agenda
- Encouraging all members to contribute to discussions and activities of the group
- Remaining aware and constantly attentive to small group processes, including how the group interacts, communicates, and makes decisions
- Establishing a climate of trust and mutual respect among members while remaining sensitive to preexisting inter-professional tensions and hierarchies
- Maintaining a unified group discussion free of sub-conversations and dominance
- Encouraging constructive debate without forcing agreement
- Winding up repetitive debate and disagreements through careful negotiation
- Summarizing main points and key decisions of a debate
- Delegating writing assignments and integrating completed assignments and group feedback into the draft guideline
The chair is appointed by a selection panel that includes organizational leadership, steering committee representation, the guideline staff lead, and the technical consultant. An ideal chair should be efficient and motivated, have demonstrated leadership ability, have prior experience with evidence-based guideline development, have demonstrated skills in scientific writing, and be fluent with using the internet, e-mail, and e-mail attachments. Candidates for chair will be asked to submit a curriculum vitae and declaration of competing interests, and to confirm that they understand and accept the substantial time commitment involved.
The chair should ideally not
be a content expert for the guideline topic, but should be familiar with the scientific literature and management of the clinical condition. Content experts are usually abundant in an organization and can be readily added to the working group to fill in knowledge gaps. Conversely, the chair should be an impartial leader who stimulates discussion, not an advocate who injects their own opinions.17
One or two assistant chairs should be identified who will be asked to chair the next guideline development effort. To maintain a pipeline of guideline projects, a continuing source of leadership for upcoming projects is needed. The best way to groom new chairs is to have them serve on one or two prior guideline groups to learn methodology and expectations early on. An ideal assistant chair should have experience with evidence-based medicine, but does not necessarily need prior guideline development experience.
The chair is ultimately responsible for moving along the guideline process and keeping the group focused and task oriented. Having more than one chair is inadvisable, because responsibilities can be easily shifted and diffused. Instead, the structure should include one chair and one or more assistant-chairs, as noted above.
6.5 Identify partner organizations
Guideline development panels should include individuals from a range of relevant stakeholder groups to minimize bias. Multidisciplinary participation helps identify and evaluate all relevant evidence, builds support among the intended guideline users, and increases the chances of addressing practical problems related to implementation.10
Many guidelines warrant input from nursing, consumers, and primary care clinicians. Based on the target population and setting, the working group may include internists, pediatricians, geriatricians, family practitioners, and emergency medicine physicians. Additional specialty clinicians are recruited as dictated by the specific topic or condition under study. Allied health professions are similarly recruited, and may include audiologists, physical therapists, speech-language pathologists, and others.
An excellent source of consumer participants for guideline development is Consumers United for Evidence-based Healthcare (CUE), a national coalition of health and consumer advocacy organizations, which empowers consumers through critical appraisal of articles, guidelines, and systematic reviews.22
CUE is a project of the U.S. Cochrane Center and works closely with the Cochrane Consumer Network.
If another discipline is to be a full partner in developing the guideline, they are approached early to secure interest and cooperation. Alternatively, working group members can be selected to represent their “discipline,” not their “organization.” In this model a pediatrician member of the working group would provide essential input for pediatrics as a discipline, but would not necessarily represent the American Academy of Pediatrics or imply their specific endorsement of the resulting guideline.
6.6 Identify guideline working group members
In deciding what disciplines other than otolaryngology to include in guideline development, a useful approach is to ensure that every discipline or organization that would be involved with implementation, including consumers, has a voice at the table. This will nearly always include one or more primary care clinicians, since invariably they will be involved in counseling the patient and coordinating care with the specialist. Representatives of all relevant medical specialties other than otolaryngology must also be considered.
A single specialty group will reach different conclusions than a multidisciplinary group when presented with the same evidence.17
Individuals from a single discipline are often biased towards procedures in which they have a vested interest. Involving multiple disciplines tends to balance bias and produce more valid guidelines.8
Potential members of the working group can be identified by organizational leadership, partner organizations, the working group chair, and the staff liaisons. An understanding of evidence-based medicine is desirable. Individuals are invited as representatives of their field or discipline, but need not be content experts for the guideline topic. Content experts should be a minority voice on the working group to limit bias.
Specific responsibilities of the working group members include:
- Participating in all conference calls
- Attending all meetings with a commitment to teamwork and clear communication
- Reading all relevant materials and providing constructive comments and feedback during and between meetings
- Checking and responding to e-mails on a regular basis
- Completing personal assignments to meet deadlines
- Maintaining confidentiality
- Disclosing fully any potential conflicts or interest
The importance of choosing an appropriate working group cannot be overemphasized. This is called a “working” group for a reason: producing a guideline requires substantial time and effort. All members have a responsibility to other participants to behave with integrity, commitment, and a fully professional demeanor.
Despite the upfront commitment of all working group members to participate fully in guideline development, conflicts or unexpected circumstances may arise that threaten validity if an important discipline is not represented. Therefore, certain disciplines, which include primary care and selected others depending on the topic, should be represented by two group members to ensure representation.
6.7 Compile contact information grid
The staff lead should compile a grid of contact information for all working group members and organizational representatives. Included in the grid should be (1) name and degrees, (2) working group role, (3) organizational affiliation, (4) clinical and academic titles, (5) mailing address, (6) disclosed conflicts of interests, and (7) contact information.
6.8 Conflict of interest disclosure
A conflict of interest exists when a participant or the participant’s institution has financial or personal relationships with other people or organizations that may inappropriately influence (bias) his or her actions.
Despite good intentions, it is not appropriate for individuals to decide if a particular relationship causes conflict; their role is to declare, not interpret. The group as a whole must ultimately determine if a conflict may result in bias, and whether or not the degree of conflict excludes the individual from participating in the entire guideline or selected sections.
Financial relationships are easily identifiable, but conflicts can also occur because of personal relationships, academic competition, or intellectual passion. Examples of financial conflicts include employment, consultancies, stock ownership, honoraria, paid expert testimony, patents or patent applications, and travel grants. Full disclosure is advised regardless of whether the participant considers the relationship relevant to the guideline content.
The contact and disclosure list should be distributed to all members for verification and should be updated, as needed, during guideline development and prior to publication.
6.9 Determine dates for conference calls and meetings
Adhering to a predetermined, specific timeline allows publication of the guideline within 18 months. Arranging dates for conference calls and meetings is particularly difficult when dealing with individuals representing multiple organizations and disciplines. Therefore, it is critical to plan early in the process. Events are planned using the timetable in
- Conference call #1 takes place in month 2
- Conference call #2 takes place about 4 weeks later, in month 3
- In-person meeting #1 takes place about 4 weeks later, in month 4
- In-person meeting #2 takes place about 6–8 weeks later, in month 6
Conference calls are often most feasible if planned to start at 8:00 p.m. Eastern Time. Calls should be generally scheduled for 2 hours. In-person meetings can begin at noon with a light working lunch to allow attendees to fly in the same morning. Similarly, they can end by noon the next day to allow a return flight the same day. A group dinner should be planned the first day. A convenient schedule is to begin on either Friday or Sunday, and end the next day.
The staff lead prepares a grid of potential dates for the calls and meetings. The grid is circulated by electronic mail to the chair, assistant chair, and technical consultant to determine to determine available dates for the first two conference calls. For the in-person meetings and future conference calls, the grid may be circulated to the entire working group to assess availability. There will clearly be a need for compromise by some group members, since the odds of finding dates agreeable to all are extremely low. Group members must commit to attending these meetings at the start.
The importance of having all working group members participate in all conference calls and attend all meetings cannot be overemphasized. Advance planning is the best guarantee of success, since maximal time is available for group members to adjust their schedules as needed and block out event dates in their calendars. If a group member cannot make this commitment, an alternate should be found as soon as possible.