Checklist for Guideline Panel or Guideline Development Group Chairs©

v.3.4 | 20221114

Name of meeting:

Getting yourself prepared for meetings

• Familiarize yourself with:
  • The organization’s process and rules for guideline development
  • What the expected deliverables are
  • The organization’s policies for declaring and managing conflict of interest (COI)
  • Background material, particularly evidence tables and Evidence to Decision (EtD) tables
  • Panelists, their areas of expertise and their declared COI
  • Observers, technical staff and other meeting participants
  • Controversial issues
• To be taken seriously by the panel members, familiarize yourself with the terminology/language and content related to the guideline topic
• Check how the patient or people representatives been prepared/trained for participation in guideline panels?
• Ensure involvement of panel members in the question (PICO) development process
• Ensure background material (particularly evidence tables and EtD tables) is disseminated to panel members ahead of time for their review and input

Getting the team prepared for the meeting

• Allow for sufficient face-to-face or video remote meeting time with the technical team (coordinating team, systematic reviewers and guideline methodologists) before the meeting
• Ensure meeting worksheets (e.g., evidence to decision frameworks, including straw recommendations – see below) are ready for the meeting in hard copies or online
• Make sure that the technical team has sufficient time to present
• Ensure that members review the checklist for guideline participants
• Identify one or two persons to record the minutes of the meeting. In addition, consider video or audio taping
• Identify one person to edit to the Evidence to Decision frameworks on a live screen. Instruct that person to refer to the chair for final decisions or notes to record
• Agree with the systematic review or knowledge synthesis team on what specific information to present (PICO question, summary of the search and its results, major results of the review, and evidence profile)
• Ask members of the technical team (particularly systematic reviewers) to address during the meeting any clarification questions from the panel, but not to make any judgments or value-laden comments
• Discuss how to conduct the voting
• Assign seating appropriately
• Assign someone to help with time keeping if needed
• Ensure that a copy of the electronic files (minutes, filled EtD, voting results, etc) is saved elsewhere as well
• Be familiar with conflicting or related recommendations done by the same organization (if applicable)
• Circulate among the panel members a cheat sheet of the signalling questions for the EtD framework related to GRADE
• Prepare/familiarize yourself with the relevant analytical framework for the diagnostic and management pathways

At the beginning of the meeting

• Clarify specific objectives/goals of the meeting and scope of the guidelines
• For online meeting: consider disabling the chat box if you think it may bias the project
• Make appropriate acknowledgments
• As people introduce themselves, note names and seating of panelists
• Establish rules for addressing people (first name vs last vs titles etc.)
• Solicit any new COI since they were last declared
• Remind panelist about COI management policy, and that you will not allow strong advocacy: “please state your point clearly once, and don’t keep hammering the same point. Everyone is clever enough to understand it from the first time, and to ask for clarification if they feel the need to”.
• Remind panelists about the confidentiality of the discussions, up to the point they are published or made public by the responsible parties, e.g. the organization
• Clarify to panelists that they are expected not to ask after the meeting for changes in the quality of evidence or strength of recommendation agreed on during the meeting unless errors are apparent.
• Clarify ground rules (rules of process)
• Stress importance of adhering to methodology and that “this is not the time to discuss its value”; “you have accepted to be part of this game, so you need to play by its rules and not attempt to bend the rules”.
• Clarify who is a voting panelist, non-voting panelist, observer, technical advisor and other participants
• Establish the order in which you hear people: voting members first; officers; observers (if allowed to speak)
• Tell panel members: when you disagree, provide a solution as well
• Review agenda and stress importance of adhering to schedule
• Check if panel members are representing organizations; in most cases even if selected from organizations, panel members should use their own judgments

Structuring the discussion

• Structure the discussion around the GRADE Evidence to Decision (EtD) Frameworks (and the criteria that affect the final recommendation)
• Repeat the PICO at the beginning of each recommendation, present the straw recommendation, work through the EtD criteria
• Do not refer to or show recommendations made by other groups or guideline panels
• As panelists raise points that are relevant but not directly related to criteria that not directly affect the recommendation, attempt to classify them as: conditions/key remarks to go underneath the recommendation statement; implementation considerations; monitoring considerations; implications for future research or other content of the EtD
• Show the straw or mock recommendation as a starting point for discussing the recommendation statement
  • g. For patients suspected of having TB who are smear negative, the WHO expert panel suggests/recommends using/not using test … over test … (conditional/strong recommendation, … certainty in the evidence)

Building consensus

• When there seems to be a consensus on the direction of the recommendation (for vs. against), agree on the direction first, then attempt to achieve consensus on its strength (strong vs. conditional)
• When there seems to be a consensus on the strength of the recommendation (strong vs. conditional), agree on that strength by consensus
• In trying to achieve consensus among panelists:
  • Check first whether there is agreement by allowing anyone on the panel to speak up
  • If there seems to not be agreement, label the disagreement; clarify what people are agreeing on and what they are disagreeing on; and check whether those disagreeing would be willing to accept the majority’s opinion (“can you live with it”?).
  • If not, ask whether a modification or addition would make them agreeable.
  • If not, resort to voting. In the case of voting, keep track of and note results

Managing COI

• Enforce the COI management strategy
• Stay alert to strong advocacy (e.g., aggressive argumentation, leading statements, repeating the same point)
• When you detect strong advocacy label it in front of the panel and ask the person of interest to cease strong advocacy; also consider reminding about the specific COI in front of the panel
• Repeat statement about confidentiality (in particular, if the meeting lasting more than one day)
• Avoid statements that reflect your own views on the matters being discussed; aim to show neutrality

Additional points for the meeting

• Enforce time keeping
• Note to minute taker important points to go in the meeting report or guideline document; this is particularly relevant when you need to ensure transparency
• Clarify conceptual issues as needed
• Ensure everyone has the chance to participate, particularly community/patient representative
• Allow for time to debrief with the technical team during the meeting at regular intervals and as needed
• When presenting about the GRADE methodology (either on the certainty of the evidence or the EtD criteria), consider using examples from the same field of the guideline under discussion depending on how well the content will be accepted.
• In order to highlight the value of the different EtD criteria and clarify their application, collect and use in your “GRADE presentation examples provided by the experts (during the usual opening discussions and presentations) that you could match to the EtD criteria (see Annex 1) and may not be known as doing this by panel members.

At the end of the meeting

• Summarize what has been achieved
• Agree on what needs to be achieved after the meeting
• Clarify communication plan
• Provide a timeline for the work to be done
• Make appropriate acknowledgments
• Repeat statement about confidentiality of discussion (describe what should remain confidential and what can be made public in agreement with relevant organization)
• Let the panelists know that they may be receiving a survey for feedback and evaluation (when applicable)

After the meeting

• Send a follow-up message the day after the meeting summarizing the positive aspects of the meeting and accomplishments
• Provide summary information in writing about what is next (including a timeline)