Topic

Additional Items for QAI

1. Organization, budget, planning, and training

1. Define the care pathway related to the health topic, and identify the data and input parameters from different parts of it that implementers need to report on people-/patient-important outcomes and quality indicators.

2. Explore whether outsourcing of specific tasks (e.g., systematic review conduct or QA work) is required to conduct the work.

3. Determine the scope of the QAI scheme; in particular, clarify whether the group will be expected to specify performance indicators (thresholds) linked to quality indicators and performance measures. This may require specific expertise.

2. Priority setting

4. Modification to original item 4: Identify the perspective that is taken for the QAI scheme (clinical individual, clinical health system, coverage decisions, public health, or population/societal). This may be the same as or different from the perspective taken for the guideline.

5. Search for quality indicators and performance measures already existing for the topic.

6. Consider where quality indicators should be assessed in relation to the evidence pertinent to the decision-making process: parallel groups assessing the guideline recommendations and QAI scheme, integrated with the recommendations, or sequentially (if sequentially, the QAI scheme would usually follow completion of the guideline).

7. Identify current use and gaps in accreditation/certification schemes on the QAI topic and any existing evidence on the use of these schemes to improve outcomes.Consider which quality indicators may be feasible for use in certification and/or accreditation schemes (e.g., those that are easy to measure and collect, already available, or ready to benchmark). This may affect acceptability and feasibility of any quality indicators emerging from the guideline/QA scheme.

3. Guideline group membership

8. Determine whether project subgroups are required for focused work on specific topics, including the QA scheme development. Determine which individuals will be needed in subgroups and how they will interact with the larger panel.

4. Establishing guideline group processes

9. Predefine the process that will be used by the group to select final quality indicators; performance measures; and, if applicable, performance indicators from a candidate list of guideline recommendations and, if appropriate, from existing quality indicators and performance measures (e.g., consensus, nominal group process).

5. Identifying target audience and topic selection

10. Decide whether the evaluation of proposed existing quality indicators currently in use might be an intervention question with evidence reviewed by the guideline or conducted following publication of the guideline.

6. Consumer and stakeholder involvement

11. Modification to original item 1: Identify the appropriate stakeholders to involve and/or consult in the development of the guideline and QA scheme to incorporate views of all who might be affected by them (e.g., professional groups, health managers, policymakers, industry representatives).

12. Modification to original item 2: Identify the appropriate consumers to involve in the group and/or consult in the development of the guideline and QA scheme to incorporate views of all who might be affected by them (e.g., individual people/patients, people who provide nonreimbursed care and support [such as family/caregivers] to patients, members of the public as potential patients and as funders of health care through taxation, QA experts and implementers, community organizations that represent the interests of patients, and advocates representing the interests of patients and people who care for them).

7. Conflict of interest considerations

13. Apply the same declaration of interest and conflict of interest management rules to all participants and their institutions involved in the guideline and QA.

14. Consider potential conflicts of interest of the institution of participants in the process (e.g., if an institution is doing very well on an indicator and advocates for it to be included).

8. PICO question generation

15. Visually describe where PICO questions being addressed and possible quality indicators sit using a logic model/analytical pathway/disease model/analytical PICO framework, including steps from prevention to diagnosis to treatment to outcomes.

16. Determine whether outcomes identified in the PICO question are appropriate as quality indicators for QA purposes. If they are not appropriate, consider whether surrogate outcomes are necessary for PICO question outcomes to be included as quality indicators (see item 27 below).

17. If a guideline question and the related recommendation are focused on whether to recommend the use of a quality indicator, the group should consider using an intervention EtD framework to assess the quality indicator.

18. Identify all relevant input parameters from the different parts of the pathway to the overall people-/patient-important outcomes and quality indicators.

9. Considering importance of outcomes and interventions, values, preferences, and utilities

19. Rate or select a small and well-defined but sufficient number of candidate quality indicators. The number should be linked to the number and breadth of recommendations in a guideline. The feasibility of implementation and monitoring should be considered. We propose no more than 10 quality indicators as reasonable for a moderate-sized guideline/QA scheme.

20. Consider the relation of candidate quality indicators to the people-/patient-important outcomes.

21. When considering candidate outcomes as quality indicators, consider the perspective of key stakeholders (people/patients, clinicians, decision makers) on the appropriateness of quality indicators (see item 27 below).

10. Deciding what evidence to include and searching for evidence

22. Evaluate whether evidence supports that the use of a quality indicator improves people-/patient-important outcomes, particularly if the performance measure selected relates to a process outcome.

23. Consider which outcomes are measurable, feasible, scientifically sound, and relevant as quality indicators or performance measures.

11. Summarizing evidence and considering additional information

None

12. Judging quality, strength, or certainty of a body of evidence

None

13. Developing recommendations and determining their strength

None

14. Wording of recommendations and of considerations of implementation, research, monitoring, and evaluation (including considerations about QA)

24. Select quality indicators and performance measures based on the prioritization of people-/patient-important outcomes.

25. Consider which outcomes, linked to recommendations, are appropriate as quality indicators (see item 27 below).

26. Identify, assess, and mitigate unintended consequences that a quality indicator may have on the target population (people/patients, clinicians, or decision makers).

15. Reporting and peer review

27. Develop or adopt a standardized format for reporting the guideline and QA framework, with specific structure, headings, and content.

28. Report in the monitoring and evaluation section of the EtD framework how the quality indicator fits in relation to item 27 considerations of an appropriate quality indicator.

16. Dissemination and implementation (including considerations about QA)

29. Refer quality indicators that were flagged for potential use for certification and/or accreditation schemes to the appropriate individuals/organizations for implementation.

30. Report in the monitoring and evaluation section of the EtD framework the appropriateness of the quality indicators and performance measures (see item 27 above).

31. Propose mechanisms to document quality indicators or performance measures in a standardized (even anonymous) fashion to allow synthesis of data, collaboration, and shared learning across different health care systems and jurisdictions. This might provide a feedback mechanism for larger-scale improvement and updating of guidelines.

17. Evaluation and use

32. Consider providing guidance on when to keep using, modify, or retire/cease using a performance measure.

33. Consider pilot testing the quality indicators and performance measures with the target end-users(e.g., with members of the target audience and stakeholders who participated in the development group). The type of pilot testing may be different for different groups depending on the timeline and feasibility of an integrated guideline and QA scheme; however, this is a critical step for ensuring feasibility and implementation.

34. Provide clarity on accountability to making the changes in quality indicators, if applicable to the guideline/QA process.

18. Updating

35. Periodically reevaluate the quality indicators, performance measures, and performance indicators in a prespecified time frame. The time frame may vary, but we suggest no longer than 2 to 3 years following release.

19. Preparation for QA and selection of quality indicators (QAI-specific topic)

36. Consider the appropriateness of outcomes as quality indicators.*

37. Use a predefined process to select from the list of candidate quality indicators.

38. Consider potential performance measures linked to selected quality indicators, including their specification and intended use.

39. Use a predefined process to select performance measures linked to each quality indicator.

40. If applicable to the scope of the QA scheme, select and specify performance indicators (thresholds) for each performance measure.