Transformative Education
for Health Professionals

5 - The WHO guideline development process

A WHO guideline is any document that contains WHO recommendations about health interventions, whether clinical, public health or policy related. A recommendation provides information about what policy-makers, health-care providers or patients should (or should not) do or consider doing. It implies a choice between different interventions that have an impact on health and that have ramifications for resource use.

Two Guideline Development Group meetings were held. The first meeting was held in Geneva, Switzerland 9-11 May 2011.  Participants in the first meeting addressed and agreed on the scope, objectives, target audience, outcomes framework, categories and the potential PICO questions under each domain of the Guidelines.  The group then proceeded to discuss and finalize PICO questions.  A GRADE example using interprofessional education, one of the PICO questions, was presented to the group and provided the opportunity to discuss the methodology and ask questions.

The systematic reviews, evidence tables and GRADE profiles were prepared in accordance with GRADE as illustrated in Figure 5 below.  11 systematic reviews were commissioned and completed by August 2012. In addition a feasibility and acceptability survey gathered the views of 136 stakeholders and potential beneficiaries of the recommendations from all WHO regions about the acceptability and feasibility of each of the interventions being considered. A similar survey with civil society survey (169 respondents) provided views and expectations on the main areas of interest.

The second Guideline Development Group meeting was held at the Pan American Health Organization in Washington DC 20-22 March 2012. The purposes of the second meeting were as follows: review the findings from the systematic reviews and other evidence gathered; finalize decision tables and agree on the wording of draft recommendations; agree on each recommendation’s direction and strength (conditional or strong); decide on next steps for finalizing the recommendations; and discuss strategies for launching and translating the recommendations into action.  In the period between the two meetings, a Knowledge Gateway site was created where members of the Guideline Development Group could visit to download documents posted for comment/review by the WHO Geneva Secretariat.

WHO has followed the GRADE system for developing recommendations since 2008. The system separates the rating of the quality of the evidence from the rating of the strength of the recommendation. The quality of evidence reflects the extent of our confidence that the estimates of an effect are adequate to support a particular decision or recommendation.  The GRADE system classifies the quality of evidence as high, moderate, low and very low.  The GRADE framework considers the following factors when deciding on the quality of evidence: type of study design, risk of bias, imprecision, indirectness, inconsistency, publication bias, dose response, large effect size and plausible confounding.

The direction and strength of the recommendation reflects the extent to which the Guideline Development Group was confident that the desirable effects of following a recommendation are greater than the potential undesirable effects. In terms of implications, a strong recommendation can be adopted as a policy in most situations. A conditional recommendation implies the need for substantial debate and involvement of stakeholders in deciding whether or not to adopt the recommendation. In some cases, the panel may decide to qualify the conditional recommendation by providing the “conditions” under which it should be considered. Examples of these conditions include: ensuring availability of experienced staff, space or equipment, conducting needs assessment, and integrating the new intervention within existing programs. One specific type of conditions is implementing the intervention “in the context of close monitoring and evaluation”. This is appropriate when monitoring of the fidelity of implementation of the intervention and evaluation of some short-term outcome can ensure optimal implementation and adaptation if necessary.  Another specific type of conditions is implementing the intervention “only in the context of rigorous research”. This is appropriate when there is a relatively high degree of uncertainty whether the desirable effects of following the recommendation are greater than the potential undesirable effects and the panel feels that the intervention should be adopted only when there is an opportunity to generate the needed evidence.

With respect to health service interventions the GRADE framework considers the following factors when deciding on the direction and strength of the recommendation: the magnitude of the problem, the balance of benefits and harms, resource use, equity, acceptability and feasibility (recorded in the decision tables as at Annex 7).  The decision table is a tool that: provides a systematic and explicit approach to making recommendations; makes transparent the judgements about the factors affecting the recommendations; provides supporting evidence to judgements; and provides guidance to policy makers on what to take into account when considering a recommendation.

Figure 5: Process of formulating recommendationsThe Guidelines Development Group used a standardized decision table for transparently recording the panellists’ judgments (Annex 7).  All decisions were reached by agreement through discussion and consensus, including the direction and strength of recommendations and key considerations attached to the recommendations. Information from the evidence tables was used to develop the GRADE profiles for the final list of recommendations.  The development of the Guidelines document was iterative with drafts of the Guidelines document circulated via email to the Guidelines Development Group, and then to peer reviewers for comment. Differences in points of view were resolved through email discussions. Questions and requests for clarification were also addressed prior to incorporation into the final draft.


Acknowledgment: Dr. Holger Schunemann

Annex 7 provides the decision tables used by the Guidelines Development Group. The domains used to determine the strength of recommendations are described below in table 5.1.

Table 5.1 Domains of decision tables

Problem The magnitude of the problem in terms of the numbers of the target group affected.

Benefits and risks of options Desirable effects (benefits) need to be weighed against undesirable effects (harms). The more that the benefits outweigh the risks, the more likely that a strong recommendation will be made.

Resource use Lower costs (monetary, infrastructure, incremental costs, equipment or human resources)

Feasibility, acceptability, equity These judgements give an indication of the likelihood of the implementation of the recommendation.