Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Pediatr Blood Cancer. Author manuscript; available in PMC 2014 April 1.
Published in final edited form as:
PMCID: PMC3819170

A Worldwide Collaboration To Harmonize Guidelines For The Long-Term Follow-Up Of Childhood And Young Adult Cancer Survivors: A Report From The International Late Effects Of Childhood Cancer Guideline Harmonization Group


Childhood and young adult cancer survivors should receive optimum care to reduce the consequences of late effects and improve quality of life. We can facilitate achieving this goal by international collaboration in guideline development. In 2010 the International Late Effects of Childhood Cancer Guideline Harmonization Group was initiated. The aim of the harmonization endeavor is to establish a common vision and integrated strategy for the surveillance of late effects in childhood and young adult cancer survivors. With the implementation of our evidence-based methods we provide a framework for the harmonization of guidelines for the long-term follow-up of childhood and young adult cancer survivors.

Keywords: Guidelines, Late effects, Childhood and young adult cancer survivors, Collaboration


Over the last few decades advances in the treatment of childhood and young adult cancer have greatly improved survival [1-3]. As a result most children and young adults diagnosed with cancer are expected to become long-term survivors [4]. Unfortunately, the improved prognosis has been accompanied by the occurrence of late, treatment-related complications [5,6]. These late effects may emerge many years, even decades, after completion of treatment. The most frequent occurring late effects affecting quality of life include second malignancies, organ dysfunction, endocrine and metabolic disorders, and cognitive and psychosocial problems [5-7]. Previous studies showed that about three in four childhood cancer survivors are likely to develop late effects, which can be severe or life-threatening in 25% [5,6]. Late treatment-related sequelae can be expected to increase the incidence of chronic diseases in adult survivors of childhood and young adult cancer and may ultimately reduce life expectancy [8]. Long-term follow-up care for childhood and young adult cancer survivors is important to facilitate early detection of late effects and timely initiation of interventions to preserve health [4,9,10].

This overview describes the process of a worldwide collaborative effort to harmonize the recommendations for surveillance of late effects after treatment for childhood and young adult cancer and to prioritize topics for this harmonization endeavor.

Rationale and scope of the international guideline harmonization

To ensure that childhood and young adult cancer survivors receive optimum health care, clinical practice guidelines (CPGs) are essential. According to the Institute of Medicine, CPGs are defined as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” [11]. CPGs are an important part of the “quality circle” for health care improvement and can help to bridge the gap between research and clinical practice (see Figure 1) [12,13]. CPGs are considered as powerful tools to improve the quality of care, to reduce variability in daily practice, and to reduce costs [14-16]. However, studies have shown that many existing CPGs are of poor quality and that recommendations may differ among CPGs on the same topic [17-20].

Figure 1
The quality circle for healthcare improvement

Currently, CPGs addressing the surveillance for late effects in long-term survivors of childhood and young adult cancer have been published by the North American Children’s Oncology Group (COG), the Dutch Childhood Oncology Group (DCOG), the United Kingdom Children’s Cancer and Leukaemia Group (CCLG), and the Scottish Intercollegiate Guidelines Network (SIGN) [21-25]. As each group worked independently to develop their recommendations, patient risk groups, diagnostic tests and screening intervals vary among the CPGs. This non-integrated approach of CPG development leads to duplication of work, inefficient use of resources, and uncertainty about which guideline to follow. Until recently international collaboration in CPG development for the long-term follow-up of childhood and young adult cancer survivors was lacking. In 2010, recognition regarding the potential benefits of collaboration (i.e., reduce duplication of effort, optimize use of expertise, and enhance research opportunities) resulted in an international endeavor to harmonize CPG recommendations for survivors of childhood and young adult cancer: the International Late Effects of Childhood Cancer Guideline Harmonization Group.

The main goal of this ongoing initiative is to establish a common vision and integrated strategy for the surveillance of late effects in childhood and young adult cancer survivors throughout the world. With the implementation of the guideline, we intend to increase the quality of life and decrease complication-related healthcare costs for childhood and young adult cancer survivors throughout the lifespan. We aim to promote healthy lifestyles, to provide ongoing monitoring of health status, to facilitate early detection of late effects, and to advise about timely intervention strategies for late effects to preserve health.

For each recommendation we will address five key issues: 1) Who needs surveillance?; 2) At what age or time from exposure should surveillance be initiated?; 3) At what frequency should surveillance be performed?; 4) What surveillance modality should be used?; 5) What effective treatments are available if health problems are identified?. This latter question is relevant since if no management options are available, screening recommendations are not needed. The guideline is not intended to recommend on specific treatments, but only advise about possible management if health problems are identified.

The guideline is aimed at clinicians who provide ongoing healthcare to survivors of childhood and young adult cancer. The recommendations are important for physicians, nurse practitioners, physician assistants and nurses in the fields of pediatrics, oncology, internal medicine, family practice, and gynecology, as well as subspecialists in many other fields (e.g., endocrinology, cardiology, pulmonology).

Working groups

The International Late Effects of Childhood Cancer Guideline Harmonization Group is directed by a multidisciplinary core group of eleven representatives from the COG, DCOG, CCLG and SIGN guideline groups (Supplemental Appendix I). This core group includes healthcare professionals, content experts, and methodologists with skills in evidence appraisal and synthesis, who held leadership positions in the development of their national/cooperative group guidelines. For each guideline topic, 20-30 individuals with expertise pertinent to the specific topic will be invited by the core group to participate in the harmonization working group. The expert members will represent all relevant disciplines, ranging from pediatric and radiation oncologists and survivorship care providers to guideline methodologists and epidemiologists. In addition, we will involve patient representatives.

To avoid duplication of work, new national or international groups with guideline initiatives for the follow-up care of childhood and young adult cancer survivors will be asked to actively participate in this process. All involved members are asked to disclose any conflicts of interest.

Principles of the harmonization process

The core group established the overall policy for the harmonization process and topic selection. This proposal was discussed with an extended group with scientific and clinical research expertise relevant to guideline development for childhood and young adult cancer survivors (Supplemental Appendix I). The key steps used in the harmonization process are outlined in Figure 2. According to the Appraisal of Guidelines for Research and Evaluation (AGREE) Collaboration [26] and the standards for Developing Trustworthy CPGs of the Institute of Medicine [11], we aim to develop harmonized surveillance recommendations based on scientific evidence, that are easily implemented and effective in practice. For each separate topic, the concordances and discordances and underlying evidence supporting recommendations in existing guidelines will be evaluated. Extensive evidence summaries will be organized to formulate recommendations related to the five key issues as mentioned above.

Figure 2
The international guideline harmonization process

In the case of concordance, we will determine if there is sufficient or insufficient supporting evidence as cited by the guidelines. Focused clinical questions will be formulated for the discordant surveillance recommendations and systematic literature searches will be performed to identify all available evidence. In addition, references supporting the existing recommendations will be researched and experts in the field will be asked for additional evidence. The evidence will be summarized in evidence tables. When evidence is lacking for childhood and young adult cancer survivors, we will carefully extrapolate evidence from other populations. Finally, the experts will discuss the evidence and formulate recommendations considering the quality of the evidence, the benefits versus harms of the screening intervention, and the need to maintain flexibility across health care systems. This latter criterion is essential when developing worldwide recommendations. We aim to formulate recommendations in a way that it can serve as a basis for local adaptation. Recommendations will be finalized after total group consensus has been reached following informal group discussions.

The final recommendations will be discussed at one of the international conferences for late effects in survivors of childhood and young adult cancer [27,28]. In addition, if there is insufficient supporting evidence, the unresolved clinical issues will be included in the research agenda.

For every specific guideline topic we will ask external independent experts in the field and patient representatives to review the recommendations. This review process will include not only content-related review, but also methodological review of both the evidence report and the guideline. We will continue to monitor the literature on a bi-annual basis and will update the recommendations as new evidence becomes available.

Levels of evidence and strength of recommendations

Many different evidence and recommendation grading systems have been previously published by different countries and professional groups. Within the four international pediatric cancer guideline groups, different systems for grading of evidence and recommendations have been used (Supplemental Appendix II). For the purpose of this international guideline harmonization endeavor, we will use a common system to evaluate the levels of evidence and the strength of recommendations. Within the core group, we decided to use an adapted version of the ‘Applying classification of recommendations and level of evidence’ criteria of the American Heart Association [29] and of the Grading of Recommendations Assessment Development and Evaluation (GRADE) criteria of the GRADE working group [30] (see Table I). Concerning the levels of evidence, the conclusions of the original papers will be scored based on clinical and scientific expertise following three categories: A, high level of evidence; B, moderate/low level of evidence; and C, very low level of evidence. Recommendations will be classified into four categories: class I (green), strong recommendations to do; class IIa (yellow), moderate recommendation to do; class IIb (orange), weak recommendation to do; class III (red), recommendation not to do.

Table I
Criteria for grading the levels of evidence and strength of recommendationsa

Topic selection

The core group selected the first two topics: secondary breast cancer and cardiomyopathy surveillance, considering the prevalence and severity of these outcomes as well as the availability of accurate screening modalities and effective treatment options. These same principles were deliberated in a modified Delphi survey undertaken by three authors (LK, RM, KO) to determine which surveillance recommendations should be next on the overall harmonization agenda. This method is a well-recognized process to achieve consensus, relying on anonymity, iteration, controlled feedback, and statistical aggregation of group responses [31].

Between July 2011 and October 2011 panelists completed a series of two questionnaires. Among 36 experts invited to participate in the Delphi panel, 30 returned the Round One questionnaire. One participant passively dropped out of the panel after the first round and four additional members of the harmonization initiative group participated in the second round. In all, 33 experts completed and returned the Round Two questionnaire. In total, the panel consisted of 26 members of the International Guideline Harmonization Group and 8 additional experts working in the field of late effects in childhood and young adult cancer survivors.

The Delphi process is outlined in Figure 3. In the first Delphi round, the panelists were asked to rate four questions for each late effect on a 5-point Likert scale ranging from 1 (low ranking) to 5 (high). The panelists had the option to suggest late effects that were missing. Late effects were selected for the second Delphi round if the mean scores of all four questions were at least 2.5 (see Table II). The second Delphi round included the 20 highest-scoring late effects for re-rating, as well as 12 write-in additions. The panelists were asked to rate how important it is that the international harmonization endeavor addresses the late effect. The experts had to answer this question on a 4-point Likert scale ranging from 1 (less important) to 4 (most important). Mean rating scores for each item are presented in Table III. Based on the two Delphi rounds the panelists determined that the following surveillance recommendations will be prioritized in upcoming harmonization endeavors: 1) gonadal ovarian and testicular dysfunction, 2) coronary artery disease, 3) central nervous system (CNS) malignancies, 4) growth hormone deficiency, 5) neurocognitive deficits. After the completion of the harmonization of these topics, we will proceed based on the importance score as rated in the Delphi survey.

Figure 3
The Delphi process
Table II
Results Delphi Round One questionnaire
Table III
Results Delphi Round Two questionnaire

Publication and implementation

To facilitate dissemination and implementation of the harmonized recommendations, the evidence and deliberations related to each topic will be summarized in a manuscript appropriate for publication in a peer-reviewed journal. The evidence summaries will be published in additional online tables.

Until the International Guideline Harmonization Group has completed the harmonization of the entire long-term follow-up guideline content, the four guideline groups will incorporate the new harmonized recommendations in their existing guidelines. In addition, the harmonized recommendations will give new guideline opportunities for countries without any guidelines.

To facilitate future updating for the topic guidelines an interactive website will be developed. Within this website, the evidence summaries, uniform recommendations and research agenda will be available as well as an interactive tool to add new evidence and to adapt the recommendations by experts in the field if necessary.

For years it has been known that development of a guideline does not mean that this guideline will be implemented in practice [32]. To optimize implementation of future guidelines, special attention should be paid to obstacles in the professional setting, in patient behavior, in the organization of care and in available resources. The development of specialized education for professionals, health policy makers and patients, development of interactive clinical decision-support tools, shared care models, and quality indicators to monitor the process and the yield of surveillance will help to overcome obstacles to the implementation of our future guidelines. Focused implementation strategies will be important to improve the care for all survivors of childhood and young adult cancer.


Until recently guideline development for the surveillance of childhood young adult cancer survivors had significant shortcomings. With the initiation of this international guideline harmonization endeavor we have taken the first step in optimizing collaboration in guideline development that we anticipate will reduce duplication of effort and improve the quality of care for survivors of childhood and young adult cancer. We welcome all experts who are willing to contribute to the initiative.

Supplementary Material

Supp Appendix S1

Supp Appendix S2


We would like to thank the following people for their participation in the Delphi survey: Mark Brougham, Richard Cohn, Fiona Cowie, Rachel Cox, Elvira van Dalen, Mary Dwyer, Angela Edgar, Riccardo Haupt, Mike Hawkins, Tara Henderson, Marry van den Heuvel, Lars Hjorth, David Hodgson, Yasushi Ishida, Hiroyuki Ishiguro, Helen Jenkinson, Shunicho Kato, Alison Leiper, Miho Maeda, Paul Nathan, Helena van der Pal, Liedeke Postma, Cécile Ronckers, Charles Sklar, Wim Tissing, and Susumu Yokoya.

Role of funding source This work was supported by the Dutch Cancer Society, Amsterdam, the Netherlands. The sponsor had no involvement in the design and content of the project, in writing the manuscript and in the decision to submit it for publication.


Conflict of interest statement The authors have declared no conflicts of interest.


1. Curry HL, Parkes SE, Powell JE, Mann JR. Caring for survivors of childhood cancers: the size of the problem. Eur J Cancer. 2006;42:501–508. [PubMed]
2. Gatta G, Zigon G, Capocaccia R, et al. Survival of European children and young adults with cancer diagnosed 1995-2002. Eur J Cancer. 2009;45:992–1005. [PubMed]
3. Howlader N, Noone AM, Krapcho M, Neyman N, Aminou R, Waldron W, Altekruse SF, Kosary CL, Ruhl J, Tatalovich Z, Cho H, Mariotto A, Eisner MP, Lewis DR, Chen HS, Feuer EJ, Cronin KA, Edwards BK, editors. SEER Cancer Statistics Review, 1975-2008. National Cancer Institute; Bethesda, MD: 2011., based on November 2010 SEER data submission, posted to the SEER web site.
4. Childhood Cancer Survivorship . Improving Care and Quality of Life. National Cancer Policy Board; Washington, DC: 2003. [PubMed]
5. Oeffinger KC, Mertens AC, Sklar CA, et al. Chronic health conditions in adult survivors of childhood cancer. N Engl J Med. 2006;355:1572–1582. [PubMed]
6. Geenen MM, Cardous-Ubbink MC, Kremer LC, et al. Medical assessment of adverse health outcomes in long-term survivors of childhood cancer. JAMA. 2007;297:2705–2715. [PubMed]
7. Wallace HB, Green DM, editors. Late effects of childhood cancer. Arnold Publication; London: 2004.
8. Yeh JM, Nekhlyudov L, Goldie SJ, et al. A model-based estimate of cumulative excess mortality in survivors of childhood cancer. Ann Intern Med. 2010;152:409–8. [PMC free article] [PubMed]
9. Wallace WH, Blacklay A, Eiser C, et al. Developing strategies for long term follow up of survivors of childhood cancer. BMJ. 2001;323:271–274. [PMC free article] [PubMed]
10. Oeffinger KC, Hudson MM, Landier W. Survivorship: childhood cancer survivors. Prim Care. 2009;36:743–780. [PubMed]
11. IOM (Institute of Medicine) Clinical Practice Guidelines We Can Trust. The National Academies Press; Washington, DC: 2011.
12. Gibbons RJ, Smith SC, Jr., Antman E. American College of Cardiology/American Heart Association clinical practice guidelines: Part II: evolutionary changes in a continuous quality improvement project. Circulation. 2003;107:3101–3107. [PubMed]
13. Califf RM, Peterson ED, Gibbons RJ, et al. Integrating quality into the cycle of therapeutic development. J Am Coll Cardiol. 2002;40:1895–1901. [PubMed]
14. Bero LA, Grilli R, Grimshaw JM, et al. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ. 1998;317:465–468. [PMC free article] [PubMed]
15. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342:1317–1322. [PubMed]
16. Woolf SH, Grol R, Hutchinson A, et al. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318:527–530. [PMC free article] [PubMed]
17. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999;281:1900–1905. [PubMed]
18. Thomson R, McElroy H, Sudlow M. Guidelines on anticoagulant treatment in atrial fibrillation in Great Britain: variation in content and implications for treatment. BMJ. 1998;316:509–513. [PMC free article] [PubMed]
19. Richmond TK, Freed GL, Clark SJ, Cabana MD. Guidelines for adolescent well care: is there consensus? Curr Opin Pediatr. 2006;18:365–370. [PubMed]
20. Grilli R, Magrini N, Penna A, et al. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000;355:103–106. [PubMed]
21. Landier W, Bhatia S, Eshelman DA, et al. Development of risk-based guidelines for pediatric cancer survivors: the Children’s Oncology Group Long-Term Follow-Up Guidelines from the Children’s Oncology Group Late Effects Committee and Nursing Discipline. J Clin Oncol. 2004;22:4979–4990. [PubMed]
22. Children’s Oncology Group Long-term follow-up guidelines for survivors of childhood, adolescent, and young adult cancers. (Version 3.0) 2008 Available online:
23. Dutch Childhood Oncology Group . Richtlijn follow-up na kinderkanker meer dan 5 jaar na diagnose. SKION; Den Haag/Amsterdam: 2010. Available online:
24. Scottish Intercollegiate Guidelines Network Long term follow up of survivors of childhood cancer. A national clinical guideline. 2004 Available online:
25. United Kingdom Children’s Cancer Study Group Late Effects Group Therapy based long term follow up practice statement. 2011 Available online:
26. The AGREE Collaboration Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12:18–23. [PMC free article] [PubMed]
27. European Symposium on Late Complications after Childhood Cancer.
28. International Conference on Long-Term Complications of Treatment of Children and Adolescents for Cancer. [PubMed]
29. Gibbons RJ, Smith S, Antman E. American College of Cardiology/American Heart Association clinical practice guidelines: Part I: where do they come from? Circulation. 2003;107:2979–2986. [PubMed]
30. GRADE Working Group Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490–1494. [PMC free article] [PubMed]
31. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311:376–380. [PMC free article] [PubMed]
32. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–1230. [PubMed]