PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmjoInstructions for authorsCurrent ToCBMJ Open
 
BMJ Open. 2012; 2(5): e001918.
Published online Oct 26, 2012. doi:  10.1136/bmjopen-2012-001918
PMCID: PMC3488737
Consensus diagnostic criteria for fetal alcohol spectrum disorders in Australia: a modified Delphi study
Rochelle E Watkins,1 Elizabeth J Elliott,2,3,4 Raewyn C Mutch,1,5 Janet M Payne,1 Heather M Jones,1 Jane Latimer,4 Elizabeth Russell,6 James P Fitzpatrick,2,4 Lorian Hayes,7 Lucinda Burns,8 Jane Halliday,9 Heather A D'Antoine,10 Amanda Wilkins,1,5 Elizabeth Peadon,2,3 Sue Miers,11 Maureen Carter,12 Colleen M O'Leary,1,13 Anne McKenzie,1 and Carol Bower1
1Telethon Institute for Child Health Research, Centre for Child Health Research, The University of Western Australia, Perth, Western Australia, Australia
2Discipline of Paediatrics and Child Health, Sydney Medical School, University of Sydney, Sydney, New South Wales, Australia
3The Children's Hospital at Westmead, Sydney, New South Wales, Australia
4The George Institute for Global Health, Sydney, New South Wales, Australia
5Department of Health Western Australia, Child and Adolescent Health Service, Perth, Australia
6Russell Family Fetal Alcohol Disorders Association, Cairns, Queensland, Australia
7Centre for Chronic Disease, School of Medicine, University of Queensland, Brisbane, Queensland, Australia
8National Drug and Alcohol Research Centre, University of New South Wales, Sydney, New South Wales, Victoria, Australia
9Public Health Genetics, Genetic Disorders, Murdoch Childrens Research Institute, Melbourne, Australia
10Menzies School of Health Research, Charles Darwin University, Darwin, Northern Territory, South Australia, Australia
11National Organisation for Fetal Alcohol Syndrome and Related Disorders, Adelaide, Australia
12Nindilingarri Cultural Health Services, Fitzroy Crossing, Western Australia, Australia
13Centre for Population Health Research, Curtin University, Perth, Western Australia, Australia
Correspondence to Dr Rochelle E Watkins; rwatkins/at/ichr.uwa.edu.au
Received August 14, 2012; Accepted September 26, 2012.
Objective
To evaluate health professionals' agreement with components of published diagnostic criteria for fetal alcohol spectrum disorders (FASD) in order to guide the development of standard diagnostic guidelines for Australia.
Design
A modified Delphi process was used to assess agreement among health professionals with expertise or experience in FASD screening or diagnosis. An online survey, which included 36 Likert statements on diagnostic methods, was administered over two survey rounds. For fetal alcohol syndrome (FAS), health professionals were presented with concepts from the Institute of Medicine (IOM), University of Washington (UW), Centers for Disease Control (CDC), revised IOM and Canadian diagnostic criteria. For partial FAS (PFAS), alcohol-related neurodevelopmental disorder (ARND), and alcohol-related birth defects (ARBD), concepts based on the IOM and the Canadian diagnostic criteria were compared.
Setting/participants
130 Australian and 9 international health professionals.
Results
Of 139 health professionals invited to complete the survey, 103 (74.1%) responded, and 74 (53.2%) completed one or more questions on diagnostic criteria. We found consensus agreement among participants on the diagnostic criteria for FAS, with the UW criteria most commonly endorsed when compared with all other published criteria for FAS. When health professionals were presented with concepts based on the Canadian and IOM diagnostic criteria, we found consensus agreement but no clear preference for either the Canadian or IOM criteria for the diagnosis of PFAS, and no consensus agreement on diagnostic criteria for ARND. We also found no consensus on the IOM diagnostic criteria for ARBD.
Conclusions
Participants indicated clear support for use of the UW diagnostic criteria for FAS in Australia. These findings should be used to develop guidelines to facilitate improved awareness of, and address identified gaps in the infrastructure for, FASD diagnosis in Australia.
Article summary
Article focus
  • There are no recommended standard criteria for diagnosis of fetal alcohol spectrum disorder (FASD) in Australia and there is a little information on clinician practice and preference to guide the development of guidelines for diagnosis.
  • We aimed to evaluate health professionals' agreement with components of published diagnostic criteria for FASD in order to guide the development of standard diagnostic guidelines for Australia.
Key messages
  • There is a clear consensus among health professionals on the diagnostic criteria for fetal alcohol syndrome (FAS) in Australia.
  • National guidelines for FASD diagnosis in Australia should incorporate components of the University of Washington diagnostic criteria.
Strengths and limitations of this study
  • Our findings provide evidence that is relevant to the development of guidelines for the diagnosis of FASD in Australia based on consultation with health professionals.
  • Evaluation of diagnostic criteria for partial FAS (PFAS), alcohol-related neurodevelopmental disorder (ARND), and alcohol-related birth defects only compared concepts from the original (Institute of Medicine) and the most recent (Canadian) published guidelines. Based on our finding of a clear preference for the University of Washington criteria for FAS, further work is required to identify whether there is consensus agreement for University of Washington criteria for PFAS and ARND.
Prenatal exposure to alcohol is associated with a wide range of impacts,1 including intellectual disability, behavioural disorders, growth restriction, birth defects and dysmorphic facial features.2 Among the range of diagnostic outcomes that may be identified, fetal alcohol syndrome (FAS) is most easily recognised.3 4 However, the spectrum of disorders more frequently includes neurodevelopmental disorders that are not accompanied by the characteristic FAS-facial anomalies and are more difficult to diagnose.4
Information on the diagnosis of fetal alcohol spectrum disorders (FASD) in Australia is limited, with no routine national surveillance for FASD, no recommended national standards for diagnosis and evidence of inconsistent and inadequate5–7 diagnostic practices. In a large survey of paediatricians in Western Australia approximately 20% were able to identify the diagnostic features of FAS, despite almost 50% reporting that they had diagnosed the condition.8 The absence of accepted guidelines for referral and diagnosis has likely contributed to poor case ascertainment, misdiagnosis, and a consequent lack of access to health, education and social services.9
Variation in the diagnosis of FASD between practitioners can be attributed to the absence of well-defined criteria for assessment,1 poor awareness of diagnostic criteria and the inconsistent application of guidelines for diagnosis of FASD.2 4 9–11 Diagnostic guidelines are not always implemented consistently12 and there is substantial overlap between guidelines used internationally. The most recently published diagnostic guidelines2 sought to provide an agreed, evidence-based diagnostic standard for Canada. The Canadian guidelines harmonise the two main approaches to diagnosis, adopting terminology from the Institute of Medicine (IOM) diagnostic criteria4 and the University of Washington (UW) FASD 4-Digit Diagnostic Code10 approach to diagnostic assessment and measurement.2
A standardised national approach to the diagnosis of FASD would improve diagnostic capacity and consistency in Australia; however, there is little empirical information available to compare the performance of published guidelines and inform the development of standard guidelines. There are no internationally agreed standards for FASD diagnosis and a survey of health professionals demonstrated that most were unsure about whether any of the existing diagnostic guidelines should be adopted in Australia.13 Consistent with the recognised need to evaluate guidelines when they are adopted in different contexts14 15 we aimed to evaluate health professionals' agreement with components of published diagnostic criteria for FASD in order to guide the development of standard diagnostic guidelines for Australia.
Questionnaire design
A modified Delphi process16 with two survey rounds was used to assess health professionals' agreement with diagnostic criteria for FASD derived from existing diagnostic guidelines.2 4 9–11 Agreement with different methods of service delivery, including several concepts included in the Western Australian FASD model of care17 were also explored. The round 1 questionnaire included: 9 statements which evaluated agreement with general diagnostic processes; 21 statements which evaluated agreement with general and specific components of the IOM,4 UW,10 Centers for Disease Control (CDC),9 revised IOM11 and Canadian2 diagnostic criteria for FAS; and six statements which evaluated agreement with diagnostic criteria for partial FAS (PFAS), alcohol-related neurodevelopmental disorder (ARND) and alcohol-related birth defects (ARBD) based on a comparison of concepts from the original IOM diagnostic criteria4 and the more recently developed Canadian guidelines.2 Participants were asked to rate their agreement with each statement on a five-point Likert scale which ranged from ‘strongly agree’ to ‘strongly disagree’ or to select ‘no comment’ if a statement was outside their area of expertise. These statements on diagnosis were administered as part of a larger survey on the screening and diagnosis of FASD in Australia.
Panel recruitment
As the Delphi study method requires participants with expertise relevant to the study objective,18–20 a large panel of health professionals with expertise or experience in the screening and diagnosis of FASD was recruited. The first round of the survey started with 139 panel members: 40 were recruited having reported a child with FAS to the Australian Paediatric Surveillance Unit (APSU) in a previous study,5 68 were identified by study investigators as having experience or expertise in FASD screening or diagnosis (including nine international experts) and 31 responded to calls to health-professional organisations for individuals with relevant experience or expertise.
Questionnaire administration
The password-protected questionnaire was administered online from a secure web server. A personalised email was sent to all panel members inviting them to complete the first round within 14 days. Two email reminders about questionnaire completion were sent prior to the round deadline. Non-responders for whom we had contact details were contacted by telephone and given another 8 days to respond. Participants who did not complete the round 1 questionnaire were excluded from round 2. Due to the potential for feedback about findings from round 1 to influence participation in round 2, individuals who first responded to the statements on diagnostic criteria in round 2 were excluded from this analysis.
Questionnaire revision
The following criteria were used to determine whether consensus agreement was reached on the statements included in round 1 and whether they required reassessment in round 2 or were rejected:
  • if at least 70% of participants agreed or strongly agreed with a statement, it was considered endorsed. This level of consensus was decided a priori. Endorsed statements were omitted from the round 2 questionnaire if not closely related to other statements included in round 2;
  • if 60–69% of participants agreed or strongly agreed with a statement, it was re-administered in round 2 in its original or in a modified form;
  • if fewer than 60% of participants agreed or strongly agreed with a statement, the statement was rejected or modified.
In the round 2 questionnaire we provided feedback of group and individual agreement with statements in round 1.
Analysis
Descriptive statistics were generated for each statement, including response frequencies and dispersion (interquartile deviation). Achievement of consensus was evaluated based on all valid responses. Associations between statement ratings and individual characteristics of participants (occupation, experience in FASD diagnosis, completion of training on FASD diagnosis and completeness of response) were explored using the χ test or Fisher's exact test.21 The Wilcoxon-signed rank test was used to compare the level of agreement between IOM and Canadian diagnostic criteria for PFAS and ARND. All analyses were evaluated using two-tailed test statistics. Approval for this study was granted by The University of Western Australia Human Research Ethics Committee and the Western Australian Aboriginal Health Information and Ethics Committee.
Although 103 participants (74.1%) responded to the survey, only 74 (53.2%) answered one or more of the 27 round 1 questionnaire statements on diagnostic criteria. Sociodemographic characteristics for these 74 participants are summarised in table 1.
Table 1
Table 1
Participant characteristics by occupational group
Diagnostic services
Most participants agreed (responded ‘agree’ or ‘strongly agree') that a medical specialist (eg, paediatrician or clinical geneticist) should confirm the diagnosis of FASD (78.6% after round 2) and exclude alternative diagnoses (89.5% after round 2). Although some participants commented that diagnosis by general practitioners may be appropriate in rural and remote areas, there was not consensus agreement on the involvement of general practitioners in diagnosis in rural and remote areas (table 2). There was consensus agreement on the need for multidisciplinary assessment, that multidisciplinary assessment clinics should be available in major cities and that assessment teams should visit regional centres to support workforce training and development. Paediatricians were less likely than other health professionals to agree on the need for multidisciplinary assessment and scheduled visits to regional centres (table 2).
Table 2
Table 2
Comparison of agreement with statement about diagnostic processes and services between paediatricians and other health professionals
Diagnostic criteria for FAS
There was clear consensus agreement on general and specific statements regarding diagnostic criteria for FAS that were consistent with the UW guidelines (tables 3 and and4).4). Most participants (75.5% overall) agreed with components of the UW criteria for central nervous system (CNS) abnormality (statement 11 table 4). There was a substantial difference in agreement between the UW and the IOM criteria (44.9% overall, statement 10 table 4) for CNS abnormality (z=-2.93, p=0.003); however, there was no strong evidence of difference between the Canadian (60.0% overall, statement 12 table 4) and the IOM criteria (z=-1.42, p=0.16). Fewer than half the paediatricians (42.9%) agreed with the Canadian criteria for CNS abnormality compared with 81.8% of other health professionals (statement 12 table 4).
Table 3
Table 3
Comparison of agreement with statements about general diagnostic criteria for FAS between paediatricians and other health professionals
Table 4
Table 4
Comparison of agreement with statements about specific diagnostic criteria for FAS between paediatricians and other health professionals
Diagnostic criteria for PFAS, ARND and ARBD
There was consensus agreement on both definitions of PFAS, with no evidence of difference in agreement between the IOM (70.6% overall, statement 1, table 5) and Canadian (74.0% overall, statement 2, table 5) criteria (z=-0.50, p=0.62). There was no consensus agreement on the diagnostic criteria for ARND, with no evidence of difference in agreement between the IOM (59.5% overall, statement 3, table 5) and Canadian (57.5% overall, statement 4, table 5) criteria (z=0.31, p=0.75). No consensus agreement was reached on the IOM definition of ARBD (64.4% overall, statement 6, table 5), and 90.4% of participants agreed that better evidence is required on the association between alcohol and birth defects before the diagnostic category is clinically useful (table 5).
Table 5
Table 5
Comparison of agreement with statements about diagnostic criteria for PFAS, ARND and ARBD between paediatricians and other health professionals
Response completeness and survey non-response
Only 36 of the 74 participants (48.6%) completed 22 or more of the 27 diagnostic criteria statements. Most participants who completed 21 or fewer of the 27 diagnostic criteria statements indicated through use of the ‘no comment’ response that the omitted statements were outside their area of expertise. Completion of 21 or fewer diagnostic criteria statements was more frequent among other health professionals (65.8%), participants who had no experience in diagnosis (67.6%) and participants who had not completed training on diagnosis (86.1%) than among paediatricians (34.2%, p=0.04), participants who had experience in diagnosis (31.4%, p=0.002) and participants who had completed training on diagnosis (13.9%, p=0.001), respectively.
Among the 74 participants, those who completed 21 or fewer diagnostic criteria statements were less likely to agree with the general definition of the four features of FAS in round 1 (44.4%, statement 1 table 3) compared with participants who completed 22 or more statements (75%, p=0.01). Compared with participants who completed 22 or more statements, participants who completed 21 or fewer statements were also more likely to agree with the IOM criteria for facial anomalies (73.7%, statement 3 table 3 vs 45.7%, p=0.045), IOM criteria for CNS abnormality (71.4%, statement 10 table 3 vs 34.3%, p=0.02) and general criteria for growth deficit (statements 5, 6 and 7 table 3, all p<0.02). The exclusion of individuals who responded to the statements on diagnostic criteria for the first time in the round 2 questionnaire did not influence the achievement of consensus agreement for any statement or alter the conclusions drawn from this analysis (data not shown).
Although there was no evidence of an association between occupation and completion of any questionnaire statement on diagnostic criteria either among the 139 panel members (p=0.2) or among the 103 panel members who responded to the survey (p=0.46); non-response to the survey overall (36/139 panel members) was more frequent among paediatricians (36.6%) than among other health professionals (14.7%, p=0.003). Among the 103 survey participants, paediatricians were also more likely to complete 22 or more of the 27 diagnostic criteria statements (46.7%) than other health professionals (25.9%, p=0.03).
We found consensus agreement among participants on the diagnostic criteria for FAS. Diagnostic criteria for FAS are the most well-established22 and all published guidelines2 4 9–11 include similar and necessary elements: characteristic facial dysmorphology, growth impairment and CNS abnormality. However, there are important differences between guidelines. Participants indicated a clear preference for all main components of the case-defined UW criteria, including a substantially greater level of agreement with the UW criteria for CNS abnormality than with the IOM criteria.
The endorsed UW definition of CNS abnormalities includes either structural/neurological criteria or functional criteria that are each alone sufficient to define CNS abnormality. This may allow an FAS diagnosis to be made earlier than with use of the Canadian guidelines which, in the absence of structural CNS defects, require deficit in at least three domains of function and thus have greater dependence on neurobehavioural assessments whose use may be limited by age.2 The requirement for evidence of structural/neurological or substantial functional abnormality is common to both the UW and CDC guidelines.9 However, the use of the third percentile criterion for cranial size and functional performance indicators distinguishes the UW and CDC criteria, and results in a more conservative UW definition of abnormality.
We found no significant difference in agreement between the IOM and Canadian criteria for the diagnosis of PFAS, ARND or criteria for CNS abnormality. Both statements describing the IOM and Canadian diagnostic criteria for PFAS reached consensus agreement despite considerable differences between the criteria, including that, unlike the IOM criteria, the Canadian criteria require CNS dysfunction to be formally established.2 Lack of consensus agreement with either the Canadian or IOM diagnostic criteria for FASD other than FAS is consistent with more limited evidence on the aetiological role of alcohol for these diagnoses,4 11 23 variation in the definition of these conditions among diagnostic guidelines, limited information on the validity and comparative performance of different diagnostic methods, and exclusion of the diagnostic category ARBD from three of the four most recently published diagnostic guidelines.2 9 10
The lack of a clear preference for either the Canadian or IOM diagnostic criteria for PFAS and ARND may also be linked to the limited use and visibility of diagnostic categories other than FAS in Australia.6 This could both result from, and reinforce, uncertainty about the validity of these diagnostic categories. Although the Canadian and UW diagnostic criteria share many similarities, there are some differences between these approaches. Further exploration of agreement with the UW diagnostic criteria for PFAS and ARND is needed to distinguish a lack of support for the diagnostic category of ARND and a lack of clear preference for diagnostic criteria for PFAS from the specific and limited diagnostic criteria evaluated in this survey. However, our finding of substantially greater agreement with the UW definition of components of the diagnostic criteria for FAS suggests that there is strongest support for the UW approach to defining abnormalities, which also form the basis of the UW diagnostic criteria for PFAS and ARND (static encephalopathy alcohol exposed and neurobehavioural disorder-alcohol exposed).
Consistent with more recent guidelines for the diagnosis of FASD,2 9–11 a multidisciplinary approach to diagnosis was supported, particularly among health professionals other than paediatricians. The observed difference in agreement among paediatricians and other health professionals may reflect the difficulty in accessing multidisciplinary assessment services outside major metropolitan centres. Although participants were more likely to agree that general practitioners in rural or remote settings could confirm a diagnosis of FASD compared with all general practitioners, there was only limited support for general practitioners assuming a key role in diagnosis. Comments from some participants indicated that the lack of support for the role of general practitioners in diagnosis may be associated with the perceived need for specific expertise in FASD diagnosis, a lack of time to complete the diagnostic process in general practice, and the perceived need for a specialist multidisciplinary diagnostic team. Participants indicated most agreement with service provision strategies that help to build local diagnostic and intervention capacity. A similar strategy is advocated in the Canadian guidelines to address service provision in remote communities by establishing regionally based diagnostic teams.2
Limitations of this study include its exploratory design, the inability of the survey to represent published diagnostic criteria within their full context, and the failure to assess agreement with all published diagnostic criteria for PFAS and ARND. Despite recruiting 139 individuals known to have experience or expertise in screening or diagnosis of FASD and a high response (74.1%), only 74 participants answered one or more of the 27 questions on diagnostic criteria, and only 36 answered 22 or more questions. However, due to the recruitment of panel members based on experience or expertise on screening or diagnosis and the examination of both screening and diagnosis in the questionnaire, we did not anticipate that all panel members would respond to the statements on diagnostic criteria. We found that completeness of response to the statements on diagnostic criteria was related to training and experience in FASD diagnosis, and likely reflects the limited familiarity with diagnostic guidelines among Australian health professionals,13 and the limited diagnostic capacity for FASD in Australia.
In conclusion, we found consensus agreement on the UW criteria for the diagnosis of FAS, and few important differences between the perceptions of paediatricians and other health professionals. Our findings indicate strong support for the UW approach to defining abnormalities which fulfils key best practice criteria,24 and these data provide valuable consensus-based evidence for guideline development that should be incorporated in formal guideline development processes for FASD diagnosis in Australia. The development of standard-national guidelines for diagnosis can facilitate improved awareness of diagnostic criteria, the use of consistent diagnostic processes, and enable evaluation of the effectiveness of the prevention, detection and treatment of these disorders in Australia.
Supplementary Material
Author's manuscript
Reviewer comments
Acknowledgments
We particularly thank the health professionals who completed the survey. We acknowledge the contributions of Alison Anderson who provided specialist programming advice, Laura Bond who was employed as a project officer and Dr Bill Kean who was a project observer appointed by the Australian Government Department of Health and Ageing.
Footnotes
Contributors: CB, EJE and JMP designed the study and CB and EJE supervised the study. CB, EJE, REW, JL and HJ designed the study questionnaire, and all authors were members of a project steering group that reviewed the study methods and procedures. REW programmed the online questionnaire and analysed the data. REW drafted the manuscript and all authors edited the manuscript. All authors read and approved the final version of the manuscript.
Funding: This study was funded by the Australian Government Department of Health and Ageing. Individual contributions were also supported by the National Health and Medical Research Council (NHMRC) Research Fellowships (CB 634341 and JH 1021252), an NHMRC Program Grant (CB and JMP 572742), NHMRC Practitioner Fellowships (EJE 457084 and 1021480), an NHMRC Enabling Grant (EJE and CB 402784) and an Australian Research Council Future Fellowship (JL FT0991861).
Competing interests: None.
Ethics approval: Approval for this study was granted by The University of Western Australia Human Research Ethics Committee and the Western Australian Aboriginal Health Information and Ethics Committee.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: There are no additional data available.
1. Astley SJ, Clarren SK. Diagnosing the full spectrum of fetal alcohol-exposed individuals: introducing the 4-digit diagnostic code. Alcohol Alcohol 2000;35:400–10. [PubMed]
2. Chudley AE, Conry J, Cook JL, et al. Fetal alcohol spectrum disorder: Canadian guidelines for diagnosis. Can Med Assoc J 2005;172:S1–21. [PMC free article] [PubMed]
3. Elliott EJ, Bower C. Alcohol and pregnancy: the pivotal role of the obstetrician. Aust N Z J Obstet Gynaecol 2008;48:236–9. [PubMed]
4. Stratton K, Howe C, Battaglia F. Fetal alcohol syndrome: diagnosis, epidemiology, prevention, and treatment. Washington, DC: National Institute of Medicine, Academy Press, 1996.
5. Elliott E, Payne J, Morris A, et al. Fetal alcohol syndrome: a prospective national surveillance study. Arch Dis Child 2008;93:732–7. [PubMed]
6. Harris KR, Bucens IK. Prevalence of fetal alcohol syndrome in the top end of the Northern Territory. J Paediatr Child Health 2003;39:528–33. [PubMed]
7. Allen K, Riley M, Goldfeld S, et al. Estimating the prevalence of fetal alcohol syndrome in Victoria using routinely collected administrative data. Aust N Z J Public Health 2007;31:62–6. [PubMed]
8. Elliott E, Payne J, Haan E, et al. Diagnosis of foetal alcohol syndrome and alcohol use in pregnancy: a survey of paediatricians’ knowledge, attitudes and practice. J Paediatr Child Health 2006;42:698–703. [PubMed]
9. Bertrand J, Floyd RL, Weber MK. Guidelines for identifying and referring persons with fetal alcohol syndrome. MMWR 2005;54:1–14. [PubMed]
10. Astley SJ. Diagnostic guide for fetal alcohol spectrum disorders: the 4-digit diagnostic code. 3rd edn Seattle: University of Washington, 2004.
11. Hoyme HE, May PA, Kalberg WO, et al. A practical clinical approach to diagnosis of fetal alcohol spectrum disorders: clarification of the 1996 institute of medicine criteria. Pediatrics 2005;115:39–47. [PMC free article] [PubMed]
12. Peadon E, Fremantle E, Bower C, et al. International survey of diagnostic services for children with fetal alcohol spectrum disorders. BMC Pediatr 2008;8:12. [PMC free article] [PubMed]
13. Watkins RE, Elliott EJ, Mutch RC, et al. Health professionals’ perceptions of the adoption of existing guidelines for the diagnosis of fetal alcohol spectrum disorders in Australia. BMC Pediatr 2012;12:69. [PMC free article] [PubMed]
14. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ 2010;182:E839–42. [PMC free article] [PubMed]
15. Graham ID, Harrison MB. Evaluation and adaptation of clinical practice guidelines. Evid Based Nurs 2005;8:68–72. [PubMed]
16. Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Chichester: Wiley-Blackwell, 2011.
17. Department of Health Western Australia Fetal alcohol spectrum disorder model of care. Perth: Health Networks Branch, Department of Health, Western Australia, 2010.
18. Keeney S, Hasson F, McKenna HP. A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud 2001;38:195–200. [PubMed]
19. Kingston AH, Jorm AF, Kitchener BA, et al. Helping someone with problem drinking: mental health first aid guidelines—a Delphi expert consensus study. BMC Psychiatry 2009;9:79. [PMC free article] [PubMed]
20. Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications. Info Manag 2004;42:15–29.
21. McDonald JH. Handbook of biological statistics. Baltimore, Maryland: Sparky House Publishing, 2009.
22. Centers for Disease Control and Prevention Fetal alcohol syndrome: guidelines for referral and diagnosis. Atlanta: National Task Force on Fetal Alcohol Syndrome and Fetal Alcohol Effect, 2004.
23. Astley SJ, Carmichael-Olson H, Kerns K, et al. Neuropsychological and behavioural outcomes from a comprehensive magnetic resonance study of children with fetal alcohol spectrum disorders. Can J Clin Pharmacol. 2009;16:178–201. [PubMed]
24. Mutch R, Peadon EM, Elliott EJ, et al. Need to establish a national diagnostic capacity for foetal alcohol spectrum disorders. J Paediatr Child Health 2009;45:79–81. [PubMed]
Articles from BMJ Open are provided here courtesy of
BMJ Group