1. Introduction
Epidemiology of Autism Spectrum Disorder (ASD)
Autism spectrum disorder (ASD) is a neurodevelopmental disorder associated with difficulties in social interaction, communication, and repetitive behaviors [1]. However, individuals with ASD have a range of cognitive abilities, from substantial intellectual disability to above-average cognitive functioning, and different language skills range from non-verbal to fluent. Additionally, affected individuals may exhibit co-occurring medical diseases such as epilepsy, gastrointestinal problems, and psychiatric comorbidities, resulting in multiple layers of clinical complexity [2,3].
The American Psychiatric Association recognized autism for the first time as a distinct clinical diagnostic in the third edition of the Diagnostic and Statistical Manual of Mental Disorders [4,5]. Revisions to the diagnostic criteria were made to the fourth edition of the DSM in 1994, and five subtypes of autism were added as a result: autistic disorder, Asperger disorder, pervasive developmental disorder-not otherwise specified (PDD-NOS), childhood disintegrative disorder, and Rett’s disorder [5]. However, the last two conditions fall under the more general category of pervasive developmental disorders, while the first three categories make up autism spectrum disorder (ASD) [2]. Furthermore, ASD was reclassified in the fifth version of the DSM [1] in 2013 as a single condition with three severity categories termed specifiers.
Despite numerous studies, the prevalence of ASD is still not well defined, with multiple studies worldwide reporting differing rates in various continents and countries. However, the World Health Organization [6] estimated in 2019 the global prevalence of ASD to be 6.25 cases per 1000. The most recent version (with 2018 data) published by the Center for Disease Control and Prevention Agency (CDC), USA, which operates the Autism and Developmental Disabilities Monitoring Network, and assesses 11 North American states indicated that, among 8-year-old children, the prevalence of ASD ranged from 16.5 per 1000 in Missouri to 38.9 per 1000 in California. These data estimated an ASD prevalence of 23.0/1000, which is equivalent to 1 for every 44 children aged 8 years. It also showed that 80% of confirmed ASD cases were boys (4.2 times) [7].
The difference between genders also seems to be another relevant point in different populations [8,9]. It appears that boys are more affected (with a ratio of four times more frequent) when compared to girls [10]. Recent research has indicated that this explanation might be related to the under diagnosis of females with ASD, particularly when they are young [11]. In this context, the literature shows that studies have reported variable male-to-female ratios, varying from 2:1 to 5:1, documented in the literature [12,13].
In an attempt to improve this scenario and determine the real prevalence of ASD in different populations, early detection and correct diagnosis of individuals with ASD is critical. Two studies made an important discussion of this topic, pointing out that many ASD symptoms are noticeable in early childhood and allow diagnosis before 3 years of age [14,15]. However, despite this possibility of early diagnosis, there is a significant difference between the potential age of diagnosis and the actual age of diagnosis, which is often only performed when the child reaches school age [14,15].
Clinical practice guidelines (CPGs) offer guidance for enhancing patient care based on a systematic review (SR) of the available data and an evaluation of the advantages and disadvantages of different care options [16]. Many international evidence-based clinical practice guideline (CPG) programs have highlighted ASD as one of the top health priorities. The high volume of published ASD CPGs of variable quality has been quite confusing for ASD healthcare professionals and providers. The need has arisen for an updated systematic review and critical appraisal of these CPGs [17].
The AGREE II (Appraisal of Guidelines for REsearch and Evaluation) instrument is the gold standard for assessing the quality of CPGs. AGREE II has been validated, widely cited, and endorsed by a number of healthcare organizations [18,19,20]. AGREE II specifies components that CPGs should address in order to increase their quality and trustworthiness, as well as to achieve positive patient outcomes [18,20,21,22].
We devoted this study to reporting the outcomes of this SR and critically evaluating the recently released CPGs for children with ASD using the Appraisal of Guidelines for REsearch and Evaluation (AGREE) II, as it is a core step in the CPG adaptation process [21,22,23,24].
The main objective of this study was to update the SR of ASD CPGs that was published by Penner et al. that focused exclusively on diagnosis and to expand on the recommendations for treatment as well [25]. Reporting of the full CPG adaptation process with all of its steps and final results including the adapted CPG is out of the scope of this paper and will be published separately.
2. Materials and Methods
We started by registering the protocol for this study at the Center for Open Science (OSF) [24]. Our CPG adaptation group included two child and adolescent psychiatry consultants, a pediatric neurology consultant, three developmental behavioral pediatric consultants, a board-certified behavior analyst, a clinical psychology consultant, a senior speech and language therapist, a senior occupational therapist, and a general pediatrician and CPG methodologist. To contribute to this SR study, two international researchers in ASD joined as international collaborators.
In Saudi Arabia, there are no national CPGs for comprehensive recommendations for managing children with ASD based on evidence-based research. The Saudi Health Council, National Center for Developmental and Behavioral Disorders, and National Center for Evidence-Based Health Practice launched a national CPG adaptation project for the management of children with ASD in 2021 to address this gap and offer direction and recommendations to all pertinent stakeholders who care for these children in Saudi Arabia. The ‘KSU-Modified-ADAPTE’ was used to guide the project, which is a formal CPG adaption process including three phases: setup, adaptation, and finalization [21,22,23,24,26].
This systematic review followed the methodological guide for systematic reviews of guidelines in addition to the 2020 updated Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [23,27,28]. Systematic review registration: The research team registered the protocol in the Center for Open Science (OSF) in 2021
2.1. Data Sources and Search Strategy
We systematically searched bibliographic databases, CPG databases and repositories, and databases of ASD-related national and international professional societies (Supplementary Table S1).
The search was limited to CPGs published from November 2015 until March 2022. We decided on this period of time to update the search conducted in the previous SR of CPGs, as the last systematic review was conducted in 2018 [25]. The search was performed by a CPG methodologist (YSA). To facilitate the CPG eligibility identification procedure, we used the PIPOH (Population, Intervention, Professions, Outcomes, and Health care system) model and the PICAR statement (Population, Interventions, Comparators, Attributes of the CPG, and Recommendation characteristics) (Supplementary Tables S2 and S3) [21,22,23]. Titles and abstracts of the retrieved CPGs and papers matching the inclusion criteria were independently evaluated by two reviewers (A.H.A. and S.M.A.). Disagreements were resolved through focus group discussions following the retrieval and examination of full-text articles or complete CPG documents.
2.2. Eligibility Criteria
The following were the eligibility criteria: (1) evidence-based with described CPG development methods; (2) English or Arabic language (based on language abilities of the research team); (3) original source CPGs (de novo developed); (4) national or international scope; (5) published after October 2015; and (6) published with group or organization authorship that may be accessed from a CPG database or peer-reviewed journal. Each source CPG was only evaluated using the most recent version.
CPGs that were published in October 2015 or earlier, were not in English or Arabic, were derived from other CPGs, were presented as consensus or expert-based statements, had only one author, or had any of the aforementioned characteristics were all excluded.
2.3. The Capacity Building for Using the AGREE II Instrument
Through a hands-on training on evidence-based CPG standards and the use of the AGREE II instrument, the CPG methodologist delivered a workshop for capacity building for the CPG assigned appraisers. Following that, each reviewer graded their assigned CPGs. Each CPG was critically evaluated by all four reviewers. The whole CPG materials were read by all appraisers, including any revisions with any relevant supplemental information or links to internet web pages relating to CPG methodologies or CPG implementation tools. AGREE appraisers were told to document their reasoning for their scores in the “Comment” box for each item [29].
2.4. Quality Assessment of ASD CPGs Using AGREE II
Scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence are among the six domains that make up the AGREE II instrument’s 23 elements [18,19,29]. The AGREE II evaluation was completed using its online version, “My AGREE PLUS”, which creates a CPG appraisal group for each CPG and translates the item ratings into domain ratings and comments. Each item was graded on a 7-point Likert scale [18,19,29]. The four AGREE II appraisers included a consultant developmental behavioral pediatrician (SAA), a consultant clinical psychologist (MWB), a senior speech and language therapist (SA), and as well as a registered pediatrician with expertise in CPG methodologies (YSA).
Group discussions were used to settle significant disparities in the AGREE II appraisers’ scores of items or questions (i.e., a difference of 3 or more). The online My AGREE PLUS system automatically calculated the standardized AGREE II domain scores or ratings. For each domain score, we agreed on a cutoff point of 70%. Following the appraisal, greater emphasis was placed on the scores of domains 3 and 5 in order to facilitate the filtration and final evaluation of the reporting quality of the included CPGs. Identical cutoff values have been reported [30,31]. The evidence bases of the included CPGs, specifically their references sections, were examined for SRs or meta-analyses, particularly Cochrane SRs, through the references section of the included source CPGs in addition to the six AGREE II domains.
2.5. Analysis Plan
Using the AGREE II instrument manual, we calculated scores (0–100%) for each AGREE II domain. In a comparative tabular format, the key recommendations of the eligible CPGs were summarized. According to the rating of domain 3 (rigor of development), the quality of CPGs was categorized, with a high-quality CPG earning a standardized domain rating of more than or equal to 70%, a moderate quality CPG (40–69%), and a low-quality CPG (40%).
Inter-Rater Analysis
To measure the amount of agreement between raters, we performed inter-rater reliability tests (IRR). To examine the amount of agreement among the four raters, we employed a percent agreement inter-rater reliability assessment test for each question in each area in the four CPGs as well as the first overall assessment’s percent agreement. In order to assess the consistency of ratings or capacity for datasets collected as clusters or sorted into clusters, we also used intra-class correlation (the second overall assessment).
One of the most used IRR techniques is intra-class correlation (ICC). We resort to this technique when we have more than two raters. A high intra-class correlation coefficient (kappa) around one indicated great similarity between standards from the same set. A low Kappa value close to 0 suggested that standards from the same set are not identical. Because we had inconsistent raters/rates, we utilized ANOVA “One-Way Random” on SPSS Statistics, version 21. We chose ICC because of the variety of numerical data from groups or clusters. This aided us in determining repeatability as well as how closely peers resemble one another in terms of particular qualities or attributes. We looked at how well two ordinal scale categories agreed. Because the data came from an ordered scale, we employed Weighted Kappa (Quadratic Weights). The weights are determined as follows. Using the Cohen’s Kappa notation, we selected linear weights since the difference between the first and second categories was equivalent to the difference between the second and third categories, and so on. The Kappa (K) statistic [32,33] is used to quantify agreement: K equals 1 when there is complete agreement between the categorization systems, K equals 0 when there is no agreement greater than chance, and K is negative when agreement is worse than chance. Supplementary Table S4 shows how the K value might be interpreted [34].
3. Results
A total of 165 records were retrieved, with 161 being excluded based on eligibility criteria. According to the PRISMA flowchart [27], only four original source CPGs were found to be eligible for the quality assessment step (Supplementary Figure S1). Two reviewers (A.H.A., S.M.A.) conducted the screening, and two additional reviewers (S.M.A., Y.S.A.) discussed any discrepancies.
3.1. Characteristics of Included ASD CPGs
Table 1 outlines the characteristics of the four eligible CPGs. The CPG developer organizations included the Australian Autism CRC (ACRC), the Ministry of Health New Zealand (NZ), the National Institute for Health and Care Excellence (NICE), and the Scottish Intercollegiate Guidelines Network, Healthcare Improvement Scotland (SIGN-HIS). All these organizations were from high-income countries.
3.2. Quality Assessment of the ASD CPGs
Table 2 and Figure 1 and Figure 2 summarize the AGREE II standardized domain ratings.
3.2.1. Domain 1: Scope and Purpose
Domain 1 scores ranged from 93 to 99%. All four CPGs were higher than 70% (ACRC = 99%, NICE = 96%, NZ = 93%, and SIGN-HIS = 97%) [35,36,37,38].
All of the four appraised CPGs have specifically reported their overall objective or purpose, health questions, and patient population clearly [35,36,37,38].
3.2.2. Domain 2: Stakeholder Involvement
Domain 2 scores ranged from 71 to 99%. All four CPGs were higher than 70% (ACRC = 99%, NICE = 71%, NZ = 71%, and SIGN-HIS = 79%) [35,36,37,38].
All of the four appraised CPGs have specifically reported their CPG development groups, target end users, and patient views and preferences considerations [35,36,37,38].
3.2.3. Domain 3: Rigor of Development
AGREE II scores for Domain 3 were from 84 to 93%. In domain 3, the scores of all four CPGs were more than 70% (ACRC = 84%, NICE = 93%, NZ = 86%, and SIGN-HIS = 85%) [35,36,37,38].
All of the four appraised CPGs reported using systematic methods to search for evidence, evidence inclusion and exclusion criteria, evidence strengths and limitations, methods for formulating the recommendations, considering the risks and benefits in formulating the recommendations, links between the recommendations and supporting evidence, external review, and a clear process for review and update [35,36,37,38]. Both NICE and SIGN-HIS CPGs documented using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) method in their formal development methodology [37,38].
3.2.4. Domain 4: Clarity of Presentation
The AGREE II scores for domain 4 ranged from 93 to 97%. The scores of all four CPGs were above 70% (ACRC = 93%, NICE = 97%, NZ = 94%, and SIGN-HIS = 93%) [33,34,35,36]. All of the four appraised CPGs reported specific and clear detailed recommendations, different management options, and key or summary recommendations [35,36,37,38].
Supplementary Table S5 provides a comparison between the recommendations of the four assessed CPGs in a recommendation matrix format.
3.2.5. Domain 5: Applicability
The AGREE II scores for domain 5 reported from 54 to 92%. The scores of three CPGs were higher than 70% (ACRC = 92%, NICE = 89%, and SIGN-HIS = 85%) [30,37,38]. All the four appraised CPGs reported noted implementation considerations including facilitators and barriers to application, different CPG implementation tools, resource implications, and auditing criteria [35,36,37,38].
3.2.6. Domain 6: Editorial Independence
Scores for domain 6 ranged from 69 to 92%. The scores of three CPGs were more than 70% (ACRC = 92%, NICE = 92%, and SIGN-HIS = 92%) [30,32,33]. All appraised CPGs reported the funding body and the competing interests of the CPG development group [35,36,37,38].
3.2.7. Overall Assessment
The AGREE II standardized domain scores ranged from 88 to 96% for the first overall assessment. Scores of higher than 70% were received by all four CPGs, which correlated to excellent performance across all six AGREE II domains.
3.2.8. Recommending the ASD CPGs for Use in Practice
The second overall assessment (i.e., recommendation for using the CPG in practice) showed that all four high-quality CPGs were recommended by all reviewers [35,36,37,38]. All included CPGs had cited SRs in their reference list. The SIGN CPG (n = 25) had the most SR citations, including 12 Cochrane SRs, followed by NICE (n = 13), which also had one Cochrane SR, NZ (n = 6), which had three Cochrane SRs, and ACRC (n = 5), which had none [35,36,37,38].
3.3. Inter-Rater Analysis
The Appraisal of Guidelines for REsearch and Evaluation (AGREE II) critical group appraisal in Table 2 shows the critical group evaluation of the four selected recommendations against each domain by Research and Evaluation (AGREE II). We calculated the percentage of agreement between raters. The results of the inter-rater reliability tests showed a high degree of agreement between the four raters for each item in each of the six domains, as well as the percent agreement of the overall assessment 1 (OA1). The calculated AGREE II domain scores are displayed in Figure 1 and Figure 2.
Figure 2 shows that the majority of the Kappa scores ranged from 0.50 to 1.00, indicating good to excellent agreement. Four assessments, presented only in Figure 2, found a low level of agreement (K = 0.0): D3Q9 of SIGN 2016, D2Q5, D3Q10 of New Zealand (NZ MOH 2016), and D3Q8 of Autism CRC 2018. The SIGN 2016 review revealed 4 questions with excellent agreement (K = 1), D1Q1, D3Q7, D3Q13, and D3Q14; 7 with very good agreement (K = 0.75), D1Q2, D1Q3, D3Q12, D4Q15, D5Q19, D5Q20 and D6Q23; 11 with good agreement (K = 0.5), D2Q4, D2Q5, D2Q6, D3Q8, D3Q10, D3Q11, D4Q16, D4Q17, D5Q18, D5Q21 and D6Q22; 1 with poor agreement (K = 0.00), D3Q9; and the overall rating (1) with good agreement (K = 0.5).
The NICE 2017 evaluation revealed 4 questions with excellent agreement out of 24, D1Q3, D3Q14, D4Q15 and D5Q19; 14 with very good agreement (K = 0.75), D1Q1, D1Q2, D2Q4, D2Q6, D3Q7, D3Q11, D3Q12, D3Q13, D4Q16, D4Q17, D5Q20, D5Q21, D6Q22, and D6Q23; 5 with good agreement (K = 0.5), D2Q5, D3Q8, D3Q9, D3Q10, and D5Q18; and the overall rating (1) with very good agreement (K = 0.75). However, the New Zealand evaluation (NZ MOH 2016) revealed only 2 questions with poor agreement; 1 question with excellent agreement (K = 1) out of 24 questions, D5Q19; no questions with fair agreement; 9 questions with good agreement (K = 0.5), D1Q2, D2Q4, D2Q6, D3Q7, D3Q11, D5Q18, D5Q21, D6Q22, and D6Q23; eleven questions with very good agreement (K = 0.75), D1Q1, D1Q3, D3Q8, D3Q9, D3Q12, D3Q13, D3Q14, D4Q15, D4Q16, D4Q17, and D5Q20; and the overall assessment (1) with very good agreement (K = 0.75).
Finally, the Australian guideline evaluation (Autism CRC 2018) revealed that 1 question out of 24 had poor agreement; nine had good agreement (K = 0.5), D3Q9, D3Q11, D3Q13, D3Q14, D4Q15, D4Q16, D5Q18, D5Q20, and D5Q21; eight had very good agreement (K = 0.75), D1Q3, D2Q5, D3Q7, D3Q10, D3Q12, D4Q17, D6Q22, and D6Q23; and the overall assessment (1) had good agreement (K = 0.5). Supplementary Table S6 shows the intraclass correlation coefficient (Kappa value) among raters for the four recommendations for overall assessment 2. The number of observed agreements was six (47.26 percent of the observations), and eight agreements were predicted by chance (65.00 percent of the observations) (Kappa = 0.637; kappa SE = 0.732; 95% confidence interval: Weighted Kappa = 0.082 for values ranging from −0.112 to 0.348).
4. Discussion
4.1. Early Identification and Diagnosis
The present study demonstrated the high quality of ASD guidelines and, at the same time, differences in terms of specificity of recommendations. All guidelines were explicit on the importance of early identification; however, different approaches were proposed. For example, NICE recommends that a local autism strategy group develop a local autism pathway for recognition and referral. On the other hand, the three other guidelines emphasized early identification of children with ASD, focusing on the basic tools and screening measures. NICE and ACRC were the only guidelines recommending using a multidisciplinary team representative of all healthcare professionals. Similarly, two CPGs advocated for delivering diagnostic assessment information to families (ACRC, SIGN). They made specific suggestions, such as providing information about the assessors, the names of the assessment tools (ACRC), and descriptions of what the tools measure (ACRC).
Three CPGs advised addressing general information on ASD with families (NICE, SIGN, NZ) and/or offering resources for families to acquire this information (NICE, SIGN, NZ). With the family’s permission, half of the CPGs recommended providing copies of the ASD medical report to the family and others, such as the referral clinician (ACRC), the child’s general practitioner or pediatrician (NICE), and other relevant professions or local health care providers (NICE, ACRC). When appropriate, the ACRC also recommended distributing the report to funding or service organizations.
4.2. Interventions and Management of ASD
Interestingly, there was limited information in the Australian guidelines regarding the intervention section of the guidelines. However, the NZ evaluation emphasized more on local issues that could be not applicable in other countries (although they can be applied to minorities anywhere in the world). The NZ evaluation addresses behavioral management and offers essential recommendations for treating and managing ASD. Only NICE and NZ mentioned the non-pharmacological intervention in detail, which is missing in the other guidelines. NICE had more details that serve many aspects of the intervention part of the guidelines.
Furthermore, the Australian evaluation does not include pharmacological intervention in the guidelines. However, NICE has more specific and detailed information regarding the use of medication, where SIGN and NZ mentioned their recommendations in a brief and general statement manner.
In terms of medication use, in the NICE and NZ guidelines, the only medication in the pharmacological part that is mentioned is the antipsychotic, and there was a clear statement of what not to use regarding the other medication. Antipsychotic use was mentioned in all guidelines (except the Australian one, which did not mention medication). There was a clear emphasis on sleep management in NICE with a brief description in SIGN and NZ. Additionally, SIGN was the only guideline that emphasized the psych-education for the family.
4.3. The transition of Care and Community Support
For those looking for guidelines related to transitions, NICE has more details guiding the care in this important step in the lives of young people with autism. SIGN elaborates on the nonpharmacological interventions, including parent-mediated interventions, communication intervention, social communication and interaction intervention, and behavioral interventions. NZ has exclusive details related to living in the community, leisure and recreation, and care when in contact with the justice system. There was very brief mention related to sensory integration intervention and occupational therapy in SIGN only. In comparison, only the Australian guidelines mentioned the professional discipline of occupational therapy.
4.4. Challenging Behaviors
There is a clear gap in crisis management guidelines and support with challenging behaviors in the majority of the guidelines. However, NZ guidelines have only a brief summary of recommendations related to crisis management and mental health and forensic services. Interestingly, nothing was mentioned related to school placement and interventions in any of the guidelines. The mention of the medications is minimal and needs more elaboration and details to guide the end-user in medication use in a straightforward matter. There was no mention of the side effects and counseling the parents. There is little mention of recommended programs for intervention, such as PECS.
4.5. At Local and International Levels
Despite the publication of several reviews of ASD CPGs [25,39,40], there are no evidence-based CPGs for ASD in Saudi Arabia; therefore, doctors are forced to make the best decisions for ASD patients based only on their clinical knowledge and experience. This review adds to the knowledge base using the AGREE II instrument to systematically assess the quality of recently published CPGs of ASD in children as part of a national CPG adaptation initiative in Saudi Arabia.
Two similar SRs of ASD CPGs were published during the undertaking of our research for the CPG project [39,40].
Using the AGREE II instrument, Wickstrom et al. evaluated the quality of 36 CPGs for ASD and 47 CPGs for intellectual disability published up until January 2019. However, only scores of individual items were reported rather than the domain standardized scores. The authors did not report the two overall assessments. Moreover, they mapped the biomedical, pharmacologic, and psychosocial interventions within the assessed CPGs [39]. Similarly, Pattison et al. assessed 17 ASD CPGs published up to April 2020. Because most of the recommendations were expert-based, with limited empirical research in this area, the authors observed variations in the recommendations for providing results and diagnoses to families and caregivers for children with ASD. They have included the four CPGs identified in our study and given them similar ratings in the six AGREE II domains, an additional strength of these four high-quality CPGs (ACRC, NICE, NZ, SIGN) [40].
4.6. Strengths and Limitations
Several strengths can be identified in our study. The authorship group of this SR and quality assessment included the aforementioned multidisciplinary clinical team with expertise in providing the different aspects of care for children with ASD in addition to a CPG methodologist. The results of this SR are enhanced by using a validated CPG appraisal instrument, the AGREE II. Moreover, the findings of our SR can inform groups who develop or adapt ASD CPGs.
There are numerous limitations to our study as well. First, only CPGs published in English or Arabic were included. High-quality CPGs and recommendations published in other languages are likely to have been overlooked in our review.
Furthermore, we did not utilize the new ‘AGREE-REX’ (Recommendation Excellence) tool, which addresses some of the shortcomings of the original AGREE II by addressing the clinical credibility of CPG recommendations [41]. Another potential limitation is the use of 70% as a cutoff point for standard domain ratings, although the original AGREE II does not mandate such a cutoff. Several cutoffs ranging from 50 to 80% have been used, but there is insufficient evidence to infer that one cutoff is superior to the other [42]. Finally, all of the four CPGs were from high-income countries.
5. Conclusions
All guidelines were of high quality; however, they differed in terms of the details of recommendations which could be attributed to the differences in healthcare settings. The findings of our study suggest that clinicians’ capacity building should include how to use the AGREE II instrument or at least the short AGREE Global Rating Scale (AGREE-GRS) instrument to support their decisions when selecting CPGs for use in everyday practice.
All authors have made substantial contributions and gave final approval of the conceptions, drafting, and final version of this manuscript. Conceptualization and methodology, Y.S.A. and A.M.E.; My AGREE PLUS Online Platform and quality assessment of guidelines and validation, Y.S.A., S.A., S.A.A., M.W.B. and S.S.A.; formal analysis, Y.S.A., S.A. and A.M.E.; resources, S.A.; data curation, Y.S.A., S.A., S.A.A., M.W.B. and S.S.A.; writing—original draft preparation, Y.S.A., S.A., M.A.d.S.P., A.M.E., F.A.B., A.H.A., A.S.A., F.A., Y.A., S.S.A., S.M.A. and M.P.; writing—review and editing, all authors; visualization, Y.S.A., S.A. and A.M.E.; supervision, Y.S.A. and S.A.; project administration, Y.S.A. and S.A.; funding acquisition, S.A. All authors have read and agreed to the published version of the manuscript.
Not applicable.
Not applicable.
The data that support the findings of this study have been made available in the tables, figures, and appendices of this article. Further details could be made available from the authors upon reasonable request to the corresponding author.
This study was supported by The National Center for Developmental and Behavioral Disorders, and The National Center for Evidence-Based Health Practice at The Saudi Health Council, Riyadh, Saudi Arabia. Moreover, The King Saud University, Deanship of Scientific Research, Research Chair for Evidence-Based Health Care and Knowledge Translation, Riyadh, Saudi Arabia. Furthermore, we would like to extend our thanks to The King Saud University Medical City for logistics and resources.
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Enlarge this image.Figure 1. Radar map of the AGREE II final standardized domain scores for eligible appraised ASD CPGs.
Enlarge this image.Figure 2. Percent agreement among raters for the four ASD clinical practice guidelines focusing on every question or item in every AGREE II domain.
Characteristics of included ASD CPGs.
| Organization, Country (Abbreviation) | Health System, Economic Classification | CPG Title | Year of Publication |
|---|---|---|---|
| Australian Autism CRC, Australia (ACRC) | National Health Insurance, High-income country | Australian National Guideline for the Assessment and Diagnosis of Autism Spectrum Disorders | 2018 |
| Ministry of Health New Zealand, New Zealand (NZ) | National Health Insurance, High-income country | New Zealand Autism Spectrum Disorder Guideline | 2016 |
| National Institute for Health and Care Excellence, United Kingdom (NICE) | National Health Service, High-income country | Autism spectrum disorder in under 19s: recognition, referral and diagnosis | 2017 |
| Scottish Intercollegiate Guidelines Network (SIGN), Healthcare Improvement Scotland, Scotland (HIS) | National Health Service, High-income country | Assessment, diagnosis and interventions for autism spectrum disorders | 2019 |
AGREE II assessment results and domain scores for the four included CPGs *.
| Source CPGs/AGREE II Domains Scores (%) | ACRC |
NZ |
NICE |
SIGN |
|---|---|---|---|---|
| Domain 1. Scope and Purpose |
99% | 93% | 96% | 97% |
| Domain 2. Stakeholder Involvement |
99% | 71% | 72% | 79% |
| Domain 3. Rigor of development |
84% | 86% | 93% | 85% |
| Domain 4. Clarity and presentation |
93% | 94% | 97% | 93% |
| Domain 5. Applicability |
92% | 54% | 89% | 85% |
| Domain 6. Editorial independence |
92% | 69% | 92% | 92% |
| Overall Assessment 1 |
92% | 88% | 96% | 92% |
| Overall Assessment 2 |
Yes—3, Yes with modifications—1, No—0 | Yes—4, Yes with modifications—0, No—0 | Yes—2, Yes with modifications—2, No—0 | Yes—2, Yes with modifications—2, No—0 |
* AGREE II (Appraisal of Guidelines for REsearch and Evaluation Version II Instrument); Australian Autism CRC (ACRC); Ministry of Health New Zealand (NZ); National Institute for Health and Care Excellence (NICE); Scottish Intercollegiate Guidelines Network, Healthcare Improvement Scotland (SIGN-HIS). This AGREE II domain scores table is a standard template with a constant set of attributes that have been reported in most similar reports of AGREE II assessment of guidelines (example: Alhasan, K.A., Al Khalifah, R., Aloufi, M. et al. AGREEing on clinical practice guidelines for idiopathic steroid-sensitive nephrotic syndrome in children. Syst Rev 10, 144 (2021).
Supplementary Materials
The following supporting information can be downloaded at:
References
1. APA—American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders; 5th ed. DSM-5 American Psychiatric Association: Arlington, VA, USA, 2013.
2. Lord, C.; Elsabbagh, M.; Baird, G.; Veenstra-Vanderweele, J. Autism spectrum disorder. Lancet; 2018; 392, pp. 508-520. [DOI: https://dx.doi.org/10.1016/S0140-6736(18)31129-2]
3. Gillberg, C.; Wing, L. Autism: Not an extremely rare disorder. Acta Psychiatr. Scand.; 1999; 99, pp. 399-406. [DOI: https://dx.doi.org/10.1111/j.1600-0447.1999.tb00984.x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/10408260]
4. APA—American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders; 3rd ed. American Psychiatric Association: Washington, DC, USA, 1980.
5. APA—American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders; 4th ed. American Psychiatric Association: Washington, DC, USA, 1994.
6. World Health Organization. Autism Spectrum Disorders 2019 [Cited 2020. July 19. Cited 2019 November]. Available online: https://www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders (accessed on 1 July 2022).
7. Maenner, M.J.; Shaw, K.A.; Bakian, A.V.; Bilder, D.A.; Durkin, M.S.; Esler, A.; Furnier, S.M.; Hallas, L.; Hall-Lande, J.; Hudson, A. et al. Prevalence and Characteristics of Autism Spectrum Disorder among Children Aged 8 Years—Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2018. MMWR Surveill. Summ.; 2021; 70, pp. 1-16. [DOI: https://dx.doi.org/10.15585/mmwr.ss7011a1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34855725]
8. Mandy, W.; Lai, M.C. Towards sex- and gender-informed autism research. Autism; 2017; 21, pp. 643-645. [DOI: https://dx.doi.org/10.1177/1362361317706904]
9. Rivet, T.T.; Matson, J.L. Review of gender differences in core symptomatology in autism spectrum disorders. Res. Autism Spectr. Disord.; 2011; 5, pp. 957-976. [DOI: https://dx.doi.org/10.1016/j.rasd.2010.12.003]
10. Luo, Y.; Eran, A.; Palmer, N.; Avillach, P.; Levy-Moonshine, A.; Szolovits, P.; Kohane, I.S. A multidimensional precision medicine approach identifies an autism subtype characterized by dyslipidemia. Nat. Med.; 2020; 26, pp. 1375-1379. [DOI: https://dx.doi.org/10.1038/s41591-020-1007-0]
11. Ferri, S.L.; Abel, T.; Brodkin, E.S. Sex Differences in Autism Spectrum Disorder: A Review. Curr. Psychiatry Rep.; 2018; 20, 9. [DOI: https://dx.doi.org/10.1007/s11920-018-0874-2]
12. Idring, S.; Lundberg, M.; Sturm, H.; Dalman, C.; Gumpert, C.; Rai, D.; Lee, B.K.; Magnusson, C. Changes in the prevalence of autism spectrum disorders in 2001–2011: Findings from the Stockholm youth cohort. J. Autism. Dev. Disord.; 2015; 45, pp. 1766-1773. [DOI: https://dx.doi.org/10.1007/s10803-014-2336-y]
13. Lai, M.C.; Baron-Cohen, S.; Buxbaum, J.D. Understanding autism in the light of sex/gender. Mol. Autism; 2015; 6, pp. 1-5. [DOI: https://dx.doi.org/10.1186/s13229-015-0021-4]
14. Daniels, A.M.; Halladay, A.K.; Shih, A.; Elder, L.M.; Dawson, G. Approaches to enhancing the early detection of autism spectrum disorders: A systematic review of the literature. J. Am. Acad. Child Adolesc. Psychiatry; 2014; 53, pp. 141-152. [DOI: https://dx.doi.org/10.1016/j.jaac.2013.11.002]
15. Hill, T.L.; White, T.C.; Anthony, B.J.; Reaven, J.; Harris, B.; Reyes, N.; Anthony, L.G. Disparities in autism spectrum disorder diagnoses among 8-year-old children in Colorado: Who are we missing?. Autism; 2021; 25, pp. 102-113. [DOI: https://dx.doi.org/10.1177/1362361320950058] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32859134]
16. Graham, R.; Mancher, M.; Miller Wolman, D.; Greenfield, S.; Steinberg, E. Clinical Practice Guidelines We Can Trust; National Academies Press (US): Washington, DC, USA, 2011; Available online: https://www.ncbi.nlm.nih.gov/books/NBK209539/ (accessed on 3 July 2022).
17. Hayes, J.; Ford, T.; Rafeeque, H.; Russell, G. Clinical practice guidelines for diagnosis of autism spectrum disorder in adults and children in the UK: A narrative review. BMC Psychiatry; 2018; 18, 222. [DOI: https://dx.doi.org/10.1186/s12888-018-1800-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30005612]
18. Brouwers, M.C.; Kho, M.E.; Browman, G.P.; Burgers, J.S.; Cluzeau, F.; Feder, G.; Fervers, B.; Graham, I.D.; Grimshaw, J.; Hanna, S.E. et al. AGREE II: Advancing guideline development, reporting and evaluation in health care. Cmaj Can. Med. Assoc. J.; 2010; 182, pp. E839-E842. [DOI: https://dx.doi.org/10.1503/cmaj.090449]
19. Brouwers, M.C.; Spithoff, K.; Lavis, J.; Kho, M.E.; Makarski, J.; Florez, I.D. What to do with all the AGREEs? The AGREE portfolio of tools to support the guideline enterprise [published online ahead of print, May 29 2020]. J. Clin. Epidemiol.; 2020; 125, pp. 191-197. [DOI: https://dx.doi.org/10.1016/j.jclinepi.2020.05.025] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32473992]
20. Siering, U.; Eikermann, M.; Hausner, E.; Hoffmann-Eßer, W.; Neugebauer, E.A. Appraisal Tools for Clinical Practice Guidelines: A systematic review. PLoS ONE; 2013; 8, e82915. [DOI: https://dx.doi.org/10.1371/journal.pone.0082915] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24349397]
21. Amer, Y.S.; Wahabi, H.A.; Abou Elkheir, M.M.; Bawazeer, G.A.; Iqbal, S.M.; Titi, M.A.; Ekhzaimy, A.; Alswat, K.A.; Alzeidan, R.A.; Al-Ansary, L.A. Adapting evidence-based clinical practice guidelines at university teaching hospitals: A model for the eastern Mediterranean Region. J. Eval. Clin. Pract.; 2019; 25, pp. 550-560. [DOI: https://dx.doi.org/10.1111/jep.12927]
22. Amer, Y.S.; Elzalabany, M.M.; Omar, T.I.; Ibrahim, A.G.; Dowidar, N.L. The ‘Adapted ADAPTE’: An approach to improve utilization of the ADAPTE guideline adaptation resource toolkit in the Alexandria Center for Evidence-Based Clinical Practice Guidelines. J. Eval. Clin. Pract.; 2015; 21, pp. 1095-1106. [DOI: https://dx.doi.org/10.1111/jep.12479]
23. Johnston, A.; Kelly, S.; Hsieh, S.C.; Skidmore, B.; Wells, G.A. Systematic reviews of clinical practice guidelines: A methodological guide. J. Clin. Epidemiol.; 2019; 108, pp. 64-76. [DOI: https://dx.doi.org/10.1016/j.jclinepi.2018.11.030]
24. Alenezi, S.; Bashiri, F.A.; Alawami, A.H.; Alhazmi, A.S.; Aladamawai, S.A.; Alnemary, F.; Alqahtani, Y.; Buraik, M.W.; AlSuwailem, S.S.; Akhalifah, S.M. et al. Quality Assessment of Clinical Practice Guidelines for the Management of Autism Spectrum Disorders (ASD) in Children: A Protocol for a Systematic Review and Assessment Using the AGREE II Instrument [Internet]. OSF. 2021; Available online: https://osf.io/n39xz/ (accessed on 10 May 2022).
25. Penner, M.; Anagnostou, E.; Andoni, L.Y.; Ungar, W.J. Systematic review of clinical guidance documents for autism spectrum disorder diagnostic assessment in select regions. Autism; 2018; 22, pp. 517-527. [DOI: https://dx.doi.org/10.1177/1362361316685879]
26. Wang, Z.; Norris, S.; Bero, L. The advantages and limitations of guideline adaptation frameworks. Implement Sci.; 2018; 13, 72. [DOI: https://dx.doi.org/10.1186/s13012-018-0763-4]
27. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Moher, D. Updating guidance for reporting systematic reviews: Development of the PRISMA 2020 statement. J. Clin. Epidemiol.; 2021; 134, pp. 103-112. [DOI: https://dx.doi.org/10.1016/j.jclinepi.2021.02.003]
28. Hasim, N.; Bakar, M.A.; Islam, M.A. Efficacy and Safety of Isotonic and Hypotonic Intravenous Maintenance Fluids in Hospitalised Children: A Systematic Review and Meta-Analysis of Randomised Controlled Trials. Children; 2021; 8, 785. [DOI: https://dx.doi.org/10.3390/children8090785]
29. Login—AGREE Enterprise Website. Agreetrust.org. Available online: https://www.agreetrust.org/my-agree/ (accessed on 27 December 2021).
30. Altokhais, T.I.; Al-Obaid, O.A.; Kattan, A.E.; Amer, Y.S. CPG Collaborative Groups. Assessment of implementability of an adapted clinical practice guideline for surgical antimicrobial prophylaxis at a tertiary care university hospital. J. Eval. Clin. Pract.; 2017; 23, pp. 156-164. [DOI: https://dx.doi.org/10.1111/jep.12658]
31. Alhasan, K.A.; Al Khalifah, R.; Aloufi, M.; Almaiman, W.; Hamad, M.; Abdulmajeed, N.; Al Salloum, A.; Kari, J.A.; AlJelaify, M.; Bassrawi, R.K. et al. AGREEing on clinical practice guidelines for idiopathic steroid-sensitive nephrotic syndrome in children. Syst. Rev.; 2021; 10, 144. [DOI: https://dx.doi.org/10.1186/s13643-021-01666-w]
32. Cohen, J. A Coefficient of Agreement for Nominal Scales. Educ. Psychol. Measur.; 1960; 20, pp. 37-46. [DOI: https://dx.doi.org/10.1177/001316446002000104]
33. Fleiss, J.L.; Shrout, P.E. Approximate interval estimation for a certain intraclass correlation coefficient. Psychometrika; 1978; 43, pp. 259-262. [DOI: https://dx.doi.org/10.1007/BF02293867]
34. Bland, J.M.; Altman, D.G. Statistics Notes: Measurement error and correlation coefficients. BMJ; 1996; 313, pp. 41-42. [DOI: https://dx.doi.org/10.1136/bmj.313.7048.41]
35. Autism CRC The University of Queensland. Australian National Guideline for the Assessment and Diagnosis of Autism Spectrum Disorders|Autism CRC. 2018; Available online: https://www.autismcrc.com.au/access/national-guideline (accessed on 27 January 2021).
36. Ministry of Health New Zealand (NZ). New Zealand Autism Spectrum Disorder Guideline. 2016; Available online: https://www.health.govt.nz/publication/new-zealand-autism-spectrum-disorder-guideline (accessed on 27 January 2021).
37. National Institute for Health and Care Excellence (NICE). Autism Spectrum Disorder in under 19s: Recognition, Referral and Diagnosis | Guidance | NICE. 2017; Available online: https://www.nice.org.uk/guidance/cg128 (accessed on 27 January 2021).
38. Scottish Intercollegiate Guidelines Network (SIGN) Healthcare Improvement Scotland (HIS). Assessment, Diagnosis and Interventions for Autism Spectrum Disorders. 2016; Available online: https://www.sign.ac.uk/our-guidelines/assessment-diagnosis-and-interventions-for-autism-spectrum-disorders/ (accessed on 27 January 2021).
39. Wickstrom, J.; Dell’Armo, K.; Salzman, E.; Hooker, J.L.; Delehanty, A.; Bishop, S.; Tassé, M.J.; Wetherby, A.M.; Piergies, A.M.; Damiano, D. et al. Systematic Review: Recommendations for Rehabilitation in ASD and ID from Clinical Practice Guidelines. Arch. Rehabil. Res. Clin. Transl.; 2021; 3, 100140. [DOI: https://dx.doi.org/10.1016/j.arrct.2021.100140]
40. Pattison, E.; Ure, A.; Mittiga, S.R.; Williams, K.; Freeman, N.C. The Feedback Session of an Autism Assessment: A Scoping Review of Clinical Practice Guideline Recommendations. J. Autism. Dev. Disord.; 2021; 52, pp. 1-20. [DOI: https://dx.doi.org/10.1007/s10803-021-05067-9]
41. Brouwers, M.C.; Spithoff, K.; Kerkvliet, K.; Alonso-Coello, P.; Burgers, J.; Cluzeau, F.; Férvers, B.; Graham, I.; Grimshaw, J.; Hanna, S. et al. Development and validation of a tool to assess the quality of clinical practice guideline recommendations. JAMA Netw. Open; 2020; 3, e205535. [DOI: https://dx.doi.org/10.1001/jamanetworkopen.2020.5535]
42. Hoffmann-Eßer, W.; Siering, U.; Neugebauer, E.A.; Lampert, U.; Eikermann, M. Systematic review of current guideline appraisals performed with the Appraisal of Guidelines for Research & Evaluation II instrument—A third of AGREE II users apply a cut-off for guideline quality. J. Clin. Epidemiol.; 2018; 95, pp. 120-127. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29288133]
43. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E. et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ; 2021; 372, n71. [DOI: https://dx.doi.org/10.1136/bmj.n71] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33782057]
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Alenezi, Shuliweeh 2
; Bashiri, Fahad A 3
; Amel Hussain Alawami 4 ; Alhazmi, Ayman Shawqi 5 ; Aladamawi, Somayyah A 6 ; Alnemary, Faisal 7 ; Alqahtani, Yasser 8 ; Buraik, Maysaa W 4 ; AlSuwailem, Saleh S 7 ; Akhalifah, Shahad M 7 ; Marcela Augusta de Souza Pinhel 9 ; Penner, Melanie 10 ; Elmalky, Ahmed M 11 1 Pediatrics Department, King Khalid University Hospital, King Saud University Medical City, Riyadh 11451, Saudi Arabia; Clinical Practice Guidelines and Quality Research Unit, Quality Management Department, King Saud University Medical City, Riyadh 11451, Saudi Arabia; Research Chair for Evidence-Based Health Care and Knowledge Translation, Deanship of Scientific Research, King Saud University, Riyadh 11451, Saudi Arabia; Alexandria Center for Evidence-Based Clinical Practice Guidelines, Alexandria University, Alexandria 5424041, Egypt; Guidelines International Network, Perth PH16 5BU, Scotland, UK
2 Department of Psychiatry, College of Medicine, King Saud University, Riyadh 11451, Saudi Arabia
3 Pediatric Neurology Division, Pediatrics Department, College of Medicine, King Saud University, Riyadh 11451, Saudi Arabia;
4 Johns Hopkins Aramco Healthcare, Dhahran 34465, Saudi Arabia;
5 Developmental Pediatric Department, Children’s Hospital, King Saud Medical City, Ministry of Health, Riyadh 12746, Saudi Arabia;
6 King Abdullah bin Abdulaziz University Hospital, Riyadh 11564, Saudi Arabia;
7 Autism Center of Excellence, Riyadh 11564, Saudi Arabia;
8 King Fahad Armed Forces Hospital, Jeddah 23311, Saudi Arabia;
9 Department of Health Science, Ribeirao Preto Medical School, University of São Paulo, Ribeirao Preto 14049-900, Brazil;
10 Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON M4G 1R8, Canada;
11 Morbidity and Mortality Unit, King Saud University Medical City, King Saud University, Riyadh 11451, Saudi Arabia;