Evaluating the Validity and Reliability of PDQ-II and Comparison with DDST-II for Two Step Developmental Screening

Headnote

Abstract

Objective: This research was designed to identify the validity and reliability of the Prescreening Developmental Questionnaire 2 (PDQ-II) in Tehran in comparison with the Denver Developmental Screening Test-II (DDST-II).

Methods: After translation and back translation, the final Persian version of test was verified by three pediatricians and also by reviewing relevant literature for content validity. The test was performed on 237 children ranging from 0 to 6 years old, recruited by convenient sampling, from four health care clinics in Tehran city. They were also evaluated by DDST II simultaneously. Interrater methods and Cronbach's α were used to determine reliability of the test. The Kappa agreement coefficient between PDQ and DDST II was determined. The data was analyzed by SPSS software.

Findings: All of the questions in PDQ had satisfactory content validity. The total Cronbach's α coefficient of 0-9 months, 9-24 months, 2-4 years and 4-6 years questionnaires were 0.951, 0.926, 0.950 and 0.876, respectively. The Kappa measure of agreement for interrater tests was 0.89. The estimated agreement coefficient between PDQ and DDST II was 0.383. Based on two different categorizing possibilities for questionable scores, that is, "Delayed" or "Normal", sensitivity and specificity of PDQ was determined to be 35.7-63% and 75.8-92.2%, respectively.

Conclusion: PDQ has a good content validity and reliability and moderate sensitivity and specificity in comparison with the DDST-II, but by considering their relatively weak agreement coefficient, using it along with DDST-II for a two-stage developmental screening process, remains doubtful.

Iranian Journal of Pediatrics, Volume 21(Number 3), September 2011, Pages: 343-349

Key Words: Development; Developmental Delay; Screening; Questionnaire; Validity and Reliability

Introduction

Early detection of developmental disorders has an important role in the well-being of children and their families. This is a basic responsibility for pediatric primary care providers [1]. About 16-18% of children in various populations have developmental disorders but only 20-30% of them are recognized before school entrance.

This fact shows that early detection of developmental disorders must be provided in primary child health care services. Today there is an increasing effort for detection of developmental disorders at an earlier age because intervention services are cost effective and when provided in early childhood, have greater efficacy [2]. These services improve the developmental prognosis and have short and long term benefits [3-6]. In order to detect developmental disorders at an early age, the American Academy of Pediatrics (AAP) has recommended that pediatricians use developmental screening tools at 9, 18, 24 (or 30) months child health visits [1].

Developmental screening means using a brief, valid, rapid and standardized tool in order to detect those children who are at risk for developmental disorders and to help find those children who need an extensive developmental assessment [1,7-9].

Although there are numerous developmental screening tools, each uses different approaches. There is no unique tool that can be used for all populations and all age groups[1]. For selecting a suitable tool we must consider the age range of the desired population, the time and expenses of training test providers and of administering and interpreting the test, the developmental domains that must be assessed in the desired population, the validity, reliability, sensitivity, specificity and all other positive and negative points about the tool of the test[1,10-12]. There are two types of developmental screening tools: 1) screening tests which examiners directly observe and interpret the child`s behavior and 2) developmental questionnaires that are completed by the child`s care provider.

Parental information about child developmental status has been considered reliable and useful for many years [13-15]. Research shows that if questionnaires are presented correctly to the parents, almost all parents regardless of their socioeconomic status, geographic setting or health background and child rearing experiences, can give correct information about their child and their opinions have high validity and will lead to increasing rates of early detection and intervention of developmental disorders [10,12-18]. Furthermore, if parents have a role in the developmental screening process, they can be more effective in the detection of learning needs of their children [19]. One must keep in mind that screening tools are usually used for two step diagnosis and their suspicious or delayed results must be confirmed with diagnostic or more specialized tools that need more time and expertise to administer [7]. Research has shown that using Prescreening Developmental Questionnaire 2 (PDQ-II) decreases the use of Denver Developmental Screening Test-II (DDST-II) that needs more time, expense and expertise to administer, by 69% [20].

This study was performed from February 2008 to January 2009 in Tehran, Iran to determine validity and reliability of PDQ and to estimate its sensitivity and specificity in comparison to its origin, the DDST-II.

Subjects and Methods

At first the questionnaires were translated into Persian by two experts in English language. Then the Persian version was back translated to English by two other English experts who were unfamiliar with the original version of the test. After comparing the back-translated versions of the test with the original version, the problematic parts were identified and corrections were made in the Persian translation, after which the Persian version was finalized. Next the content validity of the final Persian version was verified by three pediatricians familiar with child development and also by reviewing relevant sources and references.

PDQ-II is a developmental prescreening tool that is derived from DDST-II. 97 of 105 items of DDST-II are changed to questions that can be answered by "YES" or "NO" by the care-giver.

They are categorized into 4 questionnaires for 0-9 month, 9-24 month, 2-4 year and 4-6 year old children. The 75th and 95th percentiles for each question and also the developmental domain to which it belongs are shown in front of it. Caregivers must continue answering questions until they arrive at 3 "NO" answers (it is not necessary that the NO answers be consecutive). The answer to each question can be: normal (which means the child is able to do the task), delayed (which means the child is not able to do the task that 90% of his/her age-matched children can do) and caution (which means the child is not able to do the task that 75% of his/her age matched children can do). For interpretation of the results, if the child has ≤1 delay or ≤2 cautions (considered as suspicious), developmental advices are given to parents and the child must return for retesting by the PDQ-II one month later. If the child is still in a 'suspicious' condition in the second visit then he/she should be referred for screening by the DDST-II. If in the first prescreening visit child has ≥2 delays or ≥3 cautions (considered as delayed), he/she should be screened by the DDST-II as soon as possible.

For performing the study, 8 examiners (with a B.S. degree in occupational therapy or clinical psychology) were trained. Convenient sampling was used and 237 children aged 0-6 years (divided to 4 age groups], without any obvious disability were tested in 4 primary health care centers situated in south, north, east and western regions of Tehran (sample size was determined by correlation coefficient formula, with 90% power and 95% confidence interval and was calculated to be 48 children in each age group). For those who were younger than 24 months and were born prematurely, we calculated and considered the corrected age. The study was approved by the research committee and thereafter by the ethical committee of the University of Social Welfare and Rehabilitation Sciences. Parents were informed about the importance of developmental screening and how the test was performed. Then their written consent was acquired. For all children, at first, age-related questionnaires were completed by caregivers and then examiners performed the DDST-II. Because we did not have accessibility to any diagnostic developmental gold standard test and also because PDQ-II is a prescreening test, we used the DDST-II as gold standard test, so that the results of the PDQ-II were cross tabulated against DDST-II to obtain its sensitivity and specificity in comparison to DDST-II and also to determine their Kappa agreement coefficient. By considering this limitation, we used co-positivity of the two tests as a substitute for sensitivity and their co-negativity, for specificity. In 10% of children in order to determine the interrater reliability of the PDQ, after parents completed the questionnaires and the DDST-II was performed, the examiners asked the parents the PDQ questions again and completed the related forms. Also, Cronbach's α was used to determine reliability of the test. Results were interpreted by researchers. Data were coded and analyzed by SPSS software. For ethical purposes, all parents whose children were detected as delayed or suspicious were referred for additional evaluations and interventions.

Findings

In the present study 237 children consisting of 108 (46%) girls and 129 (54%) boys aged 0-6 years were screened by PDQ and DDST. 96% of cases were born term. Maternal education of 84% of children was at high school level or higher. In prescreening by PDQ the 'normal', 'delayed' and 'suspect' cases were 62%, 18% and 20% respectively. In screening by DDST-II 64.5% of children (68% of girls and 62% of boys) were normal and 35.5% of them were detected as delayed. The results of the two tests are shown in Table 1.

Using the PDQ-II, the number of children falling in the categories of "caution" and "delayed" was higher in the gross motor and language domains. Whereas with DDST-II, "delays" and "cautions" were higher in the language domain and "delays" alone, were higher in fine and gross motor domains.

For determining the measure of agreement between PDQ and DDST results, considering the fact that for cross tabulating two tests, they must have similar number of answer choices, we first considered the "suspect cases" of the PDQ as "normal" and the next time, as "delayed". When suspect cases were considered as delayed, the kappa measure of agreement was 0.383 (P<0.001) and when considered as normal, it was 0.314 (P<0.001).

The mean sensitivity (co-positivity) and mean specificity (co-negativity) of PDQ in comparison to DDST were 35.7% and 75.8% respectively. If suspect cases were considered as normal, the sensitivity and specificity of PDQ were 35.7% and 92.2% respectively, and they changed to 63% and 75.8% when suspect cases were considered as delayed.

Cronbach's α was determined for all developmental domains in each questionnaire. As observed in Table 2, it was above 0.5 in all domains and in all age groups, except the personal-social domains at 4-6 years of age. Kappa measure of agreement for interrater reliability of the test was 0.89 (P<0.001).

Discussion

In our study the content validity of PDQ was verified and there was no need to change any of the questions in the questionnaires.

A study was performed in Saudi Arabia on 1219 children. In order to use PDQ as a prescreening tool for their population, researchers made some changes in the contents of the questionnaires, age intervals, normal age consideration for acquisition of developmental skills and also their order of emerging. These changes were greater in the language domain and showed the importance of adapting developmental screening tools before using them in other countries [13]. Another study in India showed that because of cultural differences between USA and India, PDQ was not suitable for that country [21].

We evaluated the reliability of the test by the Cronbach's α determination. It was "very good" for the 0-9, 9-24 months and 2-4 years questionnaires and "good" for the 4-6 years questionnaire. The interrater method was also used as another way for reliability determination and it was 0.89 which was also "very good". This study showed that the reliability of PDQ was "very good" on the whole.

In 1987 Frankenburg et al showed that the reliability of PDQ in test-retest and interrater testing (parents-teacher) were 94.1% and 83% respectively [22].

By considering suspect cases as normal, the sensitivity and specificity of the PDQ compared to the DDST-II were 35.7% and 92.2%, respectively and the referral rate (that is, the delayed cases on the PDQ who were referred for performing the DDST-II) was 12.7% and the false negative rate was high. However, if suspect cases were considered as delayed, sensitivity and specificity rose to 63% and 75.8%, respectively and the referral rate rose to 22.7%. The latter values for sensitivity and specificity are closer to the acceptable range for developmental screening tools and the referral rate is more similar to the value presented by the authors of the original test. Also because in this form, the agreement coefficient between PDQ and DDST-II is better than the previous case, it seems that for having more valid results, suspect cases must be considered as delayed. Also it is suggested that existence of .1 delays or .2 cautions be considered as referral criteria (instead of .2 delays or .3 cautions).

In India a similar study was performed using the 2-4 year-old questionnaires of PDQ and showed that if .1 delays were considered as abnormal, sensitivity, specificity and referral rates were 100%, 7.8% and 92.6%, respectively and thus two stage screening would not be meaningful. When they considered .2 delays as abnormal, sensitivity, specificity and referral rates changed to 18.2%, 42.6% and 53.9%. In this situation, sensitivity and specificity of the test were not acceptable for developmental screening. They concluded that PDQ was not suitable for use in India [21].

The present study showed that regardless of considering suspect cases as normal or delayed, the agreement coefficients of PDQ and DDST-II were weak and the 2-4 year- old questionnaire had greatest agreement with DDST-II.

Berges et al conducted a research in order to determine the agreement coefficient of PDQ, its modified version (M-PDQ) and another questionnaire named Alpern-Boil Developmental Profile-II, with DDST. They concluded that all tools had good agreement with DDST and could be used for rapid developmental screening in preschool children [23]. Another study in India showed that the 2-4 year-old questionnaire of PDQ had no good relationship with DDST [21]. In another study, term and Very low birth weight infants were screened by PDQ and the Grifffiths developmental scale at 12 months of age. The study showed that these two developmental screening tools had good agreement and the PDQ was a reliable tool for infant developmental screening [24].

There was no relationship between the presence of developmental delays and such factors as sex, place of residence and maternal education in the present study.

A study in an urban area of India showed that uneducated mothers had not good information about the developmental abilities of their children such as ability to draw a circle or a straight line. Thus due to the low educational level of mothers in that area, PDQ could not evaluate the developmental status of children correctly [21]. A study was done using the PDQ in urban and rural areas of Japan with different climates. It showed that in summer the results of children developmental screening in rural areas was lower than jn urban regions but in winter, there was no difference between them.

Researchers concluded that in summer, mothers in rural areas did not have much contact with their children because of farm work, and thus were not aware of their children's developmental status. They concluded that for using the PDQ and similar questionnaires such factors need to be considered too [25].

In the present study developmental prescreening with PDQ showed that the number of cautions and delays were greater in the gross motor and language domains whereas by using the DDST-II the total number of cautions and delays was greater in the language domain and the number of delays alone, was more in the fine and gross motor areas. A study in Shiraz, Iran showed that by using DDST-II, the developmental status of 3-6 year old children in fine and gross motor domains was lower than the Denver normative sample [26]. Other studies in England, Japan, Sweden and Israel showed that children in those countries had a slower rate of fine and gross motor development [21,25,27-30]. In Cardiff, children were better in some personal-social items on the whole, and children younger than 18 months were better in the language domain [27]. Also children of Tokyo were faster in some personal-social items [28]. In another study, developmental screening of children in Tehran showed that by using DDST-II and ASQ (Ages & Stages Questionnaires], developmental delays were greater in the gross and fine motor areas[31].

The present study showed that there is a relatively weak correlation between results of PDQ and DDST-II in children of Tehran. Thus using PDQ and DDST-II in order to perform two-step developmental screening is doubtful. Other studies have previously shown that questionnaires completed by parents, may not have good agreement with each other[32]. The important fact is that the results of screening tools should not be interpreted alone, but decision should be made by considering the child's total function and environmental factors. Furthermore, develop mental screening tools are most useful when they are repeated periodically[9,33,34].

Different studies in Iran have shown the rather considerable prevalence of developmental disorders in Iranian children [4,26,31,35,36]. Thus decision makers of child health care should consider suitable strategies for control of risk factors, emphasizing on prevention and early detection of developmental disorders and providing early intervention services in the country.

This study had some limitations such as limitation of time and resources for re-evaluating those children who were detected as cautious or delayed as well as absence of a gold standard developmental diagnostic test for determining the concurrent validity of the results of prescreening and screening with the PDQ-II and DDST-II, respectively.

Conclusion

This study showed that all questionnaires of PDQ have a good content validity and reliability and moderate sensitivity and specificity and there was no need to change any of the questions. We found that in our study there was a relatively weak agreement coefficient between PDQ and DDST-II, thus using PDQ along with DDST-II for a two-stage developmental screening process, remains doubtful. We recommended that other researchers confirm this finding with further studies.

Acknowledgment

This research was supported by a grant from University of Social Welfare and Rehabilitation sciences.

Conflict of Interest: None.

References

References

1. Council on Children With Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; Medical Home Initiatives for Children With Special Needs Project Advisory Committee. Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics 118(1):405-20.

2. Rydz D, Srour M, Oskoui M, et al. Screening for developmental delay in setting of a community pediatric clinic: A prospective assessment of Parent-Report Questionnaires. Pediatrics 2006; 118(4): e1178-86.

3. Mayson TA, Harris SR, Bachman CL. Gross Motor Development of Asian and European Children on Four Motor Assessments: A Literature Review. Pediatr Phys Ther 2007; 19(2):148-53.

4. Vameghi R, Hatamizadeh N, Sajedi F, et al. Production of a native developmental screening test: the Iranian experience. Child Care Health Development 2010;36(3):340-5.

5. Levine DA. Guiding parents through behavioral issues affecting their child's health: the primary care provider's role. Ethnicity Dis 2006;16(2 Suppl 3):S3-23.

6. Wagner J, Jenkins B, Smith JC. Nurses' utilization of parent questionnaires for developmental screening: Developmental screening tools. Pediatr Nurs 2006;32(5):409-12.

7. Glascoe FP. Screening for developmental and behavioral problems. Ment Retard Developm Disab Res Rev 2005;11(3):173-9.

8. Galascoe FP. Developmental Screening and Surveillance. In: Kliegman RM, Behrman RE, Jenson HB, Stanton BM. Nelson Textbook of Pediatrics. 18th ed. Philadelphia: Saunders. 2008; Pp: 74-81.

9. American Academy of Pediatrics - Committee on Children with Disabilities, Developmental Surveillance and Screening of Infants and Young Children. Pediatrics 2001;108(1):192-5.

10. Glascoe FP, Shapiro HL. Introduction to Developmental and Behavioral Screening. http://www.dbpeds.org/articles/detail.cfm?id=5 Access date: Access date: Jan 13, 2010.

11. Mayson TA, Harris SR, Bachman CL. Gross motor development of Asian and European children on four motor assessments: A literature review. Pediatr Phys Ther 2007;19(2):148-53.

12. Ahsan S, Murphy G, Kealy S, Sharif F. Current developmental surveillance: is it time for change? Ir Med J 2008;101(4):110-2.

13. Al-Ansari SS, Bella H. Translation and adaptation of the revised Denver pre-screening developmental questionnaire for Madinah children, Saudi Arabia. Ann Saudi Med 1998; 18(1):42-8.

14. Sonnander K. Parental developmental assessment of 18-month-old children: reliability and predictive value. Dev Med Child Neurol 1987; 29(3):351-62.

15. Knobloch H, Stevens F, Malone A, et al. The validity of parental reporting of infant development. Pediatrics 1979;63(6):872-8.

16. Glascoe FP. Early detection of developmental and behavioral problems. Pediatrics in Review 2000; 21(8):272-80.

17. Glascoe FP. The value of parents' concerns to detect and address developmental and behavioural problems. J Pediatr Child Health 1999;35(1):1-8.

18. Williams J, Holmes CA. Improving the early detection of children with subtle developmental problems. J Child Health Care 2004;8(1):34-46.

19. Kwan C, Nam SS. Utilizing parental observations and computer technology in developing a childscreening instrument in Singapore. Int J Early Years Edu 2004;12(2):117-29.

20. Frankenburg WK, Doorninck WJ, Liddell BA, Dick NP. The Denver pre-screening developmental questionnaire (PDQ). Pediatrics 1976;57(5):744- 53.

21. Awasthi S, Pande VK. Validation of revised prescreening Denver questionnaire in preschool children of urban slums. Indian Pediatr 1997; 34(10): 919-23.

22. Frankenburg WK, Fandal AW, Thornton SM. Revision of Denver prescreening developmental questionnaire. Pediatrics 1987;110(4):653-7.

23. Burgess DB, Asher KN, Doucet, et al. Parent report as a means of administering the prescreening developmental questionnaire: an evaluation study. J Develop Behav Pediatr 1984; 5(4):195-200.

24. Janson H, Squires J. Parent-completed developmental screening in a Norwegian population sample: a comparison with US normative data. Acta Paediatr 2004;93(11): 1525-9.

25. Azuma N. A cross-cultural study of the Denver prescreening developmental questionnaire, Rep Fac Educ Iwate Univ 1992;52(1):187-94.

26. Pasand F. Standardization and validity and reliability determination of DDST-II for fine and gross motor function of 3-6 year old children in Shiraz city, thesis for MS. Tehran, Tarbiat Modaress University, 2008. [in Persian]

27. Bryant GM, Davies KJ, Newcombe RG. Standardisation of the Denver Developmental Screening Test for Cardiff children. J Develop Med Child Neurol 1976;21(3):353-64.

28. Oeda RA. Standardization of The Denver Developmental Screening Test on Tokyo children. J Develop Med Child Neurol 1974;20(5):647-56.

29. Lundberg A. Gross and fine motor performance in healthy Swedish children aged fifteen and eighteen months. Neuropediatrics 1979;10(1): 35-50.

30. Shapira Y, Harles S. Standardization of the Denver Developmental Screening Test for Israeli children. Israel Med Sci 1983;19(3):246-51.

31. Shahshahani S, Vameghi R, Azari N, et al. Validity and reliability determination of Denver developmental screening test-II in 0-6 year-olds in Tehran. Iran J Pediatr 2010;20(3):53-8.

32. Sices L, Stancin T, Kirchner L, Bauchner H. PEDS and ASQ Developmental Screening Tests may not identify the same children. Pediatrics 2009;124 (4):e640-7.

33. Dworkin PH. Anderson Aldrich Award Lecture: Enhancing developmental services in child health supervision - an idea whose time has truly arrived. Pediatrics 2004;114(3):827-31.

34. Frankenburg WK. Developmental surveillance and screening of infants and young children. Pediatrics 2002;109(1):144-5.

35. Sajedi F, Vameghi R, Mohseni Bandpei MA, et al. Motor developmental delay in 7500 Iranian infants: Prevalence and risk factors. Iran J Child Neurol 2009;3(3):43-50.

36. Soleimani F, Dadkhah A. Validity and reliability of Infant Neurological International Battery for detection of gross motor developmental delay in Iran. Child Care Health Dev J 2007;33(3):262-5.