Illustration by Elizabeth Lada.
------------
In a recent report for the Abell Foundation, Kalman R. Hettleman documented the troubled history of special education in the Baltimore public school system. He attributed the failure to "the compliance maze" that special education teachers and administrators face, which consists of "ever-proliferating procedures, forms to be filled out, micro-managed administrative functions, reports and audits "all of which far exceed reason and necessity and do not measure the quality of instruction or other services." Hettleman calculated that the school district spends $28 million annually just to document its compliance with the law. Teachers and administrators confided to him that his estimates were, if anything, conservative. Hettleman found that talented teachers are discouraged from entering or remaining in special education because the job essentially requires "a Ph.D. in paperwork."
At the core of the problems in Baltimore and across the nation lies special education's reliance on procedural rules, exhaustive documentation, and legal threats as the only means of holding schools and educators accountable. In the current system, schools that get the process right -- that screen children for disabilities in a timely manner; that follow the law in designing an education plan for each disabled child; and that fill out the paperwork documenting that children have received the planned services -- have fulfilled their obli-gations under the law. This is what I call process-based accountability. Schools only need to show that they have complied with the complex regulations and court decisions that govern special education in order to shield themselves from adverse legal rulings -- the only real consequence of failure in special education. What they need not show is that disabled students are making gains in achievement.
To focus schools on achievement requires shifting from a process-based accountability system to one driven by results. The President's Commission on Excellence in Special Education has embraced such a shift. As the commission wrote in its report released in July 2002, special education "will only fulfill its intended purpose if it . . . becomes results-oriented "not driven by process, litigation, regulation, and confrontation." Still, as attractive as a results-based focus in special education is, shifting to this approach will be no simple task. Just as it took a quarter of a century to create the universal special education system we now have, so will it take years to find ways to realize its full potential.
Acronym Soup
I first began analyzing the accountability system for special education two years ago, through a fruitful collaboration with Bryan C. Hassel of Public Impact. We encountered a program that:
* intends to be responsive to disabled children and their families but is often paralyzed by red tape;
* attempts to address the needs of an amazingly diverse group of children yet often relies on standardized approaches and "box checking" oversight;
* absorbs more than $50 billion a year in public funds yet provides no consistent tracking of its performance.
The more than 6 million students who currently receive special education services are a varied lot. A small share, about 10 percent, suffer from sensory disabilities such as hearing impairments or physical and neurological disabilities such as mobility impairments and autism. The remaining 90 percent have been diagnosed with developmental disabilities such as emotional disturbance and specific learning disabilities, the most common of which are Attention Deficit Disorder (ADD) and Attention Deficit Hyperactivity Disorder (ADHD). Some disabilities, such as severe autism, can profoundly limit the academic achievement of students. Other disabilities, such as mild ADD, have such subtle effects on learning that, until recently, they were rarely diagnosed and treated. Students with specific learning disabilities (LD) now compose the largest of the 13 subcategories of special education "more than 46 percent of all special education students (see Figure 1). Since LD students have a disability that is educationally based and often mild in severity, they are especially well positioned to realize benefits from results-based accountability.
The system of accountability within special education currently rests on disabled students' having a legally enforceable right to a "free and appropriate education" (FAPE) in the "least restrictive environment" (LRE). How a school plans to meet the twin goals of FAPE and LRE must be spelled out in each disabled student's individualized education plan (IEP). These are the most important, but by far not the only, mandates of the Individuals with Disabilities Education Act (IDEA), the federal statute that, under various names, has guided special education policy since 1975.
If proper procedures are followed, the IEP is the result of recommendations from a team of specialists and teachers and discussions with a student's parents. Parents who disagree with the slate of services a school plans to give their child may protest. If school officials refuse to offer the services requested, parents can demand a due-process hearing or force a move to arbitration. If they disagree with the judgment of the hearing officer or arbiter, they can go to the courts. The threat of litigation provides a strong incentive to either meet parental demands or compile a record of evidence that will persuade court officials that those demands are unreasonable or undesirable.
IDEA guarantees parents the right to a due- process hearing to contest "any matter relating to the identification, evaluation, or educational placement of the child." The hearings are lawyer-dominated, and parents are reimbursed for their legal expenses if their complaint is affirmed. These guarantees operate on the entire special education system, leading parents to "lawyer up" at IEP meetings, parent-teacher conferences, and other venues where collaboration would be more constructive than confrontation. Education officials respond in kind. The constant threat of a negative court ruling discourages special educators from attempting interventions that are unconventional, creative, or "special" since their lawyers would be at pains to persuade a hearing officer that such an approach is "appropriate" if lawyers for the student's parents argue otherwise. The door only swings one way for parents, however, as attorneys for parents who seek unconventional education programs for their children also tend to lose the "appropriateness" argument. The IDEA law and judicial norms conspire against creative approaches to educating special-needs students, regardless of which side is advocating them.
The sad result is that education officials blaze a trail of paperwork showing that proper procedures have been followed and, usually, that a standard (therefore "appropriate") set of services has been rendered. Is a given child receiving an appropriate education? If dozens of boxes have been checked on a form verifying such things as whether parents have been notified of all the summer classes that are available to their student or whether the child has access to nutritional advice, then the answer is yes. Is he being educated in the "least restrictive environment"? If the paperwork confirms that all school facilities are handicapped-accessible and that a certain percentage of each disabled child's school day is spent with nondisabled students, then administrators feel that they can demonstrate, in court if necessary, that the child is being educated in the LRE. In Baltimore, teachers and administrators must verify that 350 separate procedures have been completed for each special education student, regardless of the type and severity of their disability. Informal surveys suggest that teachers spend as much as two days per week just filling out the necessary paperwork.
In school systems where parents are engaged and feel comfortable going toe-to-toe with school administrators, this rights-based accountability system is effective in guaranteeing access to a range of interventions for disabled children. But in areas where parents are not empowered vis-à-vis large institutions, as in urban public schools, this accountability system can't even guarantee access. Students in cities like Baltimore and Washington, D.C., where the special education systems have operated for long stretches under judicial supervision, can go years with undiagnosed and untreated learning disabilities. What a rights-based and process-focused accountability system can't do, even in the toniest of suburbs, is guarantee that disabled students are receiving a solid education.
IDEA '97
Congress and President Clinton sought to correct many of these shortcomings in the 1997 reauthorization of IDEA. The law was touted as a sea change, aligning special education with the results-based accountability movement. However, as so often happens in reform legislation, the 1997 IDEA amendments merely layered new testing and reporting requirements onto the existing system, further distracting schools and teachers from their core mission. To my knowledge, only one previous federal process regulation was dropped as a result of IDEA '97. Students with permanent disabilities, such as blindness, no longer need to recertify their disability every three years "a sensible reform, undoubtedly, but hardly a regime shift.
IDEA '97 mandated that disabled students be included in state testing and accountability programs. But the testing requirements of the 1997 amendments harbor many waivers and opt-out provisions. Students deemed "untestable" by professionals are excused from testing. States must report how many special education students didn't participate, but there is no upper limit on the opt-out rate. In addition, local administrators may, quite justifiably, provide special accommodations to students with disabilities who are tested. The accommodations can include modifications of the tests' timing, method, and physical surroundings. Critically, the testing accommodations that are provided to a given student, at the discretion of local administrators, can vary with each administration of the test, thus jeopardizing any longitudinal record of achievement gain or value added. In theory, administrators could classify more students as disabled each year (see Figure 2) and then provide each one with increasingly generous testing accommodations until a desired level of aggregate achievement was obtained. There is no hard evidence that administrators are gaming the numbers, but under the rules of IDEA '97 they could do so and no one would be the wiser.
Most important, the IDEA '97 reforms failed to change the culture of proceduralism in special education. The overwhelming majority of administrators remain obsessed with process and paperwork, whether they like it or not (many don't). Administrators admit that so many resources are plowed into completing the paperwork necessary to comply with both the procedural and results-based accountability dimensions of IDEA '97 that no one has time even to examine the compliance record once it has been painstakingly put together. Moreover, many of the "performance measures" that get reported do not actually measure genuine outcomes, but instead simply relate the percentage of disabled students being tested each year, the percent being mainstreamed, and other workload measures. Such statistics tell us nothing about whether disabled students are learning.
Results-Based Accountability
A true results-based accountability system in special education would retain the existing legal guarantees of diagnosis and services for students with disabilities at the front end of the special education process. However, the system for determining whether schools had met their obligations would focus on outcomes rather than on a strict detailing of services rendered. The hundreds of postdiagnosis procedural requirements and directives that are fixed in law or agency regulations "such as the requirements that most special education students spend 80 percent of their time in regular classrooms or that all students with disabilities have their hearing checked frequently "would be replaced by the simple dictate that schools demonstrate concretely that students are benefiting from their special education or modify the educational intervention accordingly. Education officials would still face incentives to mainstream and to regularly test student hearing, but only for the students who are likely to benefit from those approaches. The move to a results-based accountability system would entail a switch from the guarantee of a "free and appropriate education" to an assurance of a "free and effective education."
A results-based accountability system in special education would have the following specific features:
* Every student's IEP would set forth clearly: 1) what skills and knowledge the student is supposed to acquire; 2) over how long a period; 3) what specific tests would be used to measure those skills; and 4) with what specific testing accommodations.
* The tests and accommodations for each student would be applied consistently, year after year, for all students with nondegenerative disabilities.
* The process would begin with a set of baseline tests to measure initial levels of ability and achievement soon after the student has been diagnosed with a disability.
* Subsequent results would be reported as gains or losses from that baseline, noting also whether the outcomes exceed, meet, or fall short of the benchmarks established in the IEP.
* Reports also would include narratives from the teachers, counselors, and administrators who are educating the student, in order to place the gains or losses in context.
* Evidence of aggregate declines in the performance of the special education students in a given district would lead to a state or federally led intervention involving supervised programmatic changes.
* Persistent performance declines or a chronic failure to achieve sufficient progress at the individual, school, or district level would enable the parents of special education students to enroll their children in another public or private school of their choosing, with each child's entire per-pupil spending (regular and special education) following her to the chosen school.
Two elements of this proposal stand out. First, using gain scores along with benchmark achievement assessments is critical. Special education students are, well, "special." They exhibit various handicapping conditions of varied severity that affect their educational ability and achievement. By using the metric of student-specific academic gains instead of a somewhat arbitrary standard of achievement to evaluate special education students, the system would automatically control for a number of preexisting conditions that are particular to each student.
The use of gain scores also minimizes the incentives for classifying a nondisabled student as disabled, since such scores measure individual progress instead of lowering the achievement bar. A school with a 3rd grade student who was never taught to read would not be able to excuse itself of responsibility merely by classifying the child as learning disabled and providing him with "services." If the school adopted that dubious approach under a results-based accountability regime, the student's current ability level would need to be assessed and the school would be required to demonstrate that the child was making adequate yearly progress as determined by an annual assessment using the same testing accommodations. The school still would have to teach the child to read or face the consequences. Moreover, the gain-score results generated from the annual testing program would provide valuable feedback to educators regarding which LD students are responding favorably to interventions tailored to the special needs of LD students and which are not (and therefore may not really be LD). An inaccurate diagnosis of LD therefore becomes a liability to a school, where it might once have served as a cop-out.
Second, greater customer choice is likely to enhance accountability in special education. Experimental customer choice programs, such as public housing vouchers, have demonstrated that choice initiates a flight to quality. The behavior of customers who have choices provides important feedback to decisionmakers, helping them invest more money and effort in what works and waste fewer resources on what fails. The power of parents to move their disabled child out of a failing program would likely improve the outcomes for that child and motivate more teachers and administrators to achieve positive results for their students with disabilities. Currently, parents of disabled students who can afford good lawyers or self-financed private schooling generally enjoy such advantages. That opportunity should be available to all families, regardless of income, as the parental right to contest child placements should be replaced with the parental power to choose an alternative educational environment when the current one isn't working.
Here is how such a program might work in the hypothetical case of a bright 1st grader just diagnosed with severe ADHD. His initial IEP meeting includes a discussion of the seemingly best intervention program for him, but participants also spend considerable time reaching agreement as to what educational goals he should be expected to achieve and how progress toward those goals should be assessed on an annual basis. The primary goal is that he demonstrate gains of at least .8 grade level in the first year and 1.0 grade level in all subsequent years of his schooling. To ensure an accurate assessment of performance, he is to take the same annual achievement test as his peers, individually administered in a separate room, with 20 percent extra time and three times as many breaks. The accommodations are to remain consistent every year, even if the student leaves special education. The IEP also establishes a measurable behavioral goal of five-minute annual increases in the amount of time he is able to remain in his seat in class, as measured observationally.
Say the results of his first-year evaluation are disappointing, confirming the sense of his parents and teachers that the special education intervention program wasn't the best fit for his needs after all. Fearing a second year of disappointing results, and the subsequent risk of the child's transferring to a different school, his teachers try a novel approach, a computerized math instruction program, since the assessments indicated that he was the furthest behind in math. The results are better in the second and subsequent years. Special education is demonstrably working for this child.
The President's Commission on Excellence in Special Education recently endorsed many elements of this vision for special education reform. Its primary recommendation is that the accountability system for special education be redesigned to focus on educational outcomes. To realize that goal, the commission urged the Department of Education to reduce its focus on monitoring a large number of postdiagnosis procedural requirements and instead concentrate on "a much smaller number of substantive measures guided by broad federal standards that focus on performance and results." It recommends that many procedure-monitoring staff positions at the Department of Education be redesigned as positions of technical assistance to the states and localities in implementing effective results-based accountability in special education.
Answering the Critics
Critics of results-based accountability in special education typically raise four objections. They argue that it puts too much pressure on the disabled child, generates unreliable performance numbers, wastes resources that would be better spent on services for youngsters, and is unnecessary in light of changes to federal law as a result of the No Child Left Behind Act of 2001. Let's deal with each in turn.
First, results-based accountability need not unduly pressure students with disabilities. By establishing the appropriate testing system and accommodations in each student's IEP, we could ensure that no special education child is saddled with an overly aggressive testing program. The real world that most students with disabilities aspire to and will eventually enter is filled with "tests" that they will need to take and, one hopes, pass. Shielding them from the reality of testing while they are being educated will only do them harm. If learning test-taking skills enhances the performance of nondisabled students on tests, then learning test-taking skills improves the life prospects of students with disabilities, too.
Second, test scores can be reliable measures of educational achievement. Much of the lack of reliability is due to modifications in testing conditions. That is why the IDEA '97 approach to testing students with disabilities is a recipe for inconsistency, since it allows local administrators to alter testing conditions for students over time. Although test scores are not perfectly reliable indicators of achievement, a student's previous year's test score is consistently the best predictor of the student's current test scores. Including narrative evaluative comments by a student's teachers and aides can explain legitimate reasons for any large fluctuations. Finally, by focusing on general trends, as opposed to merely year-to-year variations, assessments can more accurately capture the systemic performance of schools, districts, and states in educating their students with disabilities.
Third, the resources needed to build a results-based accountability system would not be excessive. Teachers, administrators, and researchers have stated time and again that today's procedural accountability system in special education consumes massive amounts of time and money. If lawmakers were to embrace a results-based accountability system, the record of testing and progress could then replace the record of procedures and services. The net effect of substituting test results for paperwork is almost certain to result in a significant gain in time and money that can be directed toward classroom instruction, tutoring, therapy, and other services.
Fourth, the No Child Left Behind Act, which provides sanctions for schools that fail to educate disabled children, creates strong incentives to take advantage of the many waivers now present in the IDEA to excuse disabled students from testing or to modify their testing accommodations in ways that are likely to generate artificially positive results. Absent the sorts of safeguards that I have proposed, the accountability mechanisms of No Child Left Behind will be insufficient to guarantee results-based accountability in special education.
A sound results-based accountability system could even attract more funding to special education. Congressman Pete Stark (D-Calif.) has introduced legislation to reward improved special education results at the state and local level with increased funding from the federal government. Federal funding is important to the success of special education, since the costs of educating even a few severely disabled students can squeeze the finances of any district, but especially small districts. Congressman Stark's proposal is a laudable attempt to permit states and localities to earn their way to full federal funding in special education, through demonstrated results.
Conclusion
Special education has often been described as a "third rail" of policy reform. Because the current system, with all of its flaws, ensures that many of America's most vulnerable children are given access to special services and, whenever possible, significant exposure to a regular educational environment, parents and advocates oppose reforms because they fear losing the opportunities that children now have. Analysts and reformers tend to focus on the shortcomings of the system, and in so doing they risk being characterized as opponents of children with disabilities. Not surprisingly, special education reformers tend to keep quiet. Thus, the clarion call for reform that has come from liberal and conservative think tanks, respected members of the academy, Republican and Democratic members of Congress, and a bipartisan presidential commission should give us pause. Although the political conditions may not be ripe for major policy reform in special education this year, there is a growing consensus that we can and must design a better system for diagnosing, educating, and verifying the educational progress of special education students.
Legal rights to a special education program for students with disabilities must be preserved, but the specific content of a given student's program ought to be guided by evidence about what is and isn't working for the child, not "one-size-fits-all" mandates from federal, state, and local regulators.
A results-based accountability system would allow special education teachers and administrators to spend more time tracking each student's progress (and using that information to generate even more progress) and less time holding meetings and completing paperwork. Program overseers, legislators, and the general public would have a clearer idea of the extent to which the $50 billion special education system is working. Parents would know more about how their own children are doing in special education and when a dramatic change is needed. As a result of such information, more resources would be invested in the system and more wisely. Perhaps we would inch closer to the vision of what we want to achieve in terms of educating students with disabilities. At a minimum we would have a clearer idea of how much further we need to go. That alone would be a dramatic improvement.
Patrick J. Wolf is an assistant professor of public policy at Georgetown University and a former lobbyist for people with hearing impairments.
Copyright Education Next Winter 2003