Who'd have ever thought that Texas, famous for finding all sorts of silly things to boast about, would suddenly find cause to brag about its educational achievements? Not a little, but a whole lot. And who'd have thought that what some people have come to call the Texas Miracle would be regarded with great respect, if not reverence, by a worshipful press and by politicians from one end of the country to the other. In education reform circles these days, Texas is everywhere. If Governor George W. Bush is elected president, the Texas school reforms—and particularly the state's whips-and-chains accountability system—are likely to become a model for national education policy, as they already are in a large number of states. "We're no longer comparing ourselves with the poor southern states, like we used to," said Ann Smisko, an associate commissioner in the Texas Education Agency (TEA). "Now we're taking on everybody." It's on those claims of achievement that much of Bush's legitimacy as a presidential candidate rests.

At the core of the Texas claims are not only the large increases in the percentage of the state's 3.9 million students who pass TAAS, the Texas Assessment of Academic Skills—which annually tests all students in grades 3-8, and in grade 10, in reading, writing, and math—but also the even greater proportional gains in the passing rates of blacks and Hispanics. These are high-stakes tests for both students and schools. Students who don't pass either the 10th-grade exit exam or three TAAS high school subject-matter tests will not graduate (and beginning in 2003, third-graders who fail TAAS will also be held back). And Texas schools are evaluated not only on a combination of dropout rates, attendance, and the percentage of students who pass, but also on the passing rates of various major ethnic and social subgroups. (This means that if white students do well, but Hispanics don't, the school will not be rated as exemplary.) If students do very poorly, the school could be graded "unacceptable." Schools rated as exemplary will win plaudits and some extra money; schools rated as unacceptable risk serious sanctions.

That innovative emphasis on the performance of minorities has earned Texas considerable—and deserved—notice. But some of Texas's claims are so striking they border on the incredible. The state's official numbers show that even as TAAS scores were going up, dropout rates were cut from an annual 6.1 percent in 1989-90 to 1.6 percent last year. If ever there was a case of something being too good to be true, this is it.

The Bush people graciously acknowledge that Texas school reform has been ongoing since Ross Perot went around the state back in 1984 ("the days before he was crazy") persuading parents and local boosters in a lot of little towns that math and science were as important as Friday-night high school football. They also acknowledge that a lot of other governors, including Ann Richards, Bush's immediate predecessor, had a lot to do with Texas's educational success, and they put lots of stress on the involvement and constancy of the business community, which, like its counterparts elsewhere in the South, has been working for the better part of two decades to shape up a once-primitive education system. But the Bush campaign makes much of the "miracle" nonetheless, reminding anybody who asks that 1994 was the baseline year for much of the accountability system—1994, of course, being the year Bush was first elected.

Playing With Numbers

Yet there's a lot less here than meets the eye. Even Bush's people concede that TAAS covers fairly simple stuff—that the high school exit exam, for example, is at best a minimum competency test pegged to a seventh- or eighth-grade level: A passing grade is roughly equal to the 25th percentile on any standard achievement test. It's hardly surprising therefore that close to 78 percent of 10th-graders this year passed it on the first try (they can keep trying until the spring of their senior year) or that the gap in scores between minorities and whites, though still large, has been reduced. And while the number of 10th-graders passing the TAAS reading test increased from 74 to 83 percent between 1994 and 1998, and the number passing the math test went from 58 percent to an impressive 80 percent, the average score during the same period went only from 78 to 84 percent in reading and from 70 to 77 percent in math, hardly overwhelming considering the huge emphasis on test prep that went into it.

Casting further doubt on the numbers is the finding of Sandra Stotsky, an expert on reading and, until recently, a researcher at Harvard University (she's now a senior official in the Massachusetts Department of Education), that as the TEA was reporting sharp improvements in TAAS reading scores, the difficulty of the passages on the tests was going down. "If the scores students achieved in the 1998 tests were higher than those achieved by their counterparts on the 1995 tests," she said in a report prepared for the conservative Tax Research Association (TRA) of Houston, "the decline in the overall level of reading difficulty of the selections on those tests . . . suggests that there may have been no real improvement in their reading skills. There may even have been a decline."

Meanwhile, a report produced for TRA by three California mathematicians compared the math portion of the test to a new and much tougher set of California standards and concluded that TAAS "is too highly focused on minimal achievement" in which most items require no more than sixth-grade arithmetic. Given the simplicity of many of the items on the exit exam, that may be an understatement: A restaurant bought 102 pounds of beef for $318. Which (of five choices) is the best estimate of how much the restaurant paid per pound? (For anyone who wants to try it, the 1999 exam is online at www.tea.state.tx.us.)

TEA tried to answer Stotsky by saying that while the passages may have become simpler, the questions became harder, but if that's the case, as Stotsky said, the resulting items would be much less a test of reading than of general intelligence, which more or less defeats their purpose. As to the criticisms of the math test, the crux of the response was an attack on California's own inadequacies in establishing a consistent testing program. "Why [would anyone] want the well-established Texas assessment and accountability system," asked a TEA statement, "to match the new [California solution] to this mess?" But Texas effectively conceded both points last August when Mike Moses, who was then still Bush's education commissioner, abruptly announced that Texas would in factdevelop a new and tougher test. In the words of Bush education adviser Margaret LaMontagne, "We've maxed out on TAAS."

The Real Dropout Rate

But there are larger questions here, beginning, most obviously, with the dropout rate itself. Walter Haney, an analyst at the Center for the Study of Testing, Evaluation and Educational Policy at Boston College, is certain that asreported TAAS scores have gone up, dropout rates have risen sharply, particularly for minority students. In a report submitted in connection with a civil rights suit brought by the Mexican American Legal Defense and Education Fund alleging that TAAS has an unjustified and thus illegal disparate impact on blacks and Hispanics, Haney calculated ratios of high school graduation to ninth-grade enrollment between 1979 and 1996. He concluded that in the period between 1989 and 1991, when TAAS became a high-stakes graduation test, the graduates-to-ninth-grade ratio—that is, the number of students graduating relative to the number of students in ninth grade three years earlier—fell from roughly 75 percent to70 percent for whites and from slightly under 60 percent to less than 50 percent for minorities.

Those numbers, like similar data produced by the independent Intercultural Development Research Association of San Antonio, may be skewed by the fact that a lot of ninth-grade students are now being held back—professedly because they are not passing their courses, but more probably because their schools are afraid they might depress the TAAS average and thus jeopardize the school's rating and raise the chances of some sort of negative sanctions from the state. But even if one uses other numbers—say the ratio of graduates to seventh-grade enrollment five years before—they show no progress for minorities and little for whites, remain consistently grim, and reveal the state's absurdofficial numbers as bureaucratic constructs with no roots in reality. In 1989-90, there were 250,000students enrolled in the seventh grade; six years later, 170,000 (68 percent) graduated. In 1992-93, there were 266,000 enrolled in the seventh grade; in 1998, 197,000 (74 percent) graduated.

For Hispanics, the seventh-grade-to-graduates ratio for both 1989-1995 and 1992-1998 was precisely the same—60 percent, which is slightly worse than, say, California's. That doesn't necessarily indicate a 40-percent dropout rate; students move in and out of the state, and Texas has done more than most states to try to create a reliable tracking system. But as a recent Texas legislative report pointed out, the state's own figures are meaningless. While districts reported that in 1997-98, 110,000students transferred out of state or out of the country, "no information is available to confirm" those transfers and neither is it possible to confirm about 25 percent of the 126,000 who were reported as transferring within the state. In addition, 67,000 students, more than twice the number of officiallyreported dropouts, were totally unaccounted for.

What's certain is that in a high-stakes system, the temptation to skew the reports grows in proportion, "whether [in the words of the legislativereport] by intentionally falsifying data, or simply stretching the rules to create more favorable data." And since the state doesn't count as dropouts the marginal students who choose—or are urged—to pursue a GED (general equivalency diploma) or the other alternatives that require no TAAS, the temptation to push them out and thus avoid another drag on a school's average grows as well. The temptation to play with the numbers in other ways is strong, too. Low-performing students can be made to evaporate from the rolls or be redesignated as special ed students; higher-scoring students can be temporarily transformed into Hispanics, thus lending their score to the Latino average; and the test answer sheets themselves can be altered.

And in the past three years, all these things have happened in some of the largest districts in the state. Earlier this year, Travis County Attorney Ken Oden indicted not only administrators but the whole Austin school district for changing student TAAS records to eliminate some low scores fromaccountability calculations. In Houston the number of students excused as not being proficient in English or as special ed unaccountably shot up a couple of years ago; at the same time, a principal and a teacher were fired, three teachers were suspended, and another principal was reprimanded on test-tampering charges. In Dallas a state investigation found two schools guilty of tampering with TAAS answer sheets. In addition the state is now investigating a number of districts,including the 25,000-student district of Ysleta—which hadbecome a national model for its apparent academic success—for gross misreporting of dropout data.

Finally, although the TEA says that a large number of private school students move into the public system in ninth grade, the number tested goes down sharply. In 1996, 259,000 eighth-graders were tested; in 1998, 227,000 tenth-graders were tested. That kind of evaporation makes all test score increases a bit more dubious.

The Taasmanian Devil

Which brings us to what one state official whimsically calls TAASteria and others refer to as TAASmania, but is in fact deadly serious. In hundreds of high schools, the period between Christmas and the end of February, when the test will be given this year, is devoted almost entirely to TAAS preparation—the rest of the curriculum in math and English (and sometimes in science and social science) is deferred. In other schools, English teachers devote half of each class period from the opening of school in late summer until the end of February to manuals with reading passages and writing "prompts" based on previous tests. At Manor High School near Austin, for example, in the final week before TAAS, all 10th-graders also attend "TAAS Camp" in churches and other community facilities. There they are broken into "teams" of 10, each with a teacher-counselor, for last-minute tutoring, advice in test-taking techniques, and general morale building—a team cheer, a team song—followed by an all-school TAAS Olympics. (While the TAAS campers drill with a large portion of the school's teachers, the school's other students take "mini-courses," often taught by outsiders, in everything from first aid and hunter safety to country-and-western dancing.) And at places like Houston's Jeff Davis High School, there are things like a voluntary "Lock-in Friday," an all-night session that includes test prep, plus games and movies—what one teacher called a "coming-together kind of experience." In some elementary schools, students are taught how to fill in bubbles without making smudges, beginning in second grade.

And then there are the football-style pep rallies, the $1,000-a-day consultants, and everywhere the commercial cram books and booklets; the tutorial software like "HeartBeeps for TAAS," which an estimated 1,000 schools have bought at $4,200 a copy; the instructional videos that show teachers how to set up their TAAS Target Practice bulletin board; and the practice tests and the other paraphernalia that feed on the hysteria and support a large and growing industry. There's PracTaas (a "suite of thirteen TAAS-like mini-exams"), Incredible Tutor (suitable for Mac or Windows), Guerrilla Tactics, and countless other preparation tools. "Death. Taxes. TAAS," says the promo for one. "The certainties of life. KIDPROV© removes TAAS from the top 10 stress list. . . . "

TAAS, in fact, seems to drive everything before it. In many schools, any student who fails the TAAS exit exam in 10th grade takes TAAS math or TAAS English until he passes it. The test, one teacher said, has become the curriculum, which means, as Russell Wallace, Manor's principal, put it, "that a lot of other things get shorted." For many Americans, some of those things—civility and citizenship, for example—may be asimportant as the multiplication table. And while there are people who argue that if you just do your regular teaching well, TAAS will take care of itself, that's not how the system behaves. Teachers will tell you again and again that in this state, which has no collective bargaining, their chances for promotiondepend on how well their students do on TAAS.

That means, almost inevitably, that other parts of the program have to make way—not only in the elementary schools, where a lot of the curriculum focuses on the basics anyway, but also in secondary schools, which are supposed to devote themselves to higher skills. Principals, said Linda McNeil, who trains teachers at Rice University, "are telling science teachers don't teach science; teach TAAS math prep. Others tell me they don't do anything but TAAS prep drills—commas and semi-colons and topic sentences until the test. Then in the last nine weeks they can teach eighth-grade English." "Thank God," said one teacher, "that the test is earlier this year. That will give me a whole nine weeks, instead of seven, to teach the curriculum."

Even in the elementary schools, the two months before the test are devoted largely to practicing for TAAS—to sample TAAS questions and writing prompts and the particulars of fractions. There are reviews of the four types of essay—classificatory, narrative, persuasive, and how-to—along with the formulas necessary to write them properly: five paragraphs of five sentences each. Perhaps more troubling still, some teachers will tell you confidentially that they're urged to concentrate on the "bubble kids," the students who are on the borderline between passing and failing the test, and not to worry about "the kids who'll never pass." There is no way to know how widely such things occur, but to the extent that the school people have learned to triage their students this way, one kind of neglect is merely replaced with another.

Although there's a lot of talk among some educators that TAAS pressure is driving good educators into either fields or grades in which there is no TAAS test—or out of public school teaching altogether—organizations like the Texas Federation of Teachers (TFT) support much of the accountability program even though they don't particularly credit Bush with it. John Cole, who heads the TFT, says that in a state where a school principal "used to be anybody with a master's degree and two losing seasons," there is now some emphasis on academics—itself no mean achievement—even though there are still far too many untrained teachers. For a union official like Cole, there is also the fact that TAAS and the constant talk about accountability have increased general public support for the schools, which this year produced a $3,000 raise for the state's notoriously underpaid teachers. "We've gone," he said, "from the twelfth to the nineteenth century. Now we . . . [have] to get to the twenty-first."

But hardly anyone denies that TAAS has become the driving force of the system, which, instead of being a floor has in many districts become a facade of success. Many pass the TAAS reading test, Linda McNeil told me, "by being able to select among answers given; they are not able to read assignments, to make meaning of literature, to complete reading assignments outside of class, nor to connect reading assignments to other parts of the course." There is no way to quantify what's not being learned.

Nor do the improvements in TAAS seem to be reflected on most other tests. While Texas scores on the math part of the National Assessment of Educational Progress (NAEP) have risen dramatically—the result, perhaps, of all that drill and test practice—Texas students made only small gains in the NAEP reading test between 1992 and 1998, no gain whatever between 1993 and 1996 on nationally normed achievement tests in reading, and not much gain in other subjects. And they have not made much gain on the SAT, either, despite all that drill and kill. At the University of Texas, Austin, the state's flagship campus, the undergraduate admissions people say that while school conditions have improved—and began to improve long before TAAS—they've noticed no change in the preparation of their applicants. Most tellingly, perhaps, Rand researcher Steve Klein recently reported at a National Academy of Sciences panel on testing equity that when he ran the TAAS numbers, the strong negative correlations between test scores and poverty that are common on all standardized tests vanished—"same kids, exactly the same kids two weeks later . . . and it blew up, no relationship at all." For him, this made all the TAAS scores suspect.

None of those things fully undercuts the numbers in the TAAS reports—the fact, for example, that blacks and Hispanics in 1999 passed the tests at a rate (70 percent for Hispanics, 63 percent for blacks) about as high as that for whites in 1994. As Rich Hill, a testing expert at Advanced Systems, puts it, "When people are determined to see their scores increase, scores do indeed go up." But the pressure and the unimpressive scores on other tests raise unavoidable questions about what the numbers really mean and about the cost of their achievement. The higher the stakes, the greater the temptation to fudge or game the system, to hold students back or otherwise reduce the number ofat-risk students in the pool, and to find new categories toaccomplish that—if not to cheat outright. And, as is the case with other high-stakes tests, ultimately the cut-off scores arepolitically determined, something that even the people on Bush's staff concede, a compromise between what will look good and rigorous and what the system—the parents, the school people, and the voters—will accept. As Haney says, "When the same mechanism is used to hold . . . students and adults and institutions accountable, the adults and institutions are in a betterposition to protect their interests than are the students."