The undercard of the marquee fight over whether or not to lift New York's cap on charter schools is a vitally important battle over how to define these schools' success.

Resident Scholar Frederick M. Hess

I support the spread of charters as much as anyone--because I believe they offer important alternatives to traditional public schools--but these tactics deserve scrutiny.

First, some background. There has been an ongoing dispute in the world of schooling about whether it's useful to judge charter schools based on crude end-of-year test scores.

Just about every serious charter school supporter and expert has concluded that this approach is deeply flawed--because charter schools enroll a self-selected body of students that may begin the school year performing at a level significantly different from their peers in traditional schools.

As was argued in a full-page 2004 New York Times ad sponsored by the Center for Education Reform, a pro-charter group, such scores fail to "take into account such key characteristics of students known to affect their performance as parental education, household income, and the quality of learning resources at home."

Instead, the ad made clear, the appropriate way to compare charter schools with district schools is by focusing on how much students improve during the course of the school year.

Remember that logic, then flash-forward to this year.

New York's most recent test results were just released by the state Education Department. The new data show that charter school students have higher scores on math and English than do district school students, at both the fourth- and eighth-grade levels.

Yet because this time the results are favorable for charter schools, some advocates are ignoring their previous warnings, apparently concluding that now it's quite all right to employ simpleminded comparisons.

The Center for Education Reform crowed that the "report shows better \[charter school\] performance at every level." Bill Phillips, president of the New York Charter Schools Association, declared, "The state Legislature has all the evidence it needs from this report to warrant lifting the cap on new public charter schools."

Charter proponents who play it fast and loose will come to regret their stance. The next time someone uses similar data to cheap-shot charter schools, charter defenders will be standing on shakier ground. It's hard to complain about unfair treatment when one's scruples appear to be a matter of convenience.

Frederick M. Hess is a resident scholar and director of education policy studies at AEI.