Fifteen years ago, North Carolina rolled out its first “ABCs of Public Education” report, ushering in an era when student test scores shaped the reputation of schools and the communities they serve.

Thursday the state will make the last such report. The No. 2 pencils and bubble-in answer sheets that symbolize the end of a school year are being phased out for online testing. More important, school ratings based on basic skills will yield to a system that aims to better gauge how well students are prepared for college or work.

For the last time, this summer’s report will allow top performers to hang “school of excellence” banners, while saddling the worst with the “low-performing” label. Starting in 2013, state legislators have mandated that N.C. schools will be graded like students, from A to F.

Fifteen years ago the ABCs put North Carolina in the vanguard of a national quest to use test scores to size up classrooms.

“Before the ABCs, someone could say, ‘Oh, my school is great,’ or ‘Those schools are failing,’ ” says N.C. Superintendent June Atkinson. “The ABCs gave a school-by-school analysis of student growth and achievement.”

The accountability movement, which morphed into the federal No Child Left Behind Act, has changed the way people talk about public education. It has left little doubt that race and family income shape the odds of academic success for children, in the Carolinas and across America.

But, after 15 years and millions of dollars, has it made schools better?

People like Atkinson and Bill Anderson, a former Charlotte-Mecklenburg Schools principal who now leads the MeckEd advocacy group, say yes. They point to gains the state’s students have made on national exams, especially in math, over the last 15 years.

“The ABCs did make a huge difference,” Anderson said. “It forced teachers to follow a more rigorous curriculum.”

Carol Sawyer, a former CMS and long-time skeptic of the program, counters that it’s almost impossible to tease out the value of test-based school ratings.

“I think you’d be hard pressed to find real evidence that the ABCs transformed student learning in any way, shape or form,” she said.

There’s one point of near-universal agreement: Far too many students remain ill-prepared for jobs that require sophisticated thinking and strong academic skills.

The question is whether the state’s new approach, dubbed “Ready,” can help them get there.

Rollout rattles schools

N.C. lawmakers were hardly the only ones seeking ways to size up schools and force faster improvement in the mid-1990s. Across the country, educators and policymakers were grappling with what President George W. Bush would eventually dub “the soft bigotry of low expectations” -- the notion that schools weren’t providing a rigorous education for many minority and disadvantaged kids.

But North Carolina beat many states to the starting line when it launched school ratings based on reading and math exams in 1997. The ratings tallied how many students demonstrated grade-level skills -- and for the first time, highlighted the differences among schools within a district.

At the time, North Carolina was also among a small handful calculating students’ year-to-year progress. That “growth measure,” which was used to award faculty bonuses, is something that many states are trying to emulate today.

“It was cutting edge,” says N.C. testing coordinator Lou Fabrizio, who was part of the 1997 rollout.

It was also controversial. The growth rating meant that teachers at some lower-scoring schools got bonuses for making progress with struggling kids, while some high-scoring schools got nothing.

“Those folks that felt like they were high on the pedestal realized they maybe needed to do a better job,” Fabrizio said.

Anderson, a former principal of the high-performing Myers Park High, recalls how nervous principals were about the new ratings -- and how quickly schools came to take pride in the banners and bonuses that came with a good showing.

But while the exams and ratings pushed many teachers to step up their game, CMS sometimes overreacted with rigidly formatted lesson plans designed to boost test scores, Anderson says.

“It took away some of the creativity that teachers had,” he said. “We got too prescriptive.”

Summer drama

The ABC reports became a source of summer drama. State teams swooped into low-performing schools. Rival schools and districts eyed each other’s ratings.

Over the years the state added exams, including writing, science and an array of tests given at the end of high school courses. Sometimes, unexplained spikes and plunges led officials to scrap one batch of scores and try to design better tests.

The emergence of the internet made scores accessible to parents, homebuyers and reporters, who began scouring data and comparing schools. About 65,000 people a month now check out the online N.C. School Report Cards, which include ABC reports.

The federal No Child Left Behind Act, passed in 2001, put North Carolina in the national spotlight as a pioneer in accountability. But it also added a new layer of ratings, labels and penalties to the summer test-score reports. In CMS, thousands of families were offered the chance to transfer out of high-poverty schools that fell short of the federal targets, which changed every two years. After repeated years of failure, those schools faced federal pressure to replace principals and faculty.

Frequent changes in the tests, the growth formulas and the rules for testing made year-to-year comparisons difficult.

“I think the one notable success was confusing the public,” says Terry Stoops, director of education studies for the Raleigh-based John Locke Foundation.

More challenges

Like most states, North Carolina has faced controversy over how well state exams measure academic competence. The ABC reports have consistently painted a rosier picture than the “nation’s report card” tests given to a sampling of N.C. students.

Howard Haworth of Charlotte, who was on the state Board of Education when the ABCs were created, says the state sacrificed “a great opportunity to be really special” by making it too easy to pass the tests. He and others from Charlotte, including CMS leaders, have argued that it’s more important to give a clear picture of who’s falling behind than to pump up gains.

Anderson says the ABC program suffered its biggest setback four years ago, when the recession hit and lawmakers cut the money for faculty bonuses. The rewards were never huge -- the maximum was $1,500 -- but “to have the rug pulled out from under teachers, that was a morale-killer,” Anderson said.

Charlotte also emerged in the thick of a national backlash against standardized testing. Local and national critics had long questioned whether multiple-choice computer-scored exams like those used in North Carolina give a meaningful snapshot of what students have learned.

In 2010, CMS began designing dozens of new multiple-choice tests to supplement the state program. The goal: Test every child, from kindergarteners through high school seniors, in every subject, with the scores used to rate the effectiveness of teachers. The performance-pay push and the beefed-up testing sparked protest from teachers and parents, some of whom threatened to keep their children home when CMS piloted the new tests.

CMS backed down. But the new state Ready program charts a similar path.

Ready for the future?

Across the country, the next wave of school accountability calls for better tests and more consistent standards, with officials using the data to help teachers improve their work, says Daria Hall of The Education Trust, a research and advocacy group based in Washington, D.C.

That’s what N.C. officials are trying to do with the Ready program, which includes an array of new tests and new approaches to using them.

North Carolina is part of a group of states designing new reading and math tests that will link to a national “common core” curriculum. CMS Deputy Superintendent Ann Clark recently told the school board the lessons and the tests will focus more on problem-solving and analytical thinking than simply picking a correct answer.

For instance, the board looked at a sample fourth-grade math question that asks test-takers to evaluate a hypothetical student’s estimate on whether 496+1,404+2,605+489 will yield an answer closer to 4,000 or 5,000. The sample item shows the student’s work, with a conclusion that the sum will be closer to 4,000.

“Is the student’s reasoning correct?,” the item asks. “In the space below, use numbers and words to explain why or why not. If the student’s reasoning is not correct, explain how she should have estimated.”

In addition, the state is working on 90 more tests to evaluate students in other subjects. Like the CMS pilot, the state plans to use those results to calculate a “value-added” rating that will show how much each teacher helped the student master the material. That data will be used to help identify the most effective teachers and help the rest do a better job of teaching the skills student need, officials say. Eventually, teachers will lose their jobs if they have a string of low ratings.

The new system also shifts the focus of high school ratings. Instead of looking at students’ scores on end-of-course exams, the new system will use results on the ACT, a college-readiness exam, and WorkKeys, a skills test used by many employers. Graduation rates will also be part of the new system.

“We think that we have evolved,” says Rebecca Garland, chief academic officer for the N.C. Department of Public Instruction.