Why Some Educators Say The State’s A-F Grading System Isn’t An Improvement

June 28, 2012 | 9:15 AM

By Rachel Morello

• 3 Comments
• Tweet

Superintendents offered a harsh critique of the state’s A-F grading scale for public schools at the statehouse earlier this month, writes Karen Francisco of the Fort Wayne Journal Gazette:

How uncomfortable Friday afternoon must have been for Republican members of the Indiana Select Commission on Education. Listening to hours of overwhelmingly critical testimony of the A-F school grading system — a centerpiece of the Republican administration’s so-called reform agenda — would be bad enough, but the critics included the Indiana Chamber of Commerce. Nothing like irritating the state chamber if you’re intent on maintaining your pro-business profile.

The problem, speakers said, is not the fact that the state is assessing schools. It’s how the state is making the assessments. And to understand why educators don’t like the system that will replace No Child Left Behind in Indiana, we dive into the world of norm- and criterion-referenced testing.

‘If We Don’t Implement, We Lose The Waiver’

Make no mistake — Indiana schools will get a letter grade next year, Superintendent for Public Instruction Tony Bennett said. It’s one of the conditions Indiana must meet to be released from the national NCLB standards.

“What we should be trying to do is really to build systems,” Bennett told legislators after nearly four hours of public testimony on the A-F grading scale. “Because as a group, we can’t probably solve the specifics but we can build systems that allow the (superintendents who spoke) to do just those things with their teachers and schools.”

Bennett agreed with the educators who said Indiana’s system isn’t perfect. But he also said the state didn’t have time to work out every kink before rolling out the A-F grading scale. Now that Indiana has its waiver, the legislature can use input to improve the system for the future.

Jon Gubera, the chief accountability officer at the Indiana Department of Education, stressed that without the rewired A-F scale, Indiana would have to again adopt NCLB standards.

“If we don’t implement, then we lose the waiver,” Gubera said. “The immediate effects of that is the C-capping mechanism would be back. So in other words, schools that failed in one single subgroup — even if they’re an A school — they’d be capped at a C, which was a major issue last year.”

In 2011, more than 300 Indiana schools would have received As and Bs if not for the cap.

Norm- And Criterion-Referenced Tests, Explained

But some of the superintendents and educators who provided testimony say Indiana’s letter grade system still isn’t the answer. They said Indiana’s growth model places too much emphasis on measuring how students’ test scores improve relative to their peers — and too little emphasis on what information they actually know..

Here’s how the current system works: The state looks at all of the students who scored the same on the ISTEP+ in a given year. Then it tracks their academic progress over the next school year, again using standardized test scores. That helps the Department of Education determine how much growth on the state’s test is representative of one academic year. Students are then assigned a percentile based not on how well they scored, but on how their year-to-year scores ranked against their peers.

And here’s where norm- and criterion-referenced tests come into play. But first, the difference between the two:

• Norm-referenced tests compare one test taker to the others. Norm-referenced tests can tell an educator if a student knows more or less about a subject than his peers. But they can’t say how much the student knows. When test scores are reported as a percentile, then the exam has probably been norm-referenced.
• Criterion-referenced tests measure how much the test taker knows about the subject. In theory, everyone can pass a criterion-referenced test because a student’s score isn’t based on how well his peers perform on the test. A good example of a criterion-referenced test is a driving exam — there’s no limit to the number of drivers on the road, so as long as you follow the rules, you get your license.

Another example of a criterion-referenced test is Indiana’s ISTEP+ exam. That’s where things get tricky for the DOE. Indiana’s exam might be criterion-referenced, but the state’s growth model — borrowed from Colorado — is normative.

‘Bottom 33 Percent’ Still Needs Help, Superintendent Says

Here’s how Merrillville Superintendent Tony Lux explains his problem comparing Indiana students to their peers (the Fort Wayne Journal Gazette has the full text of his remarks, which are worth a read):

The phrase ‘poverty is not an excuse’ has become an excuse — an excuse to ignore poverty and disclaim any responsibility for it and its devastating effects. The strategy is becoming all too clear — ignore poverty, blame the effects of poverty on teachers, maintain the public perception of failing teachers and schools with an A-F formula that is designed to rank order students so that the bottom 33 percent will always exist (no matter how much achievement gains are made), use it to designate teachers and schools with low grades, then create a red herring for an impatient public by offering a placebo known as charter schools and school choice to appease them.

Invariably, Lux said, Indiana will have to designate one-third of its students as “low-growth,” and the state hasn’t given school districts the tools they need to help these students achieve. As a possible fix, he proposed mandatory summer school for students who aren’t meeting grade-level expectations.

“If the state can take a year out of a student’s life for not reading at grade level by grade three, how can it not be willing to take a month out of a student’s summer?” Lux asked.

Outgoing Lafayette Superintendent Ed Eiler echoed Lux’s concerns, saying he wasn’t sure how to explain to a parent whose child scored well on standardized tests two years in a row why that student was low-growth. It’s more than a philosophical difference between norm- and criterion-referenced tests, he said.

“We really believe one of the flaws is we’re trying to use one measure, which is a student growth measure, for multiple purposes,” Eiler said. “It is appropriate as a student growth measure, but when you begin to apply and use the same measure to measure the performance of teachers and schools, that’s where it breaks down.”

Elier said he doesn’t buy the argument that the growth model averages out when it’s applied across the state. At the end of the day, educators in low-income or high-mobility areas aren’t teaching in a classroom with a perfect distribution of high-growth students, he said.

All Students ‘Can Be High Growth,’ According To Bennett

Bennett disagrees that the state’s model divides students into high-, typical- and low-growth. He says there’s been a tremendous amount of misinformation about “fixed percents” and the A-F grading scale.

“Frankly, 100 percent of your students can be high-growth,” Bennett told educators at the Select Commission on Education meeting.

He says that he’s been having conversations about putting a criterion-referenced test at the center of a norm-referenced model with Lux since the Department of Education unveiled its A-F scale. Bennett called the state’s model “statistically valid” and promised to work with educators to help dispel the myth that the system will divide Indiana students into thirds.

Bennett says the change he’d most like to see to the current system is more support for high-growth kids in the upper 25 percent.