What School Performance Can — And Can't — Tell Us About College Preparedness

August 14, 2012 | 9:53 AM

By Rachel Morello

• 1 Comment
• Tweet

In order to graduate, Indiana high schoolers have to take tenth grade tests in math and English language arts. The Indiana Department of Education takes a keen interest in seeing school-wide improvement on these tests, which factor into the school’s letter grade.

But an Indiana University researcher has found that students who attend schools where scores are improving don’t necessarily do better on the ACT college entrance exam. In fact, he found that these students actually scored half a point lower on the 36-point ACT than students at schools where the tenth grade test scores weren’t as strong.

“While a half a point different isn’t a huge amount, a bigger concern for us is, in theory, it should be in the opposite direction,” says Adam Maltese, a professor in the IU School of Education. “We should see improvement in these ACT scores in that subject and maybe even other subjects.”

Maltese is a former science teacher. The study’s co-author, Craig Hochbein of the University of Louisville, is a former school administrator. Maltese says their research was born from a concern that frequent standardized testing in math and English mean other subjects, often science, receive less class time and attention.

Whether A School Is Improving Or Declining Matters Little On ACT

So here’s what Maltese and Hochbein did: They looked at how well all of a school’s tenth graders scored on the “end-of-course assessments” to determine if the school was improving or declining. Then they looked at how well individual students who went to that school did on the ACT. Students at schools where scores on end-of-course assessments were improving performed as well or even a little worse on the ACT than their peers at schools where scores on the graduation tests were declining.

But what the researchers found is open for interpretation. Even Maltese and Hochbein caution against drawing broad conclusions from their findings.

Here are a few possible explanations the authors offer:

• Unless they attend a school that is chronically underperforming, college-bound students remain on track throughout their high school careers.
• Most efforts to lift struggling schools focus on students with the lowest scores on standardized tests, as well as students who are “on the bubble” — not college-bound students who presumably are meeting grade-level expectations.
• Limited school resources may be going to teach basic literacy and numeracy skills to students who aren’t scoring well on standardized tests, leaving less funding available for enrichment programs for high ability students.

Why End-Of-Course Exams May Not Measure College Success

In an email to StateImpact Indiana, a spokesperson for the IDOE emphasized that end-of-course assessments and college entrance exams are very different measures.

For one, the tests students have to pass to graduate from high school are typically administered to sophomores, two years before they’d be heading off to college, writes Stephanie Sample. (Students sometimes take the tests as juniors or seniors if they’ve fallen behind.)

“You probably need more than a tenth grade understanding of math and English language arts to be successful in college science courses,” Sample wrote.

She also says it’s unlikely the students who are improving on the end-of-course assessments are also taking the ACT:

There’s also the possibility in these schools that are identified as improving, they were low-performing in the past. If that’s the case, there’s the possibility the ACT isn’t as commonly taken in low-performing schools because the expectation isn’t to go to college. As the school improves, that expectation is created, and the ACT becomes more popular. So you might have students taking the ACT in these improving schools who don’t have the additional supports that help students on the ACT (such as AP courses, study sessions, resources, etc.).

More schools offer and more students take the SAT in Indiana than the ACT …. This means the population of ACT test takers may change significantly year-to-year, which makes comparisons of year-to-year achievement inappropriate.

Sample says the IDOE agrees that some college-bound seniors might be insulated from efforts to help struggling students meet minimum standards, as the authors suggest.

But Maltese argues you can’t uncouple the two groups. He says even though Indiana has a waiver from some of the federal No Child Left Behind law’s strictest provisions, things like school letter grades and teacher evaluations are still tied to standardized test scores. Maltese says research is needed on what kind of effect efforts to improve schools are having.