When Testing Technology Fails, Students Fear They Will Too
If you’ve ever suffered from a recurring nightmare, you know how stressful it can be.
These dreams are often associated with feelings of terror and extreme anxiety. In some cases, the fear of reliving those circumstances can even become debilitating.
Students in a number of Indiana’s schools suffered a shared nightmare a few years ago that has reared its head once again: technology problems on the online portion of the ISTEP+, the state’s standardized test.
This time around they got relatively lucky – problems occurred on a preliminary dry run, not the actual, graded assessment. But it still had an effect on test-takers.
Driven by the fear of encountering trouble a third time, schools and state officials are taking steps to deal with the issue so that it won’t happen again – when it really matters.
What happened?
Just a few weeks ago, students at Mitchell Community Schools logged in online to try out this year’s ISTEP+ practice test. About five minutes in, some kids started noticing problems. And only two minutes after that, everybody’s tests had stopped working. 
This was not the first time Mitchell students – along with their peers in many other districts – had problems with ISTEP+ technology. Back in 2013, multiple school corporations had to suspend testing after students had trouble logging into the test website. This time around, the issues were very similar, if not identical.
Sam Klawitter, technology director at Mitchell Schools, explains that the load on systems at CTB/McGraw Hill – the test vendor – was wildly more than they had anticipated. He equates that to a lack of preparation on CTB’s side, seeing as schools fill out a readiness survey for the vendor as well as the IDOE beforehand, providing information including the number of work stations to be used for testing.
“There should have been some foreknowledge of the types of loads that they would see during the statewide readiness test,” Klawitter says. Mitchell was one of an estimated 150 school districts to experience problems on round one of statewide readiness tests – otherwise known as “stress tests.”
- Stressed Out: Technology, Students Both Burn Out on Practice ISTEP+ TestsServer glitches kicked students offline during statewide readiness runs for the ISTEP+ this past January. And there’s not much time left to correct the situation – the first round of actual testing is less than a month away. StateImpact Indiana’s Rachel Morello reports on how schools and state officials hope to deal with the issue so that it won’t happen again, when it really matters.Download
It’s not called a “stress test” for nothing
The mishaps affect more than just scores. Sam Klawitter remembers overhearing an exchange between two students after test failures two years ago.
“One girl said, ‘how do you think you did?’ and the other girl said ‘I don’t know, I just rushed through the test and didn’t check my answers, because I was afraid that my answers would be lost,’” Klawitter recounts. “It had great, deep impacts on the students’ psyches.”
Phil Storm, assistant superintendent at Mitchell, says the nickname for the test of assessment technology couldn’t be more accurate.
“It’s extremely stressful!” Storm exclaims.
Sixth graders at Mitchell took a 20-point hit on their scores during the 2013 system outages – and Storm says he still sees the residual effects. He has a pocketful of anecdotes similar to Klawitter’s, about students who rushed through the test just to get any answer submitted – and adds that if continues to be the case, the school won’t get a very good read on what students are able to do.
School leaders say when testing technology doesn\’t work, they often aren\’t able to get an accurate read on what students are able to do. (Photo Credit: Jackie Kelly/Flickr)
Storm adds that students are well-aware of the sprawling effects of their test scores – how they play into school A-F grades, teacher evaluations, and even local property values – and it’s not helping them any.
“You hate to say this, but that’s the goal of our schools right now, is to do well on the ISTEP+,” Storm says. “Everyone understands A, B, C, D, F. The kids understand it. It’s important to them that they reflect their school.”
New Vendor, New Ideas
This is the last year in testing company CTB’s contract to provide Indiana’s statewide assessment. Right now, the Department of Education is looking for a new vendor to run the test next year and beyond.
And school leaders have plenty of suggestions for changes they would like to see that new vendor make.
Kathy Klawitter could write a whole book of ideas, if she wanted to. She has worked in the Northeast Dubois County School Corporation for 40 years – her entire professional career – so she has seen a lot of state testing.
Let’s start with just a few of her suggestions:
“If I were writing them a memo, I would ask them to support paper and pencil until technology is adequate,” Klawitter says.
This is a suggestion many have offered. According to the IDOE itself, only about six percent of school districts opted to give the ISTEP+ online. But, according to the state’s director of assessment, Michelle Walker, it actually saves money to use the Internet- and cloud-based testing system.
Michelle Walker, Director of Assessment at the IDOE, says taking the ISTEP+ using paper and pencil would be more costly for the state. (Photo Credit: David Hartman/Flickr)
“Paper-pencil is actually more expensive,” Walker explained in a January 28th testing webinar. “There’s all of the printing the shipping, the return, all of the scanning for the scoring, and frankly it doesn’t really align to college- and career- ready standards.”
But, Walker says she is up for compromise. If more schools wish to use paper-pencil after the second round of stress tests, she says, the IDOE will let them do so.
Klawitter also asks that the vendor be held accountable. “If the test collapses, they should pay the costs for staffing the tests and they should buy all the kids ice cream!” Klawitter laughs.
Walker says that’s what readiness tests are for. Since the first round failures, CTB has updated its test delivery client – the software students use to take the test. Schools will try it out during a second round of stress tests this month, on February 12.
Does Indiana need a statewide test?
Which brings us to Klawitter’s last suggestion: Why bother? Get rid of the test altogether. It’s another call echoed by plenty of other Hoosiers:
we already have a good idea how most schools will do on test, teachers are better judge of students than test — Michael Nardi (@iPublicPolicy) January 29, 2015
Suggesting changes to ISTEP only legitimizes a test being used incorrectly and abusively. ISTEP must be eliminated. — John Stoffel (@JohnnyStoffel) January 31, 2015
Indiana adopted new “college- and career-ready” academic standards last year – 2014-15 marks the first year they’re being taught in schools.
And to comply with the current version of the No Child Left Behind law, Indiana has to give a statewide test every year based on its standards. That’s a federal mandate. State Superintendent Glenda Ritz says in an ideal world, it’s not the way she would do things.
“This is not how the department or the state board of education would have chosen to implement a new assessment,” Ritz assured school leaders in the Jan. 28 webinar. “We want to have the best testing climate for our students.”
As long as having a test is the reality, Mitchell’s Phil Storm points out that having computer software that functions properly would help immensely.
“I don’t really care who the vendor is, quite honestly,” Storm sighs. “As long as the test works.”
Students sit for the applied skills portion of the ISTEP+ beginning March 2.