What Went Wrong With Indiana's Online ISTEP+ Tests — And When
Kyle Stokes / StateImpact Indiana
State superintendent Glenda Ritz takes questions from the media Wednesday afternoon.
Two separate issues requiring two separate fixes on the same testing company servers were responsible for this week’s failures in the online ISTEP+ system, state superintendent Glenda Ritz told reporters Wednesday.
After technicians solved Monday’s problem — a lack of memory on the servers — Ritz says another problem arose Tuesday. She added she will review testing company CTB/McGraw Hill’s contractual obligations and address the “high stakes issue” of whether to consider the results of the exam as valid “at some point.”
“Our first goal is to just get through the actual testing window and make sure all the students will be taking the test,” Ritz said.
CTB’s online status monitor showed a “green flag” through the school day, although Ritz had asked schools to cut their testing loads by half. State officials announced students had completed more than 300,000 testing sessions as of Wednesday afternoon.
Ritz says the testing window will remain open until May 15 — the extra three days she announced Tuesday.
School officials have expressed concerns about the extended testing window, saying reducing test loads could create scheduling problems as the school year winds to a close and other online assessments approach.
The Tick-Tock
State officials and CTB representatives are monitoring the ISTEP+ assessments in real time from the Indiana Department of Education’s offices, Ritz says. It was there that they saw the first spike in testing interruptions on Monday — from a normal 2 percent to around 8 percent around 10 a.m.
“That’s when CTB decided to go over to the backup system that was going to sustain the rest of the day,” Ritz says. “But we decided to suspend testing.”
Overnight Monday, an IDOE memo to schools says technicians from the testing company added additional memory to the servers handling the tests. While this would theoretically solve the problem, Ritz says the servers overloaded on Tuesday, causing the second freeze-up.
“Really, the same effect happened each day — that we weren’t able to sustain testing with the load that was coming through,” Ritz says. “But there were two different causes.”
State Board of Education members also expressed frustrations.
“The data we’re going to get is tainted, no question about it,” said board member B.J. Watts, a teacher, at the meeting Wednesday. “If this were data that I were taking in my classroom to inform my instruction, I’d throw it out.”
A spokesperson from CTB/McGraw Hill referred StateImpact to its most recent public statement:
The interruptions are not acceptable to the students and educators of Indiana or to CTB/McGraw-Hill. We have worked with the schools in Indiana for many years and value our relationships with them. We sincerely regret the inconvenience we have caused in these first two days of testing. We are doing everything possible to correct the issues immediately so that ISTEP+ testing will continue successfully through the remainder of the assessment window.
‘We Are [CTB’s] Largest Customer Right Now’
Here are some extended excerpts of Ritz’s press availability Wednesday:
We’ll be meeting with CTB, we’ll be talking about the validity of the questions. That is a prime concern. It is a very high stakes test in Indiana. So we have to make sure that we can ascertain the validity of the actual student assessment… Certainly we’re going to be looking at their contractual obligations regarding that. I haven’t really delved into that yet. My primary concern is that we get all of the student assessments completed. Then, I’ll proceed with other things.
We’re talking about sustainability of load — the numbers of students that are actually on at a given time and being able to sustain that load over a period of time. They are actually monitoring it in our office, we have members in our office from CTB, and we can see real-time when interruptions happen. Some interruptions, just online, are always expected, and they expect two percent… When it spiked yesterday to eight percent, we saw there was a sustainability with the load. That’s when CTB decided to go over to the backup system that was going to sustain the rest of the day. But we decided to sustain testing — we didn’t know how long that was going to take, and kick kids off. We were being proactive, really, yesterday to make sure that we didn’t have continued problems all day long.
We wanted to decrease the load in hopes that things would be totally fixed. They did several fixes last night, so we may say at some point, ‘Hey, we think that we can handle full load and go back to that capacity.’ We wanted to be sure today. My number one concern as the superintendent is that we get assessments completed. We are going to operated under the 50 percent load to be sure that we’re not going to have interruptions and get good information from the students on the assessments that they’re taking.
On Monday, they indicated that there was a memory issue and that they didn’t have enough memory in the system to be able to handle the load. They fixed that. But doing the memory pieces alone did not completely fix the problem. The next day, when we had the load on there and the sustainability issue then occurred. Really, the same effect happened each day — that we weren’t able to sustain testing with the load that was coming through. But there were two different causes. So now they think they have both causes fixed. We’ve had extensive dialogue with CTB throughout the day, all day yesterday and into the evening. Every morning by 5:30, we’re having more dialogue with CTB so I can inform our schools around 6 o’clock that the windows open at 7:30, we’re ready to go. I wanted to take precautions and do a 50 percent-reduced load so that we can ensure the students who are actually taking the test are not going to be interrupted…
The large load of testing is happening in Indiana. We are their largest customer right now. Indiana is in the forefront actually with having some technology access, so this is the first year that, really, all schools were required to give tests online if they had online access. Depending on that online access, how many grade levels they might actually be testing. So if a school had the capability to test all students, all grades online, that’s what they’re doing. If they didn’t, they might say, ‘Third graders are taking paper and pencil, the rest of them are taking online.’ This is really the first year for a huge number of students to be accessing an online assessment.
In a statement sent Wednesday afternoon, Ritz asked that schools continue to reduce their testing load by 50 percent on Thursday.