Why The Online ISTEP+ Servers Couldn't Handle The Testing Load
Photo Illustration by Kyle Stokes / StateImpact Indiana
It’s not that J.D. Ferries-Rowe doesn’t buy the explanation testing company representatives gave the Indiana General Assembly last week for the disruptions of April’s ISTEP+ exams — he does.
But Ferries-Rowe, the tech director at Brebeuf Jesuit Preparatory School in Indianapolis, wanted lawmakers to press CTB/McGraw-Hill executives further about the root cause of the problems: a lack of memory on the company’s testing servers in New Jersey.
“I wanted the follow up question, ‘Why didn’t you have enough memory on those computers?’” Ferries-Rowe says. “Was it just a cost-cutting measure, were you trying to save money? Did you grossly underestimate? Did you forget to carry the one? What happened?”
Though the state’s $95 million contract with CTB specifies the company’s online systems must be capable of testing up to 70,000 Indiana students at once, it turns out Ferries-Rowe’s question — why couldn’t the server handle the load? — is not easy to answer.
- 'Did You Forget To Carry The One? What Happened?'StateImpact Indiana‘s Kyle Stokes gives tech reporting the old college try… in hopes it might help you understand what went wrong when CTB/McGraw-Hill’s servers couldn’t handle the online ISTEP+ testing load.Download
“That sounds maybe simpler than it is. If all 70,000 students were taking the exact same test at the exact same time with the exact same pages, that might be a math problem,” says Dennis Cromwell, who helps run OnCourse, the online system at Indiana University that students use to check grades, register for classes and take online quizzes.
Overseeing a computer system with similar capacity needs to the ISTEP+ servers —OnCourse has to handle as many as 80,000 users at once during peak times — Cromwell says it’s important to remember that users are doing things at different times. He tells StateImpact:
Think about the fact that there are different ISTEP tests for different grade levels. All of the students are doing it at different times — so one might be processing one piece of the test versus another. Does that make a difference how far a student is into that test as comparison to one that’s just beginning…
I can’t say until I really know the application in some kind of detail. But you’d have to look at it. Is the load on one component of the test the same as the other?Related
Posts
- As State Officials Ponder Fines, Testing Company Execs Explain ISTEP+ Disruptions
- Testing Company Leaders To Face Lawmaker Questions In ISTEP+ Hearings
- Inside A Disrupted ISTEP+ Exam: What One Student Says About Her Score’s Validity
- Were Online ISTEP+ Testing Disruptions More Widespread Than We’re Hearing?
- With ISTEP+ Issues & New Model Coming, How Indiana Schools Will Get Their A-F Grades This Year
- After Online Testing Debacle, Ritz Appoints Third Party To Review ISTEP Results
- ISTEP+ Test Results Coming In July, Six Weeks Behind Schedule
- After Disrupted Exams, Indiana’s Largest District Says It Won’t Accept ISTEP+ Results
That load is often uneven, Cromwell says, so basically, server technicians have to strategically balance that load so the system can handle it. To do that, they allocate memory. To know when parts of the system will feel stress, Cromwell says server technicians perform “load tests” to map out how the system responds.
CTB Chief Digital Officer Stephen Laster told lawmakers Friday company technicians performed more than 50 stress tests of its servers, including 30 load tests.
Cromwell says he’s sat in the same hot seat before when things have gone wrong. After launching a new version of the OnCourse site in 2007 and 2008, Cromwell says Indiana University’s servers had the same load problems.
A fault in a tiny subcomponent caused a huge problem with the system — a problem big enough that university faculty called Cromwell before their committees to explain what went wrong.
“This kind of serious problem — if it were something obvious, it would’ve gotten fixed before McGraw-Hill delivered. There’s something subtle about the delivery of the ISTEP+ tests to the Indiana students that caused the problem,” Cromwell says.
While he empathizes with CTB staffers, Cromwell says the bottom line is that the company didn’t deliver on its promises.
Despite the scope of the disruptions — the computer problems disrupted at least 78,000 students’ exams; districts say the impacts are even broader than that — Ferries-Rowe says there’s an even bigger unanswered question. As he told StateImpact:
You have the mechanical problems of making sure that this many tests can be carried out with this many students. We’re two years away from the PARCC test coming down the pipe and we’ve never had a non-glitch e-test. It’s three years in a row now we’ve had something go wrong. So we’re not there yet, but we’re told we will be there in two years? That makes me nervous.
In some ways, this was an interesting hearing because the focus was on the mechanics of the testing process and not whether or not this test was giving us information that would be useful for students, for parents, for teachers, for student improvement. That’s a bigger discussion that would be nice to be having, but we’re not going to have that if we’re focused on, ‘Did we put enough memory in the servers?’
Podcast: Play in new window | Download