As State Officials Ponder Fines, Testing Company Execs Explain ISTEP+ Disruptions

• SLIDESHOW

June 21, 2013 | 8:33 PM

By Kyle Stokes

• 1 Comment
• Tweet

Apologetic executives from the testing company whose servers overloaded twice — interrupting April’s online ISTEP+ exams twice — told the combined membership of the Indiana House and Senate Education Committees that they took responsibility for the errors.

“I have family here in Indiana,” said Ellen Haley, president of CTB/McGraw-Hill. “It was very disappointing to have this happen and have this impact schoolchildren in Indiana. Please accept my sincere apologies and regret that anything at all happened.”

Haley’s statements came on the same day Department of Education officials announced they would be seeking at least $613,600 in damages from CTB — though a Department of Education spokesperson later stressed that the penalties “could reasonably go into the millions” — in part to pay for an outside review of the disrupted exams that’s already underway.

Citing that forthcoming review, which will offer recommendations on whether the state should throw out this year’s ISTEP+ results, co-chairs of the General Assembly’s Commission on Education — Rep. Bob Behning, R-Indianapolis, and Sen. Dennis Kruse, R-Auburn — urged lawmakers to focus their questions on the causes and immediate impact of the disruptions.

What — Exactly — Happened

Haley confirmed several of the details state superintendent Glenda Ritz sketched out in May. For instance, Haley says the most widespread disruptions, on April 29 and April 30, were both caused by two distinct memory issues with the company’s servers.

On those days, those problems rendered students and teachers powerless, facing what many of them called “the globes” or “the globe screen” — the term that describes the “Please Wait” screen test-takers saw. During her testimony, Haley explained these pauses happen by design:

The system captures every answer choice as soon as the answer choice is selected. We capture it and we transfer it from the students response to our servers — we have many servers, virtual servers, we have a database. We’re capturing every time they make an answer choice. We capture it, we transfer it. If there’s any reason why the system feels like it cannot transfer the answer, it will not let the student go on.

Thus, the globe screens.

Haley says company engineers performed 50 stress tests on its back-end servers prior to testing and engineers addressed one memory issue on Monday evening by adding more memory to the servers.

“The funnel had filled up,” explained CTB’s Chief Digital Officer Stephen Laster, so the engineers effectively added a bigger funnel.

But the first memory issue — with the server itself — had obscured the second software-related memory issue that caused problems to persist into Tuesday:

Tuesday morning seemed fine. By the middle of the morning, the late morning, schools started calling us again. We had engineers monitoring the system. We had people in the state, in the Department of Ed office, watching — we have a dashboard you can see how many students are testing, how many are being interrupted, how far along each student is in the test that they’re taking. We could see at about the same time that something was happening again. It was actually also happening on Monday, April 29, but we couldn’t see it because the main problem on Monday was masking the problem on Tuesday.

Haley pointed out, however, that after technicians addressed Tuesday’s issues, all students eventually completed their exams.

How Lawmakers Reacted

Rep. Kreg Battles, D-Vincennes, though, said memory problems with CTB’s servers didn’t explain the extent of the problems. Other local school officials have described problems with the online testing sites that go beyond the testing website kicking students out of their tests.

“On some of the testing, the full answer bank would not load,” said Battles, who’s also a teacher. “The kid literally had a question where there was no correct answer.”

After the hearing, Behning — the House Education Committee chair — told StateImpact he thought CTB officials were forthcoming, but says he still is looking for answers to questions such as those Battles raised.

“They probably brought up maybe more questions than some resolutions at this point in time,” Behning said. “We’re going to have to wait for when the validity studies are done.”

Though lawmakers do not have a direct hand in the fines — that’s the executive branch’s purview — Sen. Jean Leising, R-Oldenburg, asked Haley if she were in their shoes, what would she do about the company’s contract? Here’s Haley’s response:

I think you have to look at the whole contract and all the services we provide, to be honest — the test that’s developed, all the standard-setting that’s done, all the scoring, all the reporting, the building of the online system. Certainly, if I were a legislator, I would ask for answers about those two days and what language we have in the contract and what fines we should pay for that. And things that the state would perhaps want us to do. For example, the validity study — we’ve agreed to pay for the validity study. There are probably other things outside the exact letter of the contract to help make things right in a state we’ve had a long-term relationship with. But I think the contract, as it’s written, with a penalty section, and penalties, in contemplation of all the other things that have gone well, I wouldn’t say… I really, honestly, don’t see this as rising to the level of throwing the contract out, or the vendor. We’re dealing with technology, it needs to go right, it’s students in schools, we need to make it right, but I think the contract as it’s written gives us the opportunity to do that.

The Indiana Commission on Education will meet again on June 29.