Gov. Mike Pence released Edward Roeber and Bill Auty’s full review of the test Monday. The two men compiled their report within a matter of days after the governor hired them to aid in trimming the test last month. The document contains recommendations for immediate implementation in this spring’s test, as well as ideas for the state moving forward.
“We believe that implementing our long-term recommendations will improve the design and implementation of the ISTEP+ program in the future,” the report reads. “We remain willing to assist in and perhaps monitor efforts to implement these recommendations.”
As a refresher, the feds required Indiana to create a new test this year, after the state pulled out of the Common Core and PARCC last April. State leaders hope to better align the state test to state standards so they can create a more refined assessment for 2016 and beyond.
How can they do so? Let’s look at the short- and long-term fixes Roeber and Auty suggest…
The sooner, the better
Roeber and Auty advise that the Department of Education do the following this spring: identify the design and anticipated time for the 2016 test, and release that information publicly “to demonstrate that the testing time issues this year are a one-time event.”
The problem? Timelines. The IDOE originally anticipated releasing a request for proposals from vendors wishing to write next year’s test in late August – that got delayed. Staff at the Department of Administration had originally planned to award that testing contract last October – now, they say they hope to do so sometime this month. Since state lawmakers are already behind schedule, moving up due dates could prove problematic.
Short and sweet
Once state officials collect data from this spring’s test, they’ll be able to see how students performed on new questions – and how long they spent answering them. This will help them see if they can reduce test length in the future.
“Test reliability is directly related to test length: the longer a test, the more reliable it will be,” the report reads. “If a shorter test can produce sufficiently reliable results, testing time can be reduced.”
Strategies the consultants propose for trimming future tests include:
- Reducing the number of reading passages, items and writing prompts used to score the English/Language arts section, currently the longest component of the test, and
Developing “computer adaptive tests,” using computers programmed to customize the test by selecting items for each student based on their answers to previous questions.
The next question becomes who would do this analysis, and how soon? The state could ask its current contractor (CTB-McGraw Hill), the new contractor (yet to be determined) or an independent consultant – and whoever they select would need to work quickly to make decisions before 2016 test forms are built.
Ask for help
“The field of large-scale assessment is advancing rapidly in response to increased demands on assessment systems to support school and educator accountability as well as instruction of more rigorous and comprehensive academic standards,” the report reads.
In order to better address these demands, the consultants recommend Indiana establish a technical advisory committee to advise the IDOE and state board – something most states already do. The group would be comprised of five or six members with various backgrounds chosen to help support and troubleshoot different aspects of the state’s assessment system. Roeber and Auty strongly advise making this a priority – and making it happen fast. They suggest putting the group together and scheduling its first meeting for late spring or early summer.
The issue here: adding yet another group into the mix. The IDOE, state Superintendent Glenda Ritz, state board, general assembly and Gov. Pence all already have a hand in the world of testing – and relations haven’t always been the best. Throwing another player into the game could end up causing more harm than good.
Make a plan
Roeber and Auty say having a test blueprint to use for planning and communication will help the IDOE immensely moving forward. To avoid confusion and reliance on oral communication or scattered documents to guide the 2016 test’s design, the consultants propose developing a chart showing elements such as:
- Detailed descriptions of specific content standards to be assessed,
- “Boundaries” of assessable content, to guide item writers and show teachers exactly how content will be tested, and
- Descriptions of how test performance relates to college and career readiness.
This type of plan will be help the state’s new contractor build the 2016 test, as well as showing teachers what to expect next year.
Once Indiana hires a new vendor, Roeber and Auty recommend hiring experts to mediate a smooth transition. Such a specialist would help relay important information, keep things on time, and ensure analyses are carried out correctly.
“While this person or organizations will cost the state, the avoidance of the typical transition issues can be priceless,” the report reads.
Talk it out
In what could prove to be the most challenging of all recommendations, Roeber and Auty urge the IDOE and SBOE to work on improving communication between their staffs and with local school corporations. This has not been the group’s strong suit of late, but surely many would agree it would aid in this and many other of the state’s education efforts.
“Effective communication is difficult in all complex organizations,” the report says. “Effective leaders and project managers constantly strive to improve communication. Therefore any actions to avoid future problems with the state’s assessment system should include efforts to better coordinate agency planning, decision-making and implementation of those decisions.”
The consultants also emphasize a need to communicate better with those implementing the test in the field: local superintendents, assessment coordinators, principals, curriculum specialists, and teachers. They recommend creating flyers or newsletters to keep timely updates a priority.