Tennessee officials knew of concerns about Measurement Inc.'s ability to fulfill its five-year, $108 million contract to administer the new online TNReady standardized test even before this year's failed rollout, according to documents and interviews with education officials in other states.
During the state's contract vetting process, the company's ability to oversee online testing at the level required by Tennessee was a concern. Measurement Inc.’s online system was relatively untested at the scale the state was asking, the records and interviews show. Concerns about the company existed in at least one other state.
Connecticut officials in a 2014 Tennessee reference check first raised issues about the company’s online testing platform known as Measurement Incorporated Secure Testing, or MIST.
"Their online test delivery system, MIST, has not been top notch," according to the documents used to vet the company in the contract selection process. "MIST has not been easily responsive to changes and additions, especially for innovative tests types or test accommodations."
The Tennessee Department of General Services, led by Commissioner Bob Oglesby, awarded the contract after a review of five companies. The department declined an interview request.
"Based on the advice of counsel, we have decided that it is best not to discuss this procurement in a live interview," General Services Department spokesman David Roberson said by email.
In February, the Tennessee Department of Education halted the online portion of the TNReady test because the company could not meet demand and switched to paper tests. The problems snowballed as Measurement Inc. missed repeated deadlines to provide almost 10 million paper versions to the state’s 146 school districts.
Tennessee Education Commissioner Candice McQueen announced Wednesday the department canceled its contract with the company after the failed rollout.
Now, as the education department seeks a new testing firm, students, parents, local education officials and state lawmakers are asking questions about what went wrong with Measurement Inc.
Measurement, Inc. President Henry Scherich said the company hasn't taken any legal action, but would not rule out the necessity. The company hasn't been paid for its work in the last year.
"We want to be paid for the work we have done," Scherich said.
MIST untested on a large scale
Connecticut used the software in a limited fashion for several years, according to Abbe Smith, Connecticut Department of Education spokeswoman.
"At the time of test delivery, it was for small subset of students that required special online accommodations," she said.
The New Jersey Department of Education, another reference, never used the MIST platform, but used the company to administer paper-and-pencil tests, according to David Saenz, a state spokesman.
In Tennessee, Measurement, Inc. scored the highest among the five companies in the bidding process, as determined by the state's general services bidding process. The state was looking specifically for someone to administer online tests on a mass scale.
"Based on the highest technical score and lowest cost, (central procurement) awarded Measurement Inc. the contract," education department spokeswoman Ashley Ball said by email. "At the end of the procurement process, a state department does not have the option to refuse the apparent winning vendor."
When the state sought the new contract, the Tennessee Department of Education helped select people to evaluate the proposals and develop questionnaires in the application review, Ball said. By law, those people aren't allowed to know the name of the company whose application they are evaluating to ensure a fair process.
"The Department of Education's role was truncated," said Kevin Huffman, who was education commissioner at the time and left office in 2014. "It was a blind procurement process as the law dictates."
Tennessee used Measurement Inc.'s online platform to administer writing tests in the 2014-15 school year. Although the company has a successful history of providing paper-and-pencil tests, the company's president said it had never used its online program on such a large scale.
"When it was deployed in Tennessee, it was the largest assessment ever attempted (for MIST)," Scherich said.
On Feb. 8, the inexperience in online test administration showed.
While the state's contract asked for the company to provide enough bandwidth to administer 100,000 tests at one time, Scherich said, the company fell far short of that number. At 50,000 tests, the network slowed, he said.
"In hindsight, we could have done something different to have enough internet connectivity between servers to handle it all," he said. "But we couldn’t handle enough traffic."
Scherich is adamant it was an issue that the company could fix within a few weeks. Students were still able to take tests despite the slowdown, he said.
But the state pulled the plug on the online testing platform due to an accumulation of issues leading up to February.
Ball, the state spokeswoman, said individual district internet bandwidth could have played a part in the slowdown. The state had already worked with Measurement Inc. to fix other functionality issues with MIST, but the bandwidth issues were new, Ball said.
"Given the amount of work up to that point, we did not believe it was prudent or advisable to engage in another round of corrections in implementation," she said.
Before TNReady
Before Measurement Inc., the state contracted with Pearson Education's PARCC — or the Partnership for Assessment of Readiness for College and Careers — for its online testing platform in the 2014-2015 school year.
But in the spring 2014, political groundswell against the Common Core State Standards-aligned test led to its demise in Tennessee. As a result, its online portion was never used.
Online testing was relatively new on a mass scale, but field tests by Pearson showed successes, according to an Education Week report at the time. And PARCC had seen success in other states, despite many hiccups,
Would Tennessee have been better off if it stuck with PARCC, which was in place before the switch to TNReady?
"Hindsight is 20-20, except for that question," Wayne Miller, executive director of the Tennessee Organization of School Superintendents, said on Wednesday. "I have no idea. That's pure speculation."
Moving forward, McQueen has said Tennessee has learned plenty in its first endeavor into online testing. How those lessons play into the next procurement process is unclear. Reference checks are only a small piece of the overall procurement process, Ball said.
"Reference checks, along with roughly a dozen other metrics, fall under the 'general qualifications and experience' category," she said. "The entire category is only worth five points out of the overall 100 points. Based on the way the system is structured, a negative comment would not necessarily throw a company out of the running if the overall numeric scores were positive."
A timeline hasn't been set for a new process in choosing a vendor to deliver online tests to Tennessee students. The Tennessee Department of Education plans to move fast in that effort.
For now, the state has suspended the TNReady test for grades 3-8. High schools have the test materials, and education officials say those must go forward as planned. But that means the state also is looking for a company to score its high school tests because it terminated the contract with Measurement Inc.
Scherich, Measurement Inc.'s president, said other online tests, including PARCC, have experienced problems in other states. In April, New Jersey students saw a computer glitch that stalled PARCC testing. In the end, Scherich maintained the issues could have been worked out.
"But they've told us to stop all work," he said.
The Asbury Park Press contributed to this report.
Reach Jason Gonzales at 615-259-8047 and on Twitter @ByJasonGonzales.