Tranzform

CI5336 - Case Study 12 Reflection

Case Study 12 showed how a perfectly good idea could get into trouble without a proper evaluation strategy.

For Sam Gonzales, the idea of using video clips and multiple choice questions, all delivered by computer, was a perfectly good way to assess training. It would be easier and faster than paper-based short answer, multiple choice tests. The data from one pilot test of the assessment seemed to show that it was more accurate than the existing method in predicting which students would have actual on the job performance problems, based on their test scores.

However the more that test scores are tied to evaluation and pay in a business, the more political the assessment process can become. When two students and a trainer objected to low scoring assessments, and the students wanted to take the regular paper-based assessment, Sam was not in a good spot. He had no evaluation results that could demonstrate that the protesting students and trainer had no basis for their complaints.

By evaluating and testing his assessment idea in its define and design stages, he might have discovered:
1) cultural/political objections to the new method that he could have possibly dealt with
2) computer-based skill issues that might have made the assessment system unfair or unreliable for a significant number of individuals.

With a proper evaluation strategy, Sam might have found ways to improve his idea and increased acceptance of it before releasing it in a real-world application.