![]() |
|
| Back to Main Page | |
|
Registration Process Improved for Second Administration of Computer-Based Testing By Jonathan Marshall, Candidate Representative, Candidate Liaison Committee
While much actuarial work is done using computers, CAS candidates took an actuarial exam in front of a computer screen for the first time in September 2005. Exam 1/P (Probability) was first offered by computer-based testing (CBT) in a three-day "window" in September. The second administration took place in February 2006. Almost all candidates sitting for Exam 1 in North America and many taking it worldwide used the CBT format during these first two CBT windows. An advantage of CBT is the ability to offer exams more frequently. There will be four testing windows for Exam 1 in 2006. The registration process for the second administration went smoothly as the result of addressing challenges of the inaugural testing window. To address testing issues, the CAS, Canadian Institute of Actuaries (CIA), and Society of Actuaries (SOA) worked together with the CBT vendor Thomson Prometric. Thomson Prometric is a leading vendor of CBT, providing electronic administration of college admissions exams and licensure exams for the medical, securities, education, and accounting professions. The first testing window of the Exam 1 administration took place September 26-28, 2005. The second window was February 21-23, 2006. Instead of presenting themselves at a set examination time, actuarial candidates scheduled an appointment with local Thomson Prometric testing centers during the three-day window. Results for the September sitting were released just six weeks after the exam-on November 11, 2005-two weeks faster than previous sittings that were offered exclusively by paper and pencil. By the end of 2006, it is anticipated that candidates will receive an unofficial score immediately after completing the exam. Prior to the first testing window, the CAS, CIA, and SOA established an Implementation Team to get the ball rolling with CBT. As with any significant change, the team expected "bumps along the way" in addition to the types of problems that typically accompany a paper-and-pencil exam administration. The Team communicated regularly with Thomson Prometric to resolve issues as they came up. There were significant issues that deserve mention.
In the September window, Thomson Prometric did not have Exam 1 activated in their online scheduling system for the first few days of the announced scheduling period. As this happened over a weekend, it was difficult to get the situation resolved quickly. The SOA received so many calls from concerned candidates that their telephone system was damaged. On the following Monday, the problem was resolved, and the scheduling process proceeded. Hoping to avert such a problem in the future, the Team asked the vendor for improved internal procedures and controls. The registration for the February testing window was greatly improved. Initially, Thomson Prometric intended that all scratch work would be done on erasable whiteboards, as is the case with some of their other computer-based tests. Candidates, as well as members of the CAS, CIA, and SOA, felt that this would not work well and provided feedback to the societies. The Team discussed solutions with Thomson Prometric, and it was decided that, for the September window, Thomson Prometric's standardized scratch booklets would be used instead of whiteboards. Many candidates felt that the single booklet of four scratch sheets was not enough to finish the computationally heavy 30-question exam. Although a candidate was permitted to receive additional scratch booklets, he or she was required to ask the proctor and return the used booklet before getting the new one. The Team worked with Thomson Prometric to improve the scratch booklet. A 32-page scratch booklet was used for Exam 1 in the February 2006 administration and will be used going forward. Another issue for some candidates was that the September pre-exam information did not mention whether downtime due to a rare computer freeze counts against the three-hour time limit for the exam. Candidates are now assured in the exam materials that the clock does not continue to count down in the event of a freeze-even momentary freezes.
Despite the challenges of implementing CBT, the test day went well for the vast majority of candidates in the first window, and the registration process ran very well for the second window. (At press time, the second window had not yet taken place.)
The Team continues to review the whole process. Regular teleconferences with Thomson Prometric facilitate real-time feedback on the active testing window and planning for future improvements. Two key goals are to provide candidates with immediate unofficial grades by the end of 2006, and to expand CBT sites internationally.
|