Why Don’t They Fix That? Exam Survey Elicits Common Questions and Concerns

The exam is finally over. After taking a day or two to relax, you go online to fill out your survey. You take time to type in responses to the open-ended questions before you submit…but then what? You asked about the lined paper three sittings in a row, but nothing ever seems to change! Is anyone even reading these?

Yes! The Syllabus and Exam Committee and the Candidate Liaison Committee (CLC) members read them. Feedback from exam surveys and other sources is reviewed, and various CAS committees work to answer questions and solve issues when possible. Look no further than the additional offerings of the Course on Professionalism for a recent example of a time when feedback made a difference.   However, some things are slower to change, and some things stay the same for a reason. We have seen similar feedback on multiple exam surveys from multiple sittings, so we decided to respond to some of the most common questions and concerns here:

Syllabus and Exam Design

"Why wasn’t [insert topic here] tested this sitting? Shouldn’t all learning objectives on the syllabus be tested?"
Keep in mind that the exam is not the end goal; the end goal is education. The exam is simply designed to assess candidates’ success at learning the material. There are too many learning objectives to test each one on each exam. To encourage candidates to study all the learning objectives, the topics chosen for testing vary at each exam sitting. It is important to note that the range of weights provided on the syllabus are meant to be a guideline for candidates and not definite bounds for the exam.

"Why do we have to read multiple papers and deal with inconsistent notation? Shouldn’t there be a single textbook for each exam?"
If the notation or vocabulary in an exam question are ambiguous and cannot be determined from context, state your assumptions about how you interpreted the notation or which definition you are using. While a strict set of definitions and notations would be more convenient for exam purposes, it is not consistent with the real world. Notations and definitions can differ between countries or even between authors. In your work, you may get a request that includes a word that has variable definitions. If you didn’t know from your study experience that the word could be defined differently, you might not know to ask for clarification and could end up having to make corrections later (or worse, give someone results inconsistent with their needs). When completing your continuing education, you may read additional papers with unfamiliar notation, and your experience in learning from multiple sources will give you an advantage. Additionally, some syllabus topics are subjective, so it is important to see different perspectives.

"Why is there a guessing penalty on multiple choice exams?"
The concept is not unique to actuarial exams. The idea is to prevent test takers from earning points from a truly random guess. Without the guessing penalty on a question with five answer choices, a true guess has an expected score of +0.2 (1/5 chance of getting a point). However, if one answer is worth 1 point, but the others are each worth -0.25 points, the expected score from a random guess is reduced to [1 + (4 * -0.25)]/5 = 0. Note that if you eliminate one answer as impossible, your new expected score is [1 + (3 * -0.25)]/4 = +0.0625. Eliminating more answers produces higher expected scores (+0.167, +0.375, and +1, as you decrease the number of choices).

"Some of the questions take so long to read that I don’t have time to answer them."
The writers aim to make the questions as concise as possible while also avoiding ambiguity. Many candidates dislike having to state an assumption because the question was too vague, or they get flustered when they can’t figure out what the question is asking, so they prefer few extra words to provide clarity.

Exam Day

"Why is the exam printed double-sided? It is difficult to re-order the questions if there are multiple questions on a sheet."
This is to save on paper and shipping costs. The CLC has suggested that the CAS could reduce the amount of scratch paper provided to make up for the additional sheets of paper in the exam, and they are investigating the possibility. While no guarantees have been made, the aim is to make this adjustment if we get enough favorable feedback from candidates. If you have an opinion on this solution ("I love it!" or "No, don’t take away my scratch paper!"), please let us know by submitting your thoughts via the link at the end of this article.

"I hate the lines on the response paper."
We see a variety of suggestions on this. Some candidates request to eliminate the lines altogether, but that is an issue for candidates who struggle to keep their handwriting legible. Others say the spacing between lines should be increased or decreased, but the requests are inconsistent, so changes to the spacing are not expected. One thing we think everyone can agree on is that the lines are too dark and/or too thick. The CLC has asked for the lines to be printed thinner and/or in grey instead of black. Similar to single-sided printing, the CAS is investigating, and your feedback (see link at the end of this article) could make a difference!

"Four hours is so long! Why isn’t there a scheduled break?"
Aside from the obvious problem of candidates discussing the exam during the break, there are additional logistical issues. What if you are working on a question when the break comes, and you have to set it aside for 10 minutes? Will you lose your momentum? A potential solution for that is to split the exam into two parts, requiring you to finish Part 1, take a break, and then take Part 2. However, that creates additional work for the proctors and lengthens the overall exam process. It also limits your flexibility to go back and answer questions from the hypothetical Part 1 if you remember how to do them during Part 2. What if you are at a large site like NYC (usually around 100 candidates), and there’s a long line for the restroom? Don’t forget you may use the restroom individually (with permission from the proctor) or take a few minutes to meditate at your seat during the exam. At the large sites, it is uncommon to see more than a few people leave the room before the exam ends, so it seems to work in practice. Just remember that the clock continues to run while you are out, so it is important to avoid this if possible (consider avoiding caffeine before an exam).

Exam Frequency

"Why aren’t the upper-level exams offered more often?"
Each CAS exam takes hundreds of hours to create and grade, and all this work is done by volunteers. Because the CAS publishes the exams and examiner’s reports, questions cannot be reused. If the CAS stopped publishing the exams (and removed the option for candidates to mail themselves the exam booklets), they could start building a question bank. It might take several years, but eventually the bank of questions would be robust enough to offer more frequent sittings. Would you be willing to give up the examiner’s reports in exchange for a future with more frequent sittings? Please let us know your thoughts via the link at the end of this article.

Feedback Response

"I provided my contact information in the exam survey. Why didn’t anyone respond to my question?"
The surveys are intended to be anonymous. You should not provide any identifying information. You may also submit feedback via the link at the end of this article, and the CLC will "take appropriate action that may include communicating the concern to the appropriate CAS committee or posting a public response if warranted. Generally individual responses will not be made."

To give feedback to the CLC, use the CAS Contact Form. Also look for upcoming candidate surveys, which will provide an additional opportunity to submit feedback. And please continue to submit exam surveys. We hear you!