Inside the minds of test writers
The general aim of multiple choice tests is to gain truthful information about what students have learnt. The UMAT is slightly different as it tests an individual’s generic skills rather than specific content (as multiple choice tests usually do), however, the process by which test writers create psychometric tests such as the UMAT is very similar to the processes for creating any other multiple choice test.
Writing a multiple choice test is much harder than it seems. For every question that is ultimately published there would have been many questions and drafts that were written, trialled and rejected, prior to arriving at the final question that is issued as part of the test. ACER states that “all test questions must pass detailed panelling, trial testing, analysis and final review.” This scrutiny ensures that all of the questions on the test are relevant, fair and reliable, enabling truthful results to be attained.
The following are the two most basic and fundamental elements to writing an effective multiple choice test: firstly, test writers eliminate any barriers that could potentially prevent a knowledgeable candidate from responding correctly. Secondly, they will discard any clues that could assist a ‘less-than-knowledgeable’ candidate from correctly answering the question. This means that those who have not practiced or did not learn the skill should answer incorrectly.
The vocabulary used (other than terms you are expected to know) in multiple choice tests is generally kept simple (except in the case of Section 2 of UMAT which uses complex emotional vocabulary) and is usually high school level – sometimes a little higher. Therefore, if you are finding it quite difficult to understand the vocabulary throughout your preparation and practice tests, it is something you should work on before sitting the actual test.
Testers will not ask questions that can be answered using your common knowledge. You should always refer to and draw from any information or content provided in the question or the material you have learnt, in order to answer the questions.
When creating the answer options, test writers will aim to keep all options at relatively similar lengths because it is likely that a longer option is the correct answer due to a more lengthy explanation. Beware of this, because longer options make great distracters. Distracters are options that are extremely close to being correct and aim to put a little bit of doubt in the candidate’s mind when they think they have the correct answer. Often you will be able to eliminate options until you are left tossing up between the correct answer and the distracter. This highlights the importance of using the process of elimination to eliminate all incorrect answers first. If you need to make an educated guess, the chance of selecting the correct answer then becomes much higher. Test writers will also avoid using the same words in the stem and the correct answer as it is often an obvious clue to the solution. This can work both ways, as associations between the question stem and the answer can act as great distracters.
You should beware of trying to think like the examiner or assume that they are trying to trick you. While they are often trying to trick you, it could waste valuable time that you could use answering questions. It could also cause great confusion, not to mention the endless cycle of ‘what do they think I will think?’ Trying to anticipate the moves of a test writer can be quite dangerous, and it is generally best to just focus on answering the questions using the information you know or through your own logical reasoning. Nevertheless, you can always look out for some of the small things mentioned above when preparing for or sitting a test like the UMAT.