Don’t just think question types… think questions… then types

Photo Credit: albertogp123 via Compfight cc

Photo Credit: albertogp123 via Compfight cc

In my role, I see many module leaders present themselves with assessment questions that have been written to fit the question types offered by Blackboard (Course Resources).  In my opinion this does not always lead to effective assessment as what we are presenting to students is a collection of questions that Blackboard can handle rather than an assessment.  For example I recently saw 3 questions that collectively was supposed to ask about the students’ knowledge of the statistical averages.  The question asked was ‘What is the mean / medium / mode of the following numbers?’.  This was followed by a range of numbers and the student could pick a number from a selection presented.

This had been written after whoever wrote it had seen the question types and rather than thinking ‘what do I need to assess?’ (i.e. the students’ ability to differentiate between the different statistical averages), just simply written some simple maths questions which could be guessed at.

When I spoke to the academic in question, they informed me that the intention was to assess the student’s ability to understand and differentiate the different averages and had thought that multi-choice was the way to do this.  They actually wanted a question that asked about the different types but thought that as multi choice would make it too easy as it could possibly give the answer, instead went for a maths question as they thought that some calculation would be needed.

Going back to the original learning outcome (the ability to differentiate between the different statistical averages), I asked what would be a successful learning outcome – the answer being ‘the median being the middle value when laid out in order, the mode being the most common number and the mean being the result of summing all the numbers divided by the number of test results’.

From this we can see that there are three terms that the student needs to learn, rather than being able to do the maths.  So, straight away, there is a requirement to ask what the terms mean rather than if a student can add and divide numbers.  Therefore we instead need to ask ‘What is meant by the mean / median / mode?’.

The problem, however, is that if we ask 200 students, we could get 200 different correct answers.  This problem could be reduced by asking this as a multi-choice but without some very well written distractors the answer could be given away.  So… why not turn the question around, i.e. ‘What is the middle value when laid out in order?’.  The answer would then be ‘the median’.  This could be posed as a question that required a typed in answer as it is a single word.  Allowances for spelling errors can be made within Blackboard and as all exam results are checked, anything that may manually require an upgrade can be done.

A final thought – if you do want to test a student’s ability to actually do the maths, then don’t go for multi-choice.  Ask it as a calculated numeric.  Don’t just think question types… think questions… then types!

If you’d like to know more about this, please get in touch with me – Ian Hallsworth

2 thoughts on “Don’t just think question types… think questions… then types

  1. Very well written post, and a message that needs to be reinforced over and over again. We need teachers to focus on the desired learning outcome, and then find a way for the technology to meet them, rather than basing the education on the ability of the teacher’s knowledge of the technology.

  2. Cheers Ian, I’m over at the Leek campus and this is really helpful as we’re starting to use Blackboard in a serious way for the first time. It’s easy to get caught up in the technology and forget what is we’re actually trying to assess!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s