This year’s Computer Aided Assessment (CAA) conference was held on the 5th and 6th of July; and I attended to present a paper on our migration from one computer-based assessment system (TRIADS) to another (Blackboard). In this post I will attempt to summarise the main themes, and highlight interesting developments.
Randy Bennett delivered the Keynote address, centred around his ‘meta-proposition’ that “The use of technology in assessment should be substantively driven”; which was supported by 11 propositions – all of which can be viewed on the CAA website (http://caaconference.co.uk/wp-content/uploads/CAA-technology-7-11.pdf ).
Randy’s presentation highlighted the substantial use of eAssessment in the United States, with one report indicating that 72% of state schools provided online assessment. Two states were singled out as having particularly impressive statistics for Primary and Secondary School use, with Oregon delivering 1.5 million online tests per year, and Virginia delivering 2.1 million per year (for 2009-2010). However, central to Randy’s argument was that it was not just a numbers game, and that eAssessment should take full advantage of its potential to offer better, richer assessments.
The two symposiums at the conference started with quite a negative tone, being titled “Have we lost the plot?” and “Could budget cuts signal the end of e-assessment?”. The attendees’ responses also weren’t as animated in defence as you might expect. Although everyone in the room believed in the potential of eAssessment, there were concerns that it was too easy to do badly, and that drives to cut costs could lead to reducing quality. Despite the pressures to be more competitive in difficult times, there was a call to be more cooperative; with suggestions to share the burden of creating level 4 assessments in common fields, and possibly the creation of joint-institution e-assessment centres. Perhaps more realistic was the drive towards interoperable standards such as QTI for tests (http://www.imsglobal.org/question/) and leap2a for e-portfolios (http://www.leapspecs.org/2A/).
Nevertheless, there was a great deal of imaginative and interesting papers presented, a few of which are discussed below.
John Dermo discussed the creation of an assessment used for formative purposes, which provided extensive per-choice feedback. Particularly interesting was how the students’ attitudes towards the formative test (number of times taken, why they used it), had a significant affect on their rate of progress during the course, regardless of actual ability.
Ray Stark discussed the transfer of experience between further and higher education in the use of ePortfolios. As this is an unfamiliar area for me, it was interesting to hear about some of the underlying pedagogy; including general student engagement such as Tinto’s model of attrition and Astin’s theory of involvement, technology adoption through Roger’s diffusion of innovation model, and the underlying concept of “patchwork text”.
From a more technical point of view Margit Hofler’s paper, on the automatic creation of assessment questions from a body of text, was enlightening. The system identified potential sentences and isolated keywords within them. The question could then be text-entry – where the student supplied the subtracted keyword – or the system could generate its own distractors for MCQ-style questions.
All proceedings are freely available on the conference website: http://caaconference.co.uk/proceedings/