eAssessment Scotland 2011

Statue of penguins in Dundee

On the 25th and 26th of August I attended the eAssessment Scotland conference in Dundee.  A sound-byte repeated several times during the conference was that we should “assess in the same way that students learn”.  This was partly taken to mean the inclusion of social media and electronic devices that the stereo-typical student immerses themselves in.  However there was a more concrete issue raised both by one of the posters (voted best at the conference) and in Donald Clark’s typically boisterous keynote: students (along with everyone else) are used to the benefits of word processors, which allow creating drafts, re-editing, copy and pasting to rearrange the structure etc.  Expecting students to go into an exam, take-up the seldom used pen and paper, and write essays from start to finish in one go, simply won’t get the best out of them.

Assessing students in ways that they only even have, and only ever will use, in exams, also goes against another increasingly mentioned principle: authentic assessment.  For me this is the more important point.  As well as assessing in the way that students learn, we should be assessing in the way that current practitioners work.  The aim is to recover some of the benefits of the apprentice systems – students working on authentic tasks under guidance.  When education was industrialised the high ratio of students to teachers made this approach unfeasible; but current technologies promise to make it a realistic option once again.  For example technologies such as ePortfolios can make large volumes of student materials more manageable, while simulations allow students to perform tasks in a controlled and individually guided way.  Technologies such as SIMPLE combine the benefits of both.

A presentation by Sharon Flynn was particularly relevant to our University’s new eSubmission strategy, since she discussed the results of using Turnitin in various ways.  Reassuringly our plans matched what she presented as the better model – where students can see and reflect upon their originality reports.  To help students cope she suggested reiterating very clearly what is expected of submissions (paraphrased below):

  • Reading around the subject: some highlighting of ‘un-original’ work is not only expected, it is required.  Students should be gathering together relevant literature; but it should be referenced.
  • Originality: A substantial amount should be original.  It isn’t only about finding the answers and slotting them in to the right places.  Some areas of the report should be completely free of any highlighting, showing that it is original thought.

To avoid making the post too long I’ll briefly summarise a few more interesting presentations below, and I’d be happy to discuss them further with anyone that is interested.

  • Steve Wheeler’s keynote included two interesting systems of assessment.  Ipsative – where students are only measured against their previous achievements – and triadic – where student work goes through a sequence of self, peer, and tutor assessment (with scores averaged).
  • David Hopkins discussed ‘Time-constrained papers’, where students are given a topic and expected to submit within 24 hrs, with collusion mitigated by requesting that students incorporate experiences specific to their own work-places.
  • Pamela Kato presented a serious game to train student doctors on how to manage their own stress levels, with their heart and sweat rates providing inputs to the game.
  • Bill Foster and Christian Perfect presented their NUMBAS system; providing a free, open-source assessment tool, specifically aimed at maths subjects.
  • Donald Clark had scathing criticism for the current systems of education and assessment, including the ‘paucity’ of formative assessment, too frequent summative assessment, how all lectures should be recorded, and how ePortfolios are ‘cul-de-sacs’.

3 thoughts on “eAssessment Scotland 2011

  1. Hi Matthew

    Thank you for sharing this. I am interested in a couple things you included. One is the SIMPLE technology I have looked at the link and do not have the confidence to know if I am able to download this to explore it.

    The second issue is the point that in designing assessments for my students who are a various stages of development in becoming at one stage entering into the world of counselling and psychotherapy through to being very senior practitioners. As such the assessments are designed to help them to develop their knowledge skills and understanding which means being able to observe and critically analyse skills in practice using video demonstrations and in some cases producing and analysing their own videos. It also involves demonstrating their own ability to develop and demonstrate their own self awareness to be able to recognise what belongs to them and what belongs to others, and how they can best make the most of their own skills. They also need to understand the socio political and ethical dimensions of their work.

    So within my subject area there are no exams there
    portfolio because within a professional context they will need to become professionally accredited and the accrediting bodies expect a portfolio. Thus, the assessment reflects the real world and supports their future development. The portfolio contains items including

    – record of clinical supervision provided with a reflective analysis of the processes
    – a record of consultation on their supervision
    – a short assignment examining the legal ethical and professional issues involved in supervision
    – A personal and professional development plan

    There is video analysis either from a video they produce or a set of video clips that are produced by the team and the students then analyse them both from a formative stance using the discussion board to provide feedback on their analysis and a summative where they submit a written analysis.

    There are formal essays which allow the demonstration of theory and research in relation to the subject area.

    So I for once feel confident that on the programmes I lead we assess in a way that is consistent with the needs of professional practice. And although my programmes are not accredited by professional bodies (yet) I am confident that they will be in the future, and having completed a periodic review last semester there was certainly feedback there that what we are doing fits the bill. In terms of assessment and the relationship between the assessment and the learning within both at programme and module level. I hope during the coming year to further enhance this by making far greater use of Skype, Collaborate etc and then sharing this with colleagues and gaining further ideas from them.

  2. Hi Wendy,

    The SIMPLE system would be something that we would need to setup on your behalf. At its most simplistic it allows students to follow the process of professional practices – mainly focussing on the paper-work and communication side – in fictitious, simulated tasks. If your students are already engaging in real-world tasks, then that’s as good as it gets. The simulation could be very useful in allowing students to get an appreciation for a whole process, and managing multiple accounts as part of a team, but it wouldn’t have the detail and ‘messiness’ of the real-world.

    One possible use could be to allow students partake in more high-risk activities in a safe environment, and building joint-activities with students in other disciplines that they might work with during their career. I think any use of a system like this would require quite a lot of re-development in the modules, but it is something that could be very beneficial in the right context.

    I’ll let you know if we setup a demonstration version.

  3. Hello Matthew

    many thanks – from the sound of it there may a place for this – let me know if you can set up a demo. I can imagine that for nursing for example there may be things that would not be possible to demonstrate in the real world – so let me know


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s