Digi Know: Battle of the Forms – Google V Office 365

image

So, back in June Microsoft released Forms in Office 365 for Education, which fulfilled a gap in the Microsoft product range that Google had been offering since 2012. Using forms is a quick and easy way to create a survey or quiz and be able to analyse and evaluate responses almost immediately.

Microsoft are still relatively behind in developing this technology and currently only provide it for education, not the general public or for enterprise. However on the plus side one can fully integrate all Microsoft products now without relying on external sources such as Google Forms or Surveymonkey, this is a big plus where consistency is essential.

Let’s compare the 2 products:

compare

In my experience of using forms (Google) in the classroom, being able to provide a visual alongside a Question has been vital in providing test questions and so Microsoft not offering this simple option in their forms product is a big oversight and something that I hope will be added soon. The possibilities that Forms opens up in a classroom setting, alongside other Office 365 products such as OneNote and the Class Notebook is encouraging to a more dynamic approach providing a synergism between the Tutor and the Student.

I did experience some further frustration the other day when a colleague asked me to share with her a form I had created in Office 365 Forms. After about 5 minutes of clicking (and swearing) I Googled (Oh…the irony) the problem. You cannot have a co-author or collaborate on one form in Office 365 forms, hmmm bit of a problem when collaboration is key to good academic practice!

The ability to Upload Files in Google Forms is, in my opinion, a massive game changer. As a long time user of the Google for Education suite this feature embodies what Google Apps for Education is about and that’s a seamless integration of the products into a student’s user experience.

Come on Microsoft, I’m batting for your team now!

Using videos in forums for an online course’s student formative assessment

yasukotera

Yasu Kotera (left), Wendy David. UDOL Online Counselling. University of Derby Online (UDOL).

Students from around the world were studying on the University of Derby Online course in Counselling Skills. Part of the assessment involves students recording their counselling sessions for tutors to view and feedback on. Course Resources is limited in the way it handles large files, especially videos, so Yasu and Wendy were pleased to use Media Gallery (Kaltura) as a way of sharing video. In this case, students recorded a short video, usually on their phone, and shared it by embedding it into a forum. This allowed tutors and peers to comment on the video.

Capture

In this screenshot, the uploaded video is displayed over the forum which contains a dialogue between the student and tutor.

Yasu comments “I manage the counselling programmes at the University of Derby Online Learning. Our programmes include video assessments, where students record their counselling sessions and the tutors and peers make comments on them. We used to mail those DVDs to evaluate their counselling skills, but as the programme expanded, it just became unfeasible. Then we started to use a file transfer system, but it still took a lot of time to up / download the videos. We had been trying to find a scalable and secure way to conduct the video assessments, and heard about Media Gallery. Media Gallery enables students upload their video easily in a secure way, and the tutors and peers can see and make comments to improve their counselling skills. This is really crucial in the counselling studies online. We are planning to use Media Gallery more in our programmes

TEL me more – December

The Solitude of WinterDecember’s TEL Me More was attended by several people and rather than presentations a general discussion was held.

We talked about media based scenarios which can be used for formative and summative assessment. For example we had a demonstration of the Substance misuse videos that were created for Health and Social Care students by the TEL team.  These videos allow students to explore carefully constructed scenarios that help them build up an understanding of a case study.

We also discussed a range of apps that could be used to support learning and teaching such as OneNote and Sli.do. The TEL team will be promoting a wide range of apps in the New Year, look out for our ‘Appy Mondays campaign.

This month’s TEL Me More also coincided with our first College based TEL Me More session, where Socrative was demonstrated by an academic in the College of Life and Natural Science, who had been using it with her students after seeing it demonstrated at a previous TEL event. Socrative was used to poll the group on a range of TEL related topics as part of the demonstration on how to set it up and was then use to tie in an important question raised by the College Dean as part of Dean’s Question time.

Assessment and Feedback Notes from a Keynote Presentation by Professor Phil Race (25/03/2015)

Here are some selected key points taken from Professor Phil Race’s Keynote presentation on Assessment and Feedback. Note that most of the bulleted points have been taken as they are from the presentation. The rest of the presentation, and more information about Professor Phil Race is found on his website.

Phil identified some of the practices that are still going on in Higher Education learning, teaching and assessment which he thinks are not working anymore hence regards them as ‘madness’. These are:

  • Lecturing in the traditional way – students don’t now expect to write notes, and often don’t come with pens.
  • Hand-writing feedback – lecturers spend hours writing comments on students work but the students take little notice and they get the feedback too late.
  • Marking Essays – reliability of essays as an assessment method is notoriously poor.
  • Marking huge piles of exam scripts – written exams only measure what comes out of pens. It is what’s in heads that needs to be tested.
  • Ignoring the fact that the students can get to some of the best ‘content’ in the world by themselves, at no charge, on their laptops, tablets and mobile phones.

He suggested that instead, staff should focus on helping students to become excellent learners, getting feedback to them in ways that work much better than written comments; and transforming assessment into something which causes learning, not just attempting to measure snapshots of what has been memorised.

Seven factors underpinning successful learning

Phil suggested that staff should be doing the following in order to make learning happen:

  1. Striving to enhance our students’ want to learn;
  2. Helping students to develop ownership of the need to learn;
  3. Keeping students learning by doing, practice, trial-and-error, repetition;
  4. Ensuring students get quick and useful feedback – from us and from each other;
  5. Helping students to make sense of what they learn.
  6. Getting students deepening their learning by coaching other students, explaining things to each other (verbalising).
  7. Allowing students to further deepen their learning by assessing their own learning, and assessing others’ learning – making informed judgements.

Assessment

Is more effective when:

  • It is used to engage students in learning that is productive
  • Feedback is used to actively improve student learning.
  • Students and teachers become responsible partners in learning and assessment.
  • Students are inducted into the assessment practices and cultures of higher education.
  • Assessment for learning is placed at the centre of subject and program design.
  • Assessment for learning is a focus for staff and institutional development.
  • Assessment provides inclusive and trustworthy representation of student achievement.

The five NSS statements in the UK (to bear in mind)

  1. The criteria used in marking have been clear in advance.
  2. Assessment arrangements and marking have been fair.
  3. Feedback on my work has been prompt.
  4. I have received detailed comments on my work.
  5. Feedback on my work has helped me to clarify things I did not understand.

Students’ thoughts about assessments

Student evaluations frequently reveal poor assessment practices that:

  • Lack authenticity and relevance to real world tasks;
  • Make unreasonable demands on students;
  • Are narrow in scope;
  • Have little long-term benefit;
  • Fail to reward genuine effort;
  • Have unclear expectations and assessment criteria;
  • Fail to provide adequate feedback to students;
  • Rely heavily on factual recall rather than on higher-order thinking and problem-solving skills.

Students need assessment literacy among other things in order to succeed in Higher Education. They need to be able to do the following:

  • Make sense of key terms such as criteria, weightings, and level;
  • Handle a variety of assessment methods (e.g. presentations, portfolios, posters, assessed web participation, practicals, vivas etc) and get practice in using them;
  • Be wisely strategic in their behaviours, putting more work into aspects of an assignment with high weightings, interrogating criteria to find out what is really required and so on;
  • Gain clarity on how the assessment regulations work in their University.

Feedback and Feedforward on exams

Even good feedback on coursework does not necessarily prove much help regarding structuring exam answers. Phil suggests having typical exam answers, about 3, typed up (with students’ permission), with track changes comments added as they will be useful for future candidates. The typical answers should be made available to students in advance but not too far in advance.

Feedforward is defined as a strategy that aims to increase the value of feedback to the students by focusing comments not only on past and present, but also on the future – what the student might aim to do, or do differently in the next assignment or assessment if they are to continue to do well or to do better.

How to give feedback to a large group within 24 hours

Feedback should be timely, given to students while it matters to them and in time for them to use it towards further learning, or to receive further assistance. The following is a suggestion of how to give feedback to a large group within 24 hours. The rationale is that most students would still have been doing the work in the last 24 hours before they handed it in such that they would remember what they were doing, what they didn’t do and what they missed out because they couldn’t understand it.

An assessment deadline is set such that students hand in their work during lecture time. E.g. for a 10-11 lecture on a given day, the deadline can be set to 10:03. The lecturer hands out to the students a coloured sheet containing numbered points. Then when giving feedback, students can be referred to a particular point, e.g. ‘please see point 3, blue sheet’, etc. The coloured sheet can contain illustration of what is expected as evidence of achievement of each of the intended learning outcomes; likely mistakes; features of a good answer; frequently needed explanations; and things that the lecturer would have to write time and time again on students’ work. The students are given about 3 minutes to study the points and then the lecturer spends a few de-briefing the whole group and talking them through one point on the handout. Such feedback contains tone-of-voice, facial expression, body language, emphasis, etc. which lack in written feedback. The lecturer then takes the students’ work away to mark in less time with comments now relating more to each individual piece of work. When students get their marked work back with feedback, they are more likely to use it as it’s personal.

feedback

Adapted from Professor Phil Race’s Presentation

Feedback without marks

It is perceived that marks destroy the value of feedback to students because students may be blinded by the mark or grade they get and may not try to make sense of the feedback they receive as a result. If the mark is good, they rejoice and file the work without reading the feedback, but if it’s low they frown and bin the work. Phil suggests a way round it as giving students their work with feedback but with no marks or grades while keeping a record of their marks. Then the students are asked to work out their marks from the feedback they were given, and also from the feedback others were given. They are advised that if their self-assessment scores are within (say) 5% of their own scores, the higher number will go forward into their assessment record (this will motivate the students to assess themselves, and also that the lecturer will talk individually to those students whose score is different by more than 5% from the one they are given. The lecturer then collects the students’ marks/grades and arranges to talk to the students whose marks are more than 5%. It was observed that most students, about 9 in 10 will be within the 5%.

Students who under-estimate their grade will need their self-esteem boosted and also to be reminded about the assessment criteria, and how these illustrate the intended standards associated with the learning outcomes. Students who over-estimate their grades will require a walk through their work to find out where exactly they lost points they thought they had gained.

Don’t just think question types… think questions… then types

Photo Credit: albertogp123 via Compfight cc

Photo Credit: albertogp123 via Compfight cc

In my role, I see many module leaders present themselves with assessment questions that have been written to fit the question types offered by Blackboard (Course Resources).  In my opinion this does not always lead to effective assessment as what we are presenting to students is a collection of questions that Blackboard can handle rather than an assessment.  For example I recently saw 3 questions that collectively was supposed to ask about the students’ knowledge of the statistical averages.  The question asked was ‘What is the mean / medium / mode of the following numbers?’.  This was followed by a range of numbers and the student could pick a number from a selection presented.

This had been written after whoever wrote it had seen the question types and rather than thinking ‘what do I need to assess?’ (i.e. the students’ ability to differentiate between the different statistical averages), just simply written some simple maths questions which could be guessed at.

When I spoke to the academic in question, they informed me that the intention was to assess the student’s ability to understand and differentiate the different averages and had thought that multi-choice was the way to do this.  They actually wanted a question that asked about the different types but thought that as multi choice would make it too easy as it could possibly give the answer, instead went for a maths question as they thought that some calculation would be needed.

Going back to the original learning outcome (the ability to differentiate between the different statistical averages), I asked what would be a successful learning outcome – the answer being ‘the median being the middle value when laid out in order, the mode being the most common number and the mean being the result of summing all the numbers divided by the number of test results’.

From this we can see that there are three terms that the student needs to learn, rather than being able to do the maths.  So, straight away, there is a requirement to ask what the terms mean rather than if a student can add and divide numbers.  Therefore we instead need to ask ‘What is meant by the mean / median / mode?’.

The problem, however, is that if we ask 200 students, we could get 200 different correct answers.  This problem could be reduced by asking this as a multi-choice but without some very well written distractors the answer could be given away.  So… why not turn the question around, i.e. ‘What is the middle value when laid out in order?’.  The answer would then be ‘the median’.  This could be posed as a question that required a typed in answer as it is a single word.  Allowances for spelling errors can be made within Blackboard and as all exam results are checked, anything that may manually require an upgrade can be done.

A final thought – if you do want to test a student’s ability to actually do the maths, then don’t go for multi-choice.  Ask it as a calculated numeric.  Don’t just think question types… think questions… then types!

If you’d like to know more about this, please get in touch with me – Ian Hallsworth

Promoting student creativity in group video assignments

Charles Hancock

Charles Hancock, Senior Lecturer, Derby Business School

What was done?

During the module students were required to research a company and use these findings for their assessments, a group presentation and an individual report.

In previous iterations of the module students were required to do their presentation face-to-face to the rest of their class and the tutor. However, it was decided to change this to a group video presentation, where students could choose the technology they used to create the video.

Why was it done?

This was done in order to:

  • Engage students in the subject
  • Encourage creativity
  • Improve digital literacy skills
  • Increase confidence
  • Improve skills for employability

How was it done?

Designing the assessment

The assessments for the module were split into two parts, with the second part building upon the work done in the first. The first part of the assessment required the students to work in groups to create a group video presentation, outlining the research students carried out on the ethics and social responsibility of local Derby companies. The information gathered during the research could then be used again to form the basis of the individual report submitted for the second assessment. The marking criteria for the assessment was also designed to focus 10% of the marks on the style of the presentation and 90% on the subject matter including how they talk about and evaluate the findings of their research.

Preparing the students

Guidance was given to the students during face-to-face sessions which broke down the assessments aims and objectives, how they could structure their presentation, what they should include and a clear idea of how the work would be marked. The advice around the technology was left quite open so students could use a variety of tools, as long as they submitted the video in the correct format. By not being prescriptive with these instructions, the students were challenged to develop their skills with using technology and work together to find a technology which met their needs and those of the assessment.

Creating the videos

Students used a range of creative and innovative approaches to create their videos; using video cameras, mobile phones, microphones, text-to-speech tools, images, sound, text and video editing software. They utilised these techniques to record interviews, discuss complex models and outline the findings of their research helping the intended audience to retain the information they presented.

Submitting and marking the work

The students were required to submit the video presentation as a Windows Movie File on a DVD. This meant that not only could the academic make sure that they could view the work to mark it, but it could also be viewed by the external examiner for the module. Despite the variety of presentation styles used in the videos, the marking of them was made easier through the criteria concentrating on the content of the presentation rather than the style or technology used.

What worked well?

  • Allowing students to choose which technology they used to create the video presentation helped them to demonstrate their creativity.
  • The assessment design helped them to learn independently not only about the subject area but also how to use the technology available to them.
  • Students were engaged by the assessment and enjoyed taking part in the project, with grades improved compared to the previous method of stand up and talk presentations.
  • Students with English as a second language were able to build their confidence in using English, as they could re-record their voices or utilise other technologies like text-to-speech tools.
  • Provides them with an example to show employers their ability to get messages across to the work force simply and quickly. These transferable skills are particularly beneficial in today’s workplace especially for national and multinational companies.

What were the challenges?

  • Building students’ confidence to do something different – using technology to create a video may be something they are unfamiliar with.
  • Ensuring that other academic staff are on board and comfortable with how the work will be assessed and marked.

What could be done differently next time?

  • To set out more guidance in the sessions before hand to help students space out the workload so that the pressure is not all at the back end of the assessment.
  • To find out from students how familiar they are with creating videos before the project and look at how the groups can be organised to take this into account.

What resources would other people find useful?

  • Clear set of marking criteria and guidelines for you and your students.
  • Examples of the types of video they might create.
  • Information about how to access resources and equipment for recording and editing the videos.

eAssessment Scotland 2011

Statue of penguins in Dundee

On the 25th and 26th of August I attended the eAssessment Scotland conference in Dundee.  A sound-byte repeated several times during the conference was that we should “assess in the same way that students learn”.  This was partly taken to mean the inclusion of social media and electronic devices that the stereo-typical student immerses themselves in.  However there was a more concrete issue raised both by one of the posters (voted best at the conference) and in Donald Clark’s typically boisterous keynote: students (along with everyone else) are used to the benefits of word processors, which allow creating drafts, re-editing, copy and pasting to rearrange the structure etc.  Expecting students to go into an exam, take-up the seldom used pen and paper, and write essays from start to finish in one go, simply won’t get the best out of them.

Continue reading