Educational technologies have enhanced the process of collecting evidence of learning for making an assessment decision and also as feedback to the learner on their own learning. There are a variety of ways we can implement eAssessment and it's not just your typical online quiz. Think about the following examples and how technology has made the process of gathering evidence smarter and more efficient.
1. A building apprentice uploads photos over time demonstrating his construction of a support structure (brick wall) using his smartphone and the application foursquare. To assist in authentication, the phone records his location and time, automatically "tagging" this data to the photos.
2. An assessor conducts interviews with a learner, his colleagues and her boss using a smartphone's inbuilt microphone and an application called Evernote. These interviews are uploaded into a cloud-synced 'note'. Photographic and text evidence is added to the same notebook. Later, access to this online notebook of evidence is shared with the learner who continues to upload documents, notes and photo evidence to support her case for RPL.
3. A trainer lends out a Tablet device to a trainee to support learning in the workplace. The mobile device is pre-loaded with eBooks, Videos and Podcasts that cover the input for a particular unit. The learner completes written assessments in 'Pages' documents, which are later synced through iCloud (Apple's Cloud Service) for marking.
The gathering of evidence through technology can be unnecessary with no real benefit to the learner or assessment process. You need to way up the benefits you're seeking with the investment needed in setting up the technologies, staff training and managing the process overtime. There are some benefits to collecting Evidence Data with technology I have witnessed in practice including;
- Faster Collection
- Efficient Storage
- Multiple Access points
- Analytics from Evidence Data
- Rich Media Evidence (video, photo, audio, text)
- Authentication Technologies
Why is it that so many online courses resort to multiple choice quizzes as the main form of assessment? Organization's I've worked with can lose site of the original purpose and quality of their assessment strategy when moving to a digital strategy. A typical situation is that we want to cut down on travel time and expenses in delivering internal PD programs for staff. So we implement an LMS with self-paced content and assessment as an online PD solution. The obvious question is asked 'How do we know that they learnt anything online?'. Whenever I hear this statement I always ask 'How did you know they learnt anything before you went online?' because if you don't have methods of measuring outcomes of training now, why will it be any different online. The answer is usually that the trainer could see that the information presented has been understood through group discussion, attention of the participants and a pop quiz at the end of the session.
That only the pop quiz, and usually in the form of a simple multiple-choice quiz, is used for online assessment comes down to a lack of understanding on the range of assessments available in digital spaces. The limitations of technology, real or perceived can lead to decisions being made that compromise on the original assessment strategy. Sometimes a blended approach is required using face-to-face training and assessment with online assessments, but the pressure to put it ALL ONLINE can leave you watering down the rigor and quality of assessment tasks. If it's simply recall of facts your testing then multiple choice might be valid, but if you require higher order thinking then you may require other approaches. A very useful framework for deciding what are the best digital tools for assessment is to think about the following questions;
- What type of thinking (BLOOMS) is involved ? (Apply, Create, Understand)
- Do you require learners to collaborate on the assessment?
- What is the context in which the assessment will be conducted?
- What technology will be required by the learner to conduct the assessment?
- Does the assessment require self-paced, immediate feedback, real-time communication?
For a very good breakdown of a tools versus pedagogy framework visit the Moodle Tool Guide for Teachers by Joyce Seitzinger *can apply to any LMS or technology not just Moodle.
We have a client who delivers a course to multiple classes across thirty different high schools in Queensland. It wasn't until they implemented a new E-learning strategy for gathering evidence to a central system that they could begin to analyze the 'big picture' data of what was happening with their assessment strategy. From reports generated using the LMS they can now collect data on Activity Completion, Marking Methods, Engagement Levels. This data is used to make informed decisions then on how to best structure their courses and with which Schools, Teachers and students might need better support or even a customized solution.
How do we know that it was learner X who did the test if they weren't sitting in front of me when they did it?. How about we photograph, geotag, card swipe and eye-scan our online learners so we know they aren't cheating? We don't seem to trust learners anymore. I am reluctant to promote the use of emerging authentication technologies when there are many privacy issues surrounding them and when we seem to have become over zealous with the technology available to us to better identify our distance learners. Particularly when we used to, and still do in many other areas of life, trust a person's word and pen signature.
There is the assumption that authentication technologies and plagerism tools will solve the problem. Every technology can be cheated. Even a webcam on a learner in a timed exam doesn't stop them being fed information from an outside source.
So high risk industries aside, think of an authentication policy that does least harm and incorporates tried and true methods without technology and maybe a couple with technology.
Authentication with technology:
I am here: Location Data
I did when: Time Stamp
I look/sound like: Visual/Audio Identifiers
I am with: Third Party
I have signed: Digital Signatures
I promise: Declarations
I logged in: Passwords
I am me: BioTechnology
Authentication without technology:
Quality Assessments (Valid)
Relationship with the learner
eAssessment as Digital Content
Your eAssessment should look as good as your content. In a very real sense eAssessment is still content that happens to have some action required by the user, and it should be as elegantly designed as your content in every way. Too often we see a two step process of rich, engaging and well designed content followed by dull, boring, drudge level assessments.
- Assessment is a critical and often anxious moment in the learner's journey that needs clarity ie. the instructions are clear and the mechanism works.
- There is often less support for the learner during eAssessments but there doesn't have to be, think of the three modes of communication and how you can support the learner with your Static, Over-Time and Real-Time support.
- Consistency of Style through the full learner experience
- Helping learners recognise content threads
- Repeating engaging training components during assessment
If our learners are used to receiving content as a formal part of their learning, then where your assessment sits amongst your content is an important decision, again often overlooked. Remember that digital assessment is still digital content and needs to hold the quality and consistency of your overall design methodology. We can also make important decisions about how we position assessment tasks through our design of eAssessment and this can make all the difference for the learners experience. Assessment doesn't have to come at the tail end of content. eAssessment both formative and summative can be positioned before, after, amongst or even integral to the content experience. The proximity of content to assessment needs rethinking in the digital space as there are more opportunities to provided a learner with direct feedback, corrections and extension exercises through the fluid and connected nature of hyperlinked content.
- Think about what your current assessment strategy is and what makes it good without technology?
- Remember: eAssessments is in essence assessment that is conducted through or enhanced by digital technologies. The digital technologies help you collect evidence of a learner's abilities.
- Quick Tips for Online Assessments
- Give clear instructions
- Explain: How long, How much, What is the first step
- Have example/samples of completed assessments.
- Have a clear link back to the Content
- Consider Assessment before OR integrated to content
- Give adequate and timely feedback.
- Reinforce Completion/ Or Incompletion.
- Return to Gaps in Learning.
- Give Actions Steps, How do I begin? Where to Now?