Technology, learning and assessment
Personalisation of the learning environment is expected to increase the possibilities for a learner-centred approach to become the prevailing pedagogy. There will also be a growing emphasis on the student taking of responsibility for own learning, building on a social-constructivist theoretical underpinning. Consistent with this will also be an increasing use of e-portfolios for formative self-assessment and self-diagnosis and emerging social software tools on the Internet, such as weblogs, podcasting and wikis.
It has even been suggested that ICT offers the potential for a nonlinear approach to navigating through information and thus different routes through, and forms of, learning. Alongside these changes are likely developments of mobile phone technology and PDAs to facilitate more integration and 'just for me' information for students. This has implications for students with disabilities. For example, the use mobile technologies such as SMS messages is quoted as a major breakthrough for those with hearing impairments.
An example of empowering students and personalising their learning, which describes an initiative at the University of Napier's School of Computing. The description also includes evidence of improved academic performance, increased levels of attendance and greater participation.
Some commentators have suggested that assessment procedures in particular need to be aligned to a new set of student realities. These realities refer to the comfort, or lack of it, associated with the digital age. Those who have grown up in such an environment are called 'digital natives', while those of earlier generations, who have had to learn to live with the consequences of digitisation, are 'digital immigrants'. It is highly likely that the majority of today's students are 'natives' while the majority of staff, in contrast, are immigrants. Bringing those two approaches together is a challenge and it may be that CAA and other innovations are the loci for such a development.
Although the majority of students may be 'digital natives', there remains a minority who are not, and indeed there are increasing numbers entering HE who come from non digital backgrounds, for whom assumptions about learning through and with computers is quite alien and more than a little intimidating. It is suggested that CAA must proceed cautiously in such circumstances: assessment is stressful enough as it is, without another layer being added through the delivery of the assessment itself.
The Integrative Assessment Theme examined the 'backwash' effect of assessment, that is to say the impact upon learning produced by a particular method of testing. This notion was illustrated using research evidence that showed how multiple choice tests (MCQs) encouraged a surface approach to learning, while essays encouraged a deeper one. In a further twist it was shown that those students who took a deep approach to learning for the MCQ actually performed less well than those who had taken a surface approach.
Alongside these technological developments, a likely result will be the demise of the lecture as a staple of learning. In its place will be VLEs which allow repeated attempts at mastering the same information, somewhat in the same manner of the hierarchies of increasing difficulty in a PC game. VLEs will therefore shift from being regarded as an adornment to being acknowledged as one of the foundations of core business. Furthermore, VLE development is largely student-led rather than being driven by policy, though staff champions, strategic aim and technology itself are also cited as drivers. Read more about defining flexible programme delivery.
The extensive and developing use of VLEs well beyond content is mentioned in the Enhancement literature. But flexible programmes of content-based approaches are also becoming more sophisticated (e.g. use of video streaming parts of lectures and critical incidents, digitised materials from HERON; simulations and virtual experiences of labs and field trips etc). Altogether there is a need for, an increasing supply of flexible learning spaces.
The University of Dundee is actively working towards the integration of its VLE (Blackboard) with CAA (Questionmark Perception). This enables students to use only one log in, making the system seamless for the users. Few, if any commentators suggest that ICT, including VLEs, will transform learners and learning. VLEs are often little more than content repositories and, more worryingly it is argued that most of the use is not pedagogically informed and lacks underpinning theory.
The University of Dundee has clear policy and procedures to ensure quality assurance. It is designed to ensure that the university's systems and operations are compliant with the British Standard BS 7988:202 - Code of Practice for the use of information technology (IT) in the delivery of assessments. The introduction and running of CAA at Dundee (including staff development and training) and a useful summation of the pros and cons of CAA.
An indication of the costs involved in setting up and running CAA and further discussion on this topic, along with the notion that we are still living in a first generation environment.
In addition to the implications for students, there are also implications for staff. There is mounting evidence that electronic interaction is just as consuming of academic time - or more so - though less visible than conventional contact. In general, it is more individually targeted and more likely to lead to enhanced learning outcomes for students, but not obviously to increased cost-effectiveness. The time spent by staff in participating in online discussions or responding to emails needs to be recognised and formally timetabled. Currently, many staff are working from home in their own time to carry out these activities.
There is a suggestion in one of the Themes that technologies are beginning to have a an impact on the very nature and structure of organisations and the ways they in which they are structured and run. As a result there is a need for more investment and the inevitable rise in costs needs to be recognised by both senior management and by the Funding Councils.
The following are cross referenced from Making assessment more efficient and effective.
- Use of technology to improve individualised feedback on assessment.
- Considerable potential for effective and efficient assessment through the use of increasingly sophisticated e-assessment tools, students take control of the assessment process through computerised tests - effective interactive teaching tools.
- Interesting references to the introduction of innovative assessment methods.
- Has teaching caught up with new styles of assessment and, conversely, has assessment caught up with new styles of teaching e.g. PBL.
- Strategies for inclusivity in assessment i.e. for the 'new student'
- Use of technology in student support, email and text - pros and cons and use at Wolverhampton for assessment feedback. See also Case Study 1 in the Personalisation strand of The First Year Enhancement Theme.
Computer Aided Assessment (CAA) was examined in the Assessment Theme. It was argued that academics may wish to use CAA to:
- Increase the frequency of assessment, thereby motivating students to learn and encourage students to practice skills. Broaden the range of knowledge assessed. Increase feedback to students and lecturers. Extend the range of assessment methods. Increase objectivity and consistency. Decrease marking loads and aid administrative efficiency. Bull and McKenna (2004) in Reflection on Assessment 2.
- On the other hand there are issues surrounding CAA, centring on Student Acceptance, Practice, Disability, Equity, the sources of questions, quality assurance and security, including issues of plagiarism. Further information about some important quality assurance recommendations provided by McKenna and Bull (2000).
- Suggested models for institutional computer based assessment and an evaluation of the two methods described. One aspect of CAA that was explored in Reflection on Assessment 2 was the way in which its definition could and should be extended to include administrative and management aspects of assessments. In particular, CAA offers staff the opportunity to analyse the students' results; the examinations' validity and overall teaching effectiveness.
As part of the G21C Enhancement Theme, Heriot-Watt University created and evaluated a web portal for academics and students which allows sharing of learning and teaching experiences, resources and practices. An easy to use, keyword driven interface allows staff and students to post tips and experiences in learning and teaching and to look for help from others' postings. This system is also used to direct people to the information and resources already available on the internet, thereby, avoiding repetition of content.
In 2007 The Edinburgh College of Art introduced a new assessment and feedback process. In support of this process the College has designed and implemented a bespoke Portal and Learning Management System (LMS) which directly supports a studio-based, project-led pedagogy.
In addition to providing the digital tools commonly found in many VLEs, the LMS provides a rich development environment for staff to construct their projects / themes of study, lectures and seminars and enables these to be linked directly to their students with the associated framework for managing crits / tutorials, feedback and graded assessment. Key features of the system require students to reflect upon their learning, write their own feedback, self-evaluate and grade their own work.
Staff provide written feedback, graded formative assessment and actions in addition to validating the students own written tutorial reports and required actions to ensure that there is no misunderstanding between tutor and student regarding the progress being made. As the academic year progresses students see their formative graded profile of achievement developing along with staff feedback and actions points, complemented by their own reflections and responses. A research project has been initiated to evaluate its impact.
In The Robert Gordon School of Computing, audio and visual recordings are being used to help students understand how to solve problems. These 'Screencasts' (audio and visual recordings) are used within an introductory programming module taught to all first-year computing undergraduates. Only a minority of students have previous exposure to programming at the start of the module.
In light of this inexperience, the teaching team wanted to provide students with access to a resource which would help deepen their learning and help build confidence in solving problems early on. The activity involves the video-capture of solutions to tutorial and lab exercises with the aim of conveying the process of finding a solution, rather than just delivering content.
The teaching team feel that video is able to convey the dynamic element of solutions, while accompanying audio commentary from the lecturer provides students with access to the problem-solving approach of a skilled tutor.