Assessment “edu-speak” for Engineers 	
	Electrical and Electronic Engineering Assessment Network (e3an)	
	http://www.e3an.ac.uk/	
	University of Southampton	
Author:	Su White	
	saw@ecs.soton.ac.uk	
Partner
Details	University of Southampton 
Department of Electronics and Computer Science	
Peter Asburn, Hugh Davis, Richard Crowder, Chris Havill, Kate Dickens 	

Bournemouth University 
School of Design, Engineering & Computing	
Jim Roach, Chris Keenan	

University of Portsmouth
Department of Electrical and Electronic Engineering	
Tim Morgan, Djamel Azzi	

Southampton Institute
Faculty of Technology	
Sean Wellington	

An engineer’s guide to educspeak for assessment
Or how to bluff your way in eduspeak….
Contents
Assessment “edu-speak” for Engineers
An engineer’s guide to educspeak for assessment

Contents

•	Introduction
•	Do we need to change our approaches to assessment?
•	Health Warning
•	Levels of Learning
•	Blooms Taxonomy of Levels of Learning
•	Examples of Assessment methods
•	Quick overview of Objective Tests
•	Question analysis exercise
•	Readings references
•	Acknowledgements
•	What assessment methods do you use?
•	Assessment evaluation:

Introduction
No engineer who teaches their subject is immune from the language of eduspeak.  Assuming that you have already spent some of your precious moments engaged in the delights of writing course handbooks, programme specifications and module specifications you are already some way to knowing the language of eduspeak for engineers.  If you have strayed into the worlds of subject benchmarking (according either to the QAA or the engineering Professors) then you are probably a near expert….Add to that the fact that you were sufficiently interested in the subject to think about contributing to our test bank and we are almost imagining that there is no need for you to read this at all 
Do we need to change our approaches to assessment?
Any discussion of approaches to assessment is inevitably accompanied by some debate around existing assessment practices.  The QAA engineering benchmark group identified 18 different methods of assessment which they might expect to find being used.  
•	
•	Unseen examinations
•	Laboratory examinations
•	Oral examinations
•	Synoptic examination
•	Multiple-choice tests
•	Problem solving exercises
•	Essay assignments
•	Other types of extended writing
•	Oral presentations
•	Student-led seminars/discussions
•	Design tasks
•	Computer-based exercises
•	Work placement reports
•	Learning logs/diaries
•	Learning portfolios
•	Exhibition/poster displays
•	Groups projects
•	Independent projects

However for many of us, we use only a  few types of assessment and in a fairly limited way.  
The subject reviews of most academic areas called for more “formative” assessment for students – that is, assessment which provides the learner (and the teacher) with some feedback on how the learning process if progressing, and how effectively learning is taking place.  The criticism was that too much assessment was “summative” that is taking place at the end of a course, and being used to judge and measure the learner’s progress, rather than helping them along their way.  
Some other reasons that we might wish to change assessment methods include the fact that we need to get a full and round measurement of learners’ achievements, and using a limited range of assessment methods to measure achievement may return an unbalanced picture of the actual skills and knowledge acquired.  Some methods of assessment may give different students different opportunities to demonstrate their skills.  We may wish to change assessment methods because we want to reduce the workload, or hand over some of the workload (and incidentally additional opportunities for learning)  to the students.  
Health Warning
This guide is not in any way designed to be comprehensive, rather it is designed to be a good enough guide to education in the context of the work we have been doing in the development of test banks.  Rather than looking at education as a whole it is focussing on issues to do with assessment.  Some of our references are generic, others are specific to engineering, some are even specific to electrical and electronic engineering. 

Levels of Learning
When we are thinking about measuring the extent of a student’s learning we can use a scale of complexity first devised by a team of psychologists who looked at student learning in 1950’s America, known as Bloom’s Taxonomy (Bloom et al 1956)
According to Bloom's Taxonomy, different levels of learning can be classified as follows moving from the simplest to the most advanced as you read down the table:

Blooms Taxonomy of Levels of Learning	
Levels of Learning
At which students are required to demonstrate their ability	Associated action verbs	
Knowledge
to recall a wide range of knowledge, facts and experiences	describe, recall, define, state, recognise, name, list, underline, reproduce, measure, write, label, identify, acquire	
Comprehension
to grasp the meaning of their acquired knowledge, and to process, translate and interpret this knowledge	Comprehend, understand, draw, interpolate, extrapolate, predict, to have insight into, translate, illustrate	
Application
to apply knowledge and comprehension in different situations, relate material, and infer from facts	apply, show, demonstrate, perform, use, relate, develop, transfer, infer, construct, explain	
Analysis
to analyse data or material, breaking it down into its component parts so that its organisational structure may be understood	analyse, identify, separate, detect, break down, discriminate, categorise, distinguish	
Synthesis
to synthesise and combine elements to produce a coherent whole and make logical deductions	combine, restate, summarise, précis, generalise, conclude, derive, organise, design, deduce, classify, formulate, propose	
Evaluation
to evaluate data, make judgements and assess material	evaluate, judge, decide, choose, assess, contrast, criticise, select, defend support, attack, seek out, compare, determine.  	
You may be able to associate the verbs with activities which you describe in learning outcomes.  
Taken from Setting effective objective tests CTI Land Studies 1997 p8, adapted and based on Bloom's Taxonomy of Educational Objectives, Cognitive Domain 1956, adapted by Davies, 1 K (1976). Objectives in Curriculum Design. McGraw Hill (UK) Ltd, Maidenhead pp 150-153 
Q To what extent do you think these are workable and useful definitions?

Examples of Assessment methods
As a way of helping you look at assessment in a different way we have included extracts from a survey of assessment practices in Scottish Higher Education.  The examples on the following pages were drawn from the ASSHE project database.  The database was first established in 1996 so some information on the activities may be out of date, however the principles are still of interest.  
Quick overview of Objective Tests
What are objective tests?
An objective test is a test in which each question has an unequivocally correct answer or set of answers
Examples of types of objective test include:
•	Multiple Choice Question
•	Multiple response question
•	Ranking/Relationship
•	True False
•	Gap Filling
•	Hot Spot
•	Numerical Question
What levels of assessment are possible with objective tests?
According to Hearn et al (1997) it is possible to use objective tests in the following levels of learning:
•	Knowledge
•	Comprehension
•	Application
•	Analysis
Question analysis exercise
Take a look at some of your own questions exam, or stage test or objective test
For each question identify its topic within the subject area and then analyse which level of learning in Bloom's Taxonomy the question addresses.  Give the question a rough and ready rating on a scale of 0-3 hopeless to useful.  Comment on the question, as you see appropriate
Question	Topic	Knowledge	Comprehension	Application	Analysis	Rating	
1							
Comment:

2							
Comment:

3							
Comment:

Readings references
ASSHE  examples of assessment in engineering
http://www.ltsn.ac.uk/genericcentre/projects/assessment/asshe/default.asp
CAAcentre
no longer funded, but web site is archived at Loughborough, and has a good set of resources and links
http://caacentre.lboro.ac.uk/
Centre For Interactive Assessment Development
http://www.derby.ac.uk/ciad/dev/qdesign.html
Computer Assisted Assessment in Higher Education, Sally Brown, Phil Race and Joanna Bull, 1999, Kogan Page
This has a number of useful and interesting articles based on practitioners' experience.  
http://www.kogan-page.co.uk/asp/bookdetails.asp?key=1546 
Computer Assisted Assessment: using computers to design and deliver objective tests Eamonn Twomey, Jacqui Nicol, and Christina Smart, CTISS 1999
Part of the CTISS "Primer" series, this three page document provides an accessible overview of objective testing.  It has been updated by Martin Poulner from LTSN Economics and a revised version can be viewed at http://www.economics.ltsn.ac.uk/advice/assess.htm
A copy of the original can be found on our project website at  http://www.e3an.ac.uk/archive/docs/assess.pdf 
Constructing Achievement Tests, Norman E Gronlund, 1968, Prentice Hall
As well as giving extensive examples of objective tests on knowledge, this book contains a chapter on construcing onjective tests of complex achievement (comprehension, application, synthesis and evaluation, plus complex item types).  Again worth looking at if you are really interested in this area, you should be able to get a copy via your University Library.  
Designing Effective Objective Tests: An Introductory Workshop.  Coleen McKenna and Joanna Bull 
This useful guide is a free 15 page PDF document which takes the user through a series of worked examples to understand the methods of constructing objective tests.  Examples are drawn from a range of arts and science contexts, but there are no examples specific to engineering
It is one of the resources linked from http://caacentre.lboro.ac.uk/ 
Innovative Assessment by Graham Mohl
http://www.lgu.ac.uk/deliberations/assessment/mowl_fr.html 
It is not clear how long these will continue to be available, the deliberations site is no longer maintained.  
LTSN generic centre have assessment as a special focus area
http://www.ltsn.ac.uk/genericcentre/projects/assessment/
they have produced a useful set of guides which may be ordered or downloaded, as well as providing a range of links to sites, publications and resources.  
Setting effective objective tests. Sue Heard, Jacqui Nicol and Simon Heath, University of Aberdeen 1997.  
This can be ordered for £20 - which is quite steep for the size of the publication, but it does offer an alternative basis to work from.  Examples in the guide are predominantly drawn from the sciences.  A description of the tutorial and details of how to order are available from http://www.clues.abdn.ac.uk:8080/caa/caatut_idx.html 
Student Assessment in Higher Education  A Handbook for Assessing Performance Authors : Kevin Cox, Allen Miller and Bradford Imrie 1998 Kogan Page
Combined educational theory with examples of actual approaches to assessment.  
http://www.kogan-page.co.uk/asp/bookdetails.asp?key=1365 
Taxonomy of Education Objectives 1: Cognitive Domain B S Bloom, et al 1956, Longman.
Includes examples of test and exam questions which are designed to assess particular levels of learning.  Worth looking at if you are really interested in this area, you should be able to get a copy via your University Library.
The Art Of Assessing  Phil Race
http://www.lgu.ac.uk/deliberations/assessment/artof_fr.html
Acknowledgements
Parts of this guide were based around material in the University of Aberdeen Guide "Setting effective objective tests"
Tests and answers available online from University of Southern Australia
http://www.unisanet.unisa.edu.au/10496/midsem.htm

What assessment methods do you use?
What Assessment Methods do you use?  
What different forms or methods of assessment do you currently use in your teaching areas? 
What works really well, are there any problem areas? 
Are there any new methods of assessment you plan to introduce over the next academic year or two?
What could you use? Individually, or for your teaching degree area - complete the information in the grid below
The benchmarking group for engineering suggested the mix of assessments used in the subject discipline could cover.  Review the list, and identify how the benchmarking group's descriptions map to current practice in your own institution- indicate where the method is restricted to a particular unit, course, or year.  
Type of Assessment	what you call it	which courses, how administered	No. of hours 
+ % of total assessment	
Unseen examinations			 	
Laboratory examinations			 	
Oral examinations			 	
Synoptic examination			 	
Multiple-choice tests			 	
Problem solving exercises			 	
Essay assignments			 	
Other types of extended writing			 	
Oral presentations			 	
Student-led seminars/discussions			 	
Design tasks			 	
Computer-based exercises			 	
Work placement reports			 	
Learning logs/diaries			 	
Learning portfolios			 	
Exhibition/poster displays			 	
Groups projects			 	
Independent projects			 	
Other++				
				
Any comments on the process?

	
Assessment evaluation:
For each method of assessment which you currently use, identify those levels of learning attainment for which you currently assess your students.
Use the list of action verbs associated with each level to match with the wordings you use in your assessments, and the activities, and learning outcomes you expect from the students.  
Grade each learning level from zero to three to indicate the relative emphasis you give to each of the levels
0
no emphasis placed on assessment at this level	1
some emphasis placed on assessment at this level	2
quite strong emphasis placed on assessment at this level	3
very strong emphasis placed on assessment at this level	
 
Assessment method	y/n	Knowledge	comprehension	application	Analysis	synthesis	evaluation	
Unseen exams								
Laboratory exams								
Oral exam								
Synoptic exams								
Multiple-choice tests								
Problem solving exercises								
Essay assignments								
Other types of extended writing								
Oral presentations								
Student-led seminars/discussions								
Design tasks								
Computer-based exercises								
Work placement reports								
Learning logs/diaries								
Learning portfolios								
Exhibition/poster displays								
Groups projects								
Independent projects								
Other ++