Assessment and independent language learning

Author: Ciel Language Support Network

© Ciel Project, 2000

Table of contents

• Introduction: problems and issues
• 1 Four key terms
• 2 Aims and means: assessment methods for independent language learning
• 3 Skills and strategies
• 4 The language portfolio
• 5 Structure
• 6 Linking the portfolio to the final assessment
• 7 Presentation assessment
• 8 General conclusions
• Appendix 1: Case Study 4 - Assessment of progress and independent learning: empirical evidence
• Appendix 2: Giving oral presentations: a quick guide
• Appendix 3: Assessment criteria for oral presentations
• Appendix 4: Sample learning log (Leeds Metropolitan University)
• Appendix 5: Sample language activity worksheet (Leeds Metropolitan University)
• Appendix 6: Case Study 3, figure 1: format of the work programme
• Appendix 7: Case Study 3, figure 2
• Bibliography

Introduction: problems and issues

'The spirit and style of student assessment defines the de facto curriculum'
( Rowntree, 1987 )

This handbook raises issues related to the assessment of independent language learning. It also presents practical examples of current activity in this field through a number of case studies.

It begins with an introduction by Dr Glenn Fulcher of the English Language Institute, University of Surrey, who has contributed in many ways and on many occasions to the CIEL project. Here he outlines some of the problems and issues which face us when trying to assess independent learning in the study of languages.

Assessment of progress and independent learning

Despite the current rapid growth of interest in independent learning among language teachers (see Sinclair, 1999 ), little attention has been paid to the assessment of independent learning. The focus of debates over independent learning take a decided philosophical and ideological tone (see, for example, Pennycook, 1997 ); it is taken for granted that independent learning is "a good thing," releasing learners from the tyranny of the teacher and the classroom. There are few studies of the impact of independent learning programmes over time, and only one in the field of teaching English to my knowledge ( Soo, K-S and Ngeow, Y-H, 1997 ).

The lack of assessment of progress in independent learning is not difficult to explain. Firstly, it is unusual for learners on independent learning programmes to be denied access to teachers entirely, or some form of teacher-fronted classroom contact. Rather, the independent learning centre (or "self-access centre") is used to reduce teacher contact time. This is a shift in programme structure, rather than a radical change. It is therefore difficult to isolate just how much "learning" can be attributed to independent study, and how much to teacher-fronted classes. If ESL learners in the UK are considered, there is also the additional variable of "incidental learning" from the environment. Secondly, it is difficult to design instruments that are sensitive to small gains in language learning, and the types of programmes delivered in Universities are often over relatively short periods of time. If we find it difficult to measure language gain over short periods, it is even more difficult to identify where any measurable gain might originate. Nevertheless, standardised tests that use a large score scale are much more sensitive than performance-based tests that typically only use a small range of "bands" or "levels." The Test of English for International Communication (TOEIC) has a scale score from 10 - 990, for example, while the International English Language Testing Service only offers 9.

To add to these problems, there is the question of whether learning should be assessed in terms of product : what the learner is able to do at the end of a learning programme, or whether learning should be seen in terms of process : how the learner tackles tasks within an independent learning programme. Looking at process may include some evaluation of the "difference" between a starting and finishing point, but may simply be an assessment of the amount of effort, diligence and growth in the "ability to learn" independently. If one takes the latter view, any assessment other than the qualitative and subjective view of the language learning assistant or officer may be unnecessary. But that would only do for non-award bearing programmes.

However, assessment is also important in order to express the element of "value added" that independent learning is claimed to bring. Is it the case that learners make (more) progress when provided with self-access facilities? Only if this question can be answered can we address the issue of "value added" and at the same time understand what it is we are able to meaningfully assess in an award-bearing independent language learning programme.

Glenn Fulcher, University of Surrey
(This introduction continues with a case study focusing on empirical evidence for the value of independent learning and self assessment in Appendix 1)

Why do we assess?

Brown, et al (1994) remind us that it is important as providers of education to step back and consider why we assess. ' There are times when we become so immersed in the job that we lose sight of the exact purpose of a particular element of assessment. There is then the possibility that we are not achieving that purpose, or that we overlook another form of assessment which might be more appropriate.' This last comment is particularly relevant to the assessment of independent learning where there are a range of assessment strategies to choose from.

Some commonly seen reasons why assessment is useful are:

• Motivation
• Creating learning activities
• Feedback to the student, identifying the strengths and weaknesses
• Feedback to the staff on how well the message is getting across
• To judge performance (grade/degree classification)
• Quality assurance- internal and external to the institution

The first four of these listed reasons for assessment will apply frequently, while the last three may apply only on particular occasions.
( Brown et al, 1994 , p.5)

How and what do we assess?

Assessment of independent language learning can appear problematic. A common concern is that when independent language learning takes place, as tutors we do not have control over what is learned or how it is learned. Logically we cannot be sure what a students' language learning outcomes are as a result of this activity. How, therefore, can we accurately assess this learning and compare one learner with another?

We might start by asking the following questions of our assessment procedures and criteria.

1. Is the assessment valid ? - Does the assessment measure what we really want to measure?
2. Is the assessment reliable ? - Is all work being marked to the same standard?
3. In the assessment fair ? - Do students have an equal opportunity to succeed even if their experiences are not identical? It is also important that all assessment processes should be seen to be fair by all students.

1 Four key terms

Assessment is used for different purposes at different points in the learning process. The following are four key terms which appear with some frequency in relevant literature.

1.1 Formative

Formative assessment is a method designed to establish how much progress a student is making during learning with a view to giving feedback to the student ( Atkins, et al, 1993 , p.7). An example is the essay used as graded coursework. Comments written on it by the tutor are intended to help the student learn from mistakes and improve their work ( Gibbs & Habeshaw, 1990 , p.8)

Formative assessment is often referred to as assessment of the process of learning. In the area of independent learning we are very often concerned with this type of assessment. (See section 3.1 )

1.2 Summative

This assessment method is designed to establish what a student has achieved at the end of a unit or course. A final mark or grade is awarded. ( Atkins, et al, 1993, p.7). In contrast with formative assessment i.e. the assessment of process, summative assessment equates with the assessment of the product or outcomes of learning.

1.3 Norm-referenced

'Norm referencing allocates marks according to how well the individual did in relation to other who took the test' ( Gibbs & Habeshaw, 1990 , p.12) Using this method, only a proportion of the students will obtain a particular grade or class of degree because the grades are allocated according to where they fall on the mark distribution curve or where they are place on the list. But this figure is not predicative of the person's ability to undertake any particular task; it only indicates the person's rank in comparison to other individuals ( Gibbs & Habeshaw. 1990 , p.12).

1.4 Criterion-referenced

An 'Assessment system in which students' performance is marked and graded according to (pre-specified) criteria and standards. The criteria need not be restricted to minimum thresholds of competent, acceptable or safe performance: they can also include elements of mastery and excellence. In theory all students could fail to meet the standards set or all could achieve the highest possible grade' ( Atkins, et al, 1993 , p.7)

For an overview of assessment in language learning, the reader might refer to 'Effective Practices in Assessment in the Modern Language - An Interim Report' by Pol O'Dochartaigh and Michael Schmidt of the FDTL Assessment Project from which the above definitions of key terms are taken.

2 Aims and means: assessment methods for independent language learning

2.1 Product and process of learning

With the assessment of independent language learning we are as interested in the process of the development of learning skills and strategies as we are in the language learning product or outcomes. Assessing learning outcomes (summative assessment) typically takes place through end-of-year examinations. But how are we to go about identifying and assessing a process (formative assessment)?

To answer this it is useful to consider what is meant by 'process'. What we know of a process is what we observe of it in terms of product. Observation and, therefore, the possible understanding of a process only equates meaningfully with the observation of a series of events - what we might term 'sub-outcomes' - over time, leading to the initially stated final outcome or product. Even a 'final' outcome might be regarded as but one stage in a grander process.

The implication of this definition of 'process' for independent learning is that in order to assess the process of learning we must 'tap into' the process at regular intervals over time. The language learning portfolio, for example, potentially reveals both process and product. To observe process the development of the portfolio has to be monitored at regular intervals during the course of study. (For further discussion see sections 5.3.1 and 6.4 )

2.2 Some current assessment methods for independent learning

The following are examples of some currently used assessment methods for independent language learning. With each there is a focus on product and/ or process.

• Compilation of the Learning Portfolio in a variety of forms including learner diaries, language exercises, open-ended activities. (See case study 2 in section 6)
• Group or one-to-one discussion of portfolio content: student/s and tutor talking through IL activities from the portfolio. Portfolio items for discussion selected by tutor and/ or student/s either prior to or in discussion session. Discussion centred on a list of prepared questions to promote parity across the cohort.
• Portfolio link to examination question. Students are required to keep a portfolio of pieces of work which relate to specific aspects of their learning and participation in a particular course topic. The portfolio tasks are set by the tutor but are generally completed in the students' own time. Completed portfolios are brought to the open-book exam at the end of the semester in which each exam question asks them to make use of a particular portfolio item. (See case study 3 in section 7 )
• Group work/ Peer assessment/ Self Assessment: having completed, for example, a portfolio, a valuable learning experience is to ask students to assess them; either their own or each others. A short exercise can be to ask them: 'in the light of your experience of producing a portfolio, what do you consider that you did especially well, and what would you now do differently?' Alternatively group members discuss individual learning experiences and write a report about another group member. ( Brown, et al, 1996, p.106) (See also case study 4 in Appendix 1 )
• Oral Presentation based on classroom or IL work, possibly recorded on video/ cassette for future analysis. (See case study 3 in section 8 )

3 Skills and strategies

As stated previously, in the assessment of IL there is a focus on not only assessing outcomes but equally with observation of the learning process which leads to the outcomes. It is the development of learning skills in the student which over time give rise to a more effective and efficient learning process.

It is arguable that students have always acquired such skills in passing. Whilst there is currently much emphasis on the identification and development of key skills and study skills through academic study we must be wary of decontextualising them and teaching them as separate skills; as if they were a distinct discipline rather than an underlying requirement for effectively learning anything.

Furthermore, regardless of how explicitly we feel we should promote key skills and study skills, we should remember that for language students the principal reason for studying on a language course will be to master a language; this is primarily what students want to achieve. We might bear in mind that an element of a course which places too much emphasis on skills development may appear to avoid the perceived main purpose of the course. However, skills development as an integrated part of independent learning will progressively enhance the process of language learning. Portfolio building as part of independent learning for languages provides a potentially successful approach to combining language learning with skills development.

When looking at the explicit assessment of the development of key skills and study skills in the independent learner we should first identify the skills that we wish to observe and, possibly, assess.

3.1 Key skills

Important key skills we might want to encourage include:

• Self awareness
• Self analysis of needs
• Awareness and development of personal and interpersonal learning strategies
• Team work
• Awareness of strengths and weaknesses
• Awareness of motivational type
• Awareness and development of learning style
• IT skills
• Familiarity with range of available resources and materials
• Time management
• Research skills
• Decision making

(The FDTL Translang Project's 'Guide to Transferable Skills in Non-Specialist Language Learning' provides materials for learners to evaluate their skills in these areas.)

3.2 Language learning skills

The following language skills and a wide range of sub-skills are important:

• Speaking skills (including such micro-skills as pronunciation, appreciation of register)
• Listening skills (including such micro-skills as listening for gist, listening for detail)
• Reading Skills (including such micro-skills as reading for gist, reading for detail, appreciation of style)
• Writing Skills (including such micro-skills as spelling, punctuation, planning, style, form)
• Strategies for learning vocabulary
• Strategies for learning grammar

For further discussion on language learning strategies see, for example, Wenden (1991) , Oxford (1990) , O'Malley and Chamot (1990)

4 The language portfolio

The use of the Independent Language Learning Portfolio as a tool for the assessment of independent learning is increasingly common.

4.1 What to include in the language portfolio

The portfolio may include the following items:

• All the language work completed during the course. (See case study 1 in section 6 )
• Independent Learning Activities: samples of or all of the work carried out independently by the student during the course, i.e. work not set by the tutor.
• Guided learning activities: activities set by the tutor to be completed independently. Guided learning activities are less prescribed than what is commonly termed 'homework'. An example may be to ask the student to carry out research on a particular topic.
• Homework: Prescribed tasks set by the tutor which relate closely to and serve to embed work done in class.
• Learning Logs: a document in which learners briefly record each independent learning activity undertaken. The completed log gives tutors an idea of the range of activities done by the learner and the time devoted to each, although it is susceptible to problems with authenticity. (See Appendix 4 and case study 1 in section 6 )
• In-house designed IL worksheets, provided by tutors and/ or available in an open access area. A worksheet may be completed for an individual language activity or a connected series of activities. A typical IL worksheet may require students to:
• Comment on their reasons for choice of activity,
• Comment on the materials used
• Comment on aim of the activity
• Reflect on the activity.

A worksheet may also contain a section for language notes, which can be as varied as the activities themselves. Students might use this to note vocabulary, examples of structures, questions arising on the language of the activity.

Learning Logs and Language Activity sheets taken together provide not only a record of activities undertaken, but also evidence of how the student has interacted with the language resources and materials used.

The contents of these documents can form the basis for discussion between tutor and learner on the learner's independent learning experience encouraging self-evaluation and reflective thinking.

(A sample of a Language Learning Activity Sheet from Leeds Metropolitan University appears in Appendix 5 .)

4.2 Assessing the language portfolio

In assessing a Language Portfolio, we face difficulties which we do not have, for example, in assessing traditional written tests. In traditional written tests it is possible to identify and define quite clearly the standards required for assigning grades or a pass/fail. With the Language Portfolio there are fewer absolute standards and many variable factors.

4.2.1 Problem areas in portfolio assessment

One learner may undertake very different activities to those of another in accordance with his or her identified needs: One portfolio might be full of grammar exercises, another might focus on listening comprehension activities and note-taking. Others may focus on a couple of skill areas or language tasks while some might have covered a whole range of skills, but more thinly.

In the course of the CIEL project we have seen examples of portfolio assessment criteria which cite quality, quantity, variety and organisation as the main areas for focus. However, these terms are not concrete and are open to interpretation. In watching, for example, a satellite TV broadcast, a learner might note key words or transcribe sentences. But equally, another learner might simply make a list of points he/she wants to check with a tutor or native speaker. Both would be valid activities. How do we judge quality consistently when looking at a whole range of activities?

It may be decided that such are the complexities of assessing portfolios precisely that it is unfair to grade and easier to award simply a pass or fail, but even to do this we need some proper criteria and ways of justifying our assessment.

Other problem areas with IL Portfolio Assessment include:

• Work being rushed at the last minute to meet the requirement of a set number of pieces of independent learning for inclusion in the portfolio; too great a focus on the quantitative aspect of the work rather than the qualitative.
• Exercises being done for the sake of doing something; because students have been told they have to present a certain number of items in the portfolio, without thinking through what they really need.
• Students feeling that Independent Learning may not be of benefit in itself, but that the Portfolio is a form of coercion into doing something because it counts towards the final mark.
• The logging of activities which have not actually been done/ invented notes on activities not completed.

4.3 How can problems be overcome?

Problem areas can be broached in a number of ways.

4.3.1 Monitoring

As stated in section 2.1, training to become an Independent Learner is an ongoing process and it is desirable if the tutor - or whoever is mentoring the portfolio work - can step into the process at regular intervals. The more regular the checks on student progress the more closely the process can be observed to see that good independent learning practice is being implemented. Questions can be dealt with on specific language points and IL strategies. The monitoring also gives an opportunity for students to ask questions, air problems and re-assess their needs.

Face-to-face monitoring sessions allow discussion with the learner about:

• work done so far
• on what basis the tasks were chosen
• what resources were used and why
• whether the student feels that learning is taking place through the activities they organise

Successes can be praised and problems analysed. The tutor/ learning advisor is able to offer advice regarding the learning strategies being used and offer suggestions which will influence the work which will be done subsequently.

This engagement is similar in nature to the way in which a teacher trainer might assess a teacher trainee. These discussions will assist the learner in steering the process in the most advantageous direction. To only take the portfolio in at the end of a course and mark it as one would mark any other type of final outcome does not necessarily reveal any details of process.

This type of monitoring is, of course, time consuming. However, this is in line with all experience of independent learning activities, i.e. they take time and effort to be undertaken successfully.

When discussing work with a student in the monitoring session, possibly for assessment purposes, the following questions may be considered:

Is the learner able to:

• identify his/her needs in the language and explain why these are evident?
• adjust needs in the light of progress/necessity?
• select suitable materials- appropriate media, level- to answer the above needs?
• use good strategies in use of the materials? e.g. for listening/reading
• undertake the work regularly?
• use a variety of media and develop his/her ability to use certain language learning hardware and tools, e.g. CD-ROM, Spell-checkers?
• develop his/ her ability use research skills (library catalogue, searching on Internet, use of dictionary and grammar reference material)
• incorporate IL work in other language tasks, e.g. class discussion, set homework, other assessments?
• give himself/herself feedback on the activities and check success?
• demonstrate other techniques e.g. consulting with native speaker, checking dictionary?
• display good time management?
• organise work appropriately for easy retrieval?
• show how undertaking a specific IL activity- or series of activities- has benefited him/her by giving concrete examples?
• reflect on and evaluated the activities done?
• meet regularly with tutor, advisor to discuss progress, ask questions about IL, the process or points of language
• to highlight some points raised by the material and how he/she dealt with them?

4.3.2 Reflection

Encouraging students to step back and think about areas such as those listed above helps them to understand, take control of and modify appropriately their own learning aims, strategies and outcomes. With time and practice this process of reflection will become increasingly automatic. Ackermann (1996, p.28) underlines the value of this reflective practice either, as it were, in communication with oneself or with others: 'People cannot learn from their experience as long as they are totally immersed in it. There comes a time when they need to step back, and from a distance reconsider what has happened to them. They must take on the role of an external observer, or critic, and they must revisit their experience "as if" it were not theirs. They need to describe it to themselves and others, and in doing so, they will make it tangible.

5 Structure

Careful structuring of the portfolio work leads learners to take responsibility for their IL work and indeed every aspect of their learning. The following case study from the University of Lincolnshire and Humberside describes how students studying languages as a minor subject on generic or vocational courses are encouraged through consultation with a learning advisor to create an action plan, recognise strengths and weaknesses, set learning objectives, activities and materials, and finally reflect and evaluate work done and progress made in the relative skills areas.

The case study also brings out issues which have been raised in section 4. This includes assessment criteria, assessment of product versus assessment of process and learner support. The case study raises particularly interesting considerations for those of us already using or currently considering using portfolios for assessment of process.

Case Study 1: using the portfolio to assess independent language learning, Andy Hagyard, University of Lincolnshire and Humberside

5.1 The context

The large majority of language students at the University of Lincolnshire and Humberside learn a language as a minor subject on a generic or vocational course, typically in Business or Tourism. This module represents 20% of the award, and the focus is on communicative language learning along with increased cultural awareness to prepare students for a period of work or study abroad.

The university has a small but relatively well-resourced self-access centre, with a wide range of multimedia learning materials including audio-visual materials, satellite TV, CALL programs and foreign language journals.

5.2 Background and history

The Modern Languages department at the University took the decision in the early 1990s to invest heavily in the development of a self-access centre for independent language learning. Over the next few years the Language Centre acquired facilities that are now standard features in similar institutions across the HE sector. These include satellite recording equipment and viewing facilities, a wide range of learning materials in various media, and increasingly multimedia PCs to exploit CALL programs and the Internet. In addition two new members of staff were appointed, firstly to develop and maintain resources and support for learning, and also to offer an advisory service to help students both in their choice of material and in the development of independent learning strategies.

From an early stage it was clear that students were unlikely to make full use of the resources at their disposal unless there was some form of monitoring or assessment. Initially each language group was allocated an hour when they were expected to work in the Language Centre. Students were asked to keep a log of their activities, with each session signed by a member of the Language Centre staff as proof of attendance. These slips were then included in a portfolio of work, which was assessed at the end of each semester.

It soon became apparent that these self-access sessions were being assessed on a largely quantitative basis, with little or no evidence that any effective learning was actually taking place. Sessions were typically summarised in one line statements such as 'watched the French news' or 'read the newspaper'. Furthermore, staff could not supervise individual students, and the signature confirmed only that he or she had attended, not that the description of the session was accurate. Indeed there were even several proven cases of signatures being forged! As a result, and after much discussion between Language Centre staff and teaching colleagues, the system evolved to incorporate more qualitative evaluation, and to encourage students to take full responsibility for every aspect of their learning. The revised format of the language portfolio is described below.

5.3 The language portfolio

5.3.1 Structure

The portfolio is in fact more than a simple compilation of independent work. It is intended to form a comprehensive record of all language work completed during the year, and as such is a highly useful revision tool. Students are instructed to include :

• a clear record of work done in seminars, including preparatory and follow-up work and work done to compensate for missed seminars,
• various compulsory elements (letter, presentation, listening comprehension,...),
• evidence of independent learning, equivalent to at least one hour per week.

The Independent Learning sections accounts for 50% of the marks for the portfolio, and it is this section that is most relevant to this particular case study.

Students produce an initial action plan, which should demonstrate an awareness of their own strengths and weaknesses. They then have to set their individual learning objectives for the year, and select a variety of appropriate activities and related materials to help them achieve these objectives. Students are strongly advised to consult with the Language Learning Advisor or their tutor at this point to ensure that the action plan is well thought-out, with realistic targets and appropriate activities. The importance of this stage cannot be overstated, as it then forms the basis for all independent work undertaken during the coming year.

Each independent learning activity needs to be documented. This is done by completing activity sheets, where students record not only what they did, but also why they chose this activity, what they learnt from it, and what issues it raised for their future learning.

The portfolio concludes with an evaluation, where students reflect on their learning and assess the progress they have made in each skill area, relating this to their initial needs analysis. They review their long term aims, and set new objectives for the next year.

5.3.2 Assessment criteria

The independent learning section of the portfolio assesses one of the learning outcomes of the unit, which is "to study and learn independently...and reflect upon what they have learnt". It does NOT assess communicative language ability itself, as this is quite adequately tested through other forms of assessment. Students are assessed on their ability to analyse their own needs, and develop a strategy to overcome their weaknesses. They are expected to include a variety of appropriate activities, using a range of different media, which are consistent with their original objectives. Finally they need to demonstrate an ability to reflect on their progress and evaluate the effectiveness of each activity undertaken.

There is also a quantitative aspect to assessment, as the number of activities undertaken is taken into account.

5.4 Process or product?

When assessing independent work, there has been much debate over exactly what is being assessed. Are we looking for evidence of communicative language ability (the product), or are we instead assessing independence in learning as a transferable skill (the process)? At the University of Lincolnshire and Humberside, the emphasis is clearly on the process, although students are expected to demonstrate progress in identified weak areas. Indeed, students are encouraged to focus on areas which they find difficult, and where they will inevitably make more errors. Any suggestion that assessment is based on the product would logically encourage learners to concentrate on aspects of the language where they are already proficient, or else to select tasks that are not sufficiently challenging.

Clearly this approach is only viable because communicative ability is already fully assessed by more traditional means (grammar tests, listening comprehension, presentation,...). The implicit assumption that underpins the portfolio rationale is that if a student has selected a task of an appropriate level, which is consistent with his or her identified aims and objectives and uses a suitable medium, and if he or she reflects on the outcome of that activity, then some form of learning will have occurred.

5.5 Support for independent learning

All relevant forms (action plans, activity records, etc.) can be downloaded from the university web site, together with general guidelines, hints on successful portfolio building and a wide range of related helpsheets and worksheets covering all aspects of language learning. This material can be freely viewed at http://www.ulh.ac.uk/ls/language/langc01.htm

In addition, all new students are given an induction to the language centre to introduce them to the range of resources. This takes the form of a physical visit, and focuses on practical aspects of the language centre, such as :

• where to find relevant books, tapes and journals
• how to use the tape recorders
• on-line resources, including dictionaries, CALL programs, internet sites, etc.

Approximately 3 weeks after the start of the year, these same students then attend a separate workshop to explain the principles of independent learning. The timing of these workshops is designed to avoid information overload at the start of the semester, and to give students advice when they most need it; i.e. once they have received their unit handbooks and are hopefully settling into a routine of academic work. These sessions are delivered by the language advisor during class time, and attendance is compulsory.

5.6 Lessons learnt

In 1996/97 the language portfolio was introduced as a compulsory element in all language units at all levels. This immediately lead to a substantial increase in usage of self-access facilities in the Language Centre, which was of course one of the original aims. However the satisfaction at seeing this upturn in usage was soon offset by the realisation that all this independent work had to be assessed. Even external examiners recognised that the marking load was excessive.

The portfolio also proved unpopular with many students who found it "boring and repetitive", and claimed to spend more time filling in forms than on actual language learning activities. Those students who benefited most from the portfolios were largely those who were already good independent learners. There was little evidence that simply forcing students to be independent is successful in developing the necessary skills for autonomy.

As a result, a number of changes were introduced. The portfolio has been retained only on first year units, on the basis that having learnt to work autonomously students would continue to do so in subsequent years (without the need for assessment). To reduce the amount of form-filling, only selected activities now need to be accompanied by a detailed activity record. Typically students will undertake a number of very similar learning activities (news videos, computer grammar exercises, speaking practice,...), so to avoid repetition only four or five representative but diverse activities need to be recorded in detail.

The need for systematic learner training has also been recognised, and independent learning workshops are now routinely organised, as described above. These sessions aim to convince students of the benefits of autonomous study, and explain the process involved in the development of an effective strategy. Personal experience has shown that there are a number of key arguments in favour of independent study :

1. While classroom work may at times be boring, irrelevant or too difficult, independent work should never be. Students can work at a time that suits them, using a medium they enjoy, working on a topic they find interesting, and at a level of difficulty that is just right for them.
2. The portfolio allows anyone to achieve a good mark, regardless of their initial level. This is particularly important when working with students from varied entry routes. (The converse is also true, i.e. it deters students from taking a language as an 'easy option' if they enter with a high level of language ability)
3. Students who work hard outside the classroom will not only achieve a better mark for their portfolio, they will also perform better in their exams. There are no magical 'short cuts' to improving language skills.
4. Language learning ability is a transferable skill. While employers want people with competence in one foreign language, they may also need staff with the linguistic versatility to acquire the basics of a second or third language.

5.7 The assessment paradox

It can be argued that the only true assessment of independent learning is self-assessment, and in an ideal world this would be the case. However much we tell students that the portfolio is for their own benefit, and that their reflections are personal, students ultimately know that they are doing it to get a good mark, and that everything that is written will be read by an assessor. This has in fact been highlighted by the cases of overseas students (e.g. Swedish students learning French) who would normally keep a reflective learning log in their native language but must artificially record their personal thoughts in a foreign language.

Inevitably there is a compromise : reflections and analysis should be personal, but they have to bear in mind that they will be read by another person. Students must simply accept that external assessment is a necessary part of the educational environment. Research has confirmed that some form of assessment is a key motivator for students. The loss of this one aspect of pure autonomy is the price that must be paid to ensure that students actually do the work.

5.8 Conclusions

Motivation has been identified as one of the defining characteristics of a good independent learner. Non-specialist language students typically do not have a high level of self-motivation, so in the context of this university some form of assessment is essential to ensure that self-access resources are well-used.

Portfolio-based assessment of independent learning can be successfully integrated into the curriculum in the first year, providing that students themselves are convinced of its value and receive adequate training and support.

6 Linking the portfolio to the final assessment

In the following case study from Richard Fuller we see how the compilation of the portfolio is given particular value by being linked to the traditional exam assessment. Whilst the case study focuses on the first year of a B.Ed in secondary teaching, the methodology, which reveals process explicitly linked to product, is relevant to other disciplines including the study of languages.

Case Study 2: using portfolios in assessment to improve the quality of learning, Professor Richard Fuller, Edith Cowan University, Perth, Western Australia

6.1 Introduction

As a university teacher, I believe that it is important to help my students develop a high level of understanding of the subject matter they work with and a deep interest in it. The principles of constructivist learning indicate that if students are to develop a good understanding of the subject matter, they must relate what they are learning to their own knowledge and experience. Doing this will also help them develop and maintain a high level of motivation for the subject. Although there are many ways of getting students to make connections between what they already know and what they are learning, it can be quite difficult to get them to actually do this on a continuing basis over a semester of study. Even those who start with the best of intentions can find that other things get in the way.

The easiest way that I know to get students to take a learning task seriously and give it a high priority is to make it part of the assessment process. This virtually guarantees that all students will do the work, but it also requires the lecturer to assess the work. However, it is not always appropriate to assess students' work. In particular, assessing student's personal connections between their existing knowledge and the subject matter can easily distort the intention of the experience: once an item becomes assessable, students look for the "right answer", which prevents them from constructing their own understanding of the subject matter. Further, assessing each individual's unique connections between existing knowledge and the subject matter can be a time-consuming process, which few lecturers can afford in the current climate of declining resources and increasing work loads.

I use portfolios in a particular way that gets students to make personal connections between their own knowledge and experience and the subject matter they are learning and incorporates these connections into the assessment package without distorting the learning process or involving me in a heavy time commitment.

6.2 The process

In explaining how I use portfolios, I will relate my explanation to a single unit of study: Issues challenges and directions in education. This is a first year unit in the Bachelor of Education (Secondary Teaching), and is the only unit taken in first year which relates specifically to the field of education. (Other units focus on the subject matter of students' chosen teaching areas, such as history or mathematics.) The main objectives of the unit are to encourage students to think about teaching and learning from the perspective of the teacher, and to help students become more insightful about their own learning at university. I do not regard the specific content of the unit as necessarily being important in itself: its significance is that it provides a way of getting students to explore new ideas and review their current perspectives on teaching and learning. About 180 students enrol in this unit each year. Classes comprise a one-hour mass lecture each week and a two-hour workshop involving about 25-30 students. Assessment consists of a class presentation of a poster that answers a question on teaching or learning (20%), a written essay (20%), and a 3 hour open book examination at the end of semester (60%).

As an integral part of the learning process, students are required to keep a portfolio containing about ten pieces of work that relate to specific aspects of their learning and participation in the unit. Here are two typical examples:

1. When studying motivation, we drew your attention to four major categories of motivation: extrinsic, social, achievement, and intrinsic. Which of these is strongest for you as a university student at the present time? Was it the same when you were at school? If there has been some change, what caused your motivation to change? Is this a change for the better? Relate your response to at least one of the readings by Biggs & Moore (1993) and Woolfolk (1998).
2. Think about your own learning, either at university or in some other context. Identify a situation when you found it fairly easy to learn and remember something, and another situation when you found it more difficult. Use your knowledge of learning and learning strategies to explain why it was easy for you to learn and remember on one occasion but not the other. Relate your response to at least one of the readings set for the topic How do students learn?

Students generally complete the portfolio items in their own time, but some class time is devoted to discussing selected items and giving students opportunities to share draft responses with their peers. They are not expected to write extended answers to these questions, for that would distort their purpose. Instead, they are encouraged to write relatively brief responses (about one page each) that focus on the most significant aspects of the task. They use a loose leaf file so that it is easy to add new material, revise their work, and submit selected pages with their examination paper. They are told to bring the completed portfolio to the open-book examination at the end of semester, and that each examination question requires them to make use of a particular portfolio item. For example, the following examination questions require students to make use of the two portfolio items presented above:

1. The term "motivation for learning" can mean different things to different people. What do you understand by this term? Is there a particular type of motivation that teachers should try to foster in their students? If so, explain why this particular type of motivation is so important. Support your answer by referring to portfolio item 1. Attach your response to this portfolio item to your answer to this question.
2. Imagine that a high school student asks you for advice about how she could improve her understanding of, and memory for, her school work. She understands what is presented in class, but does not know what to do outside of class to deepen her understanding and help her remember what she has learned. Nor does she know what to do in order to learn material that was not covered in class but is available in her textbooks. What sort of advice would you give her, based on your own learning experiences and the ideas about learning presented in the reading by Woolfolk (1998)? Support your answer by referring to portfolio item 2. Attach your response to this portfolio item to your answer to this question.

At the end of the examination, students attach the relevant portfolio items to their examination scripts. When reading their answers, I allocate marks for the way they have used the portfolio item in answering each question. As a result, I do not have to analyse the portfolio items thoroughly, but I do read them to get an idea of the relationships that students make between their own experience and the subject matter and to find out about any matters I should consider when teaching the unit next time.

6.3 Discussion

When used in this way, the portfolio is an integral part of the teaching, learning and assessment processes in the unit and helps students develop their personal understanding of the subject matter. This is evident when I read their examination papers and see how they have used the portfolio to support their answers and incorporate their own knowledge and experience into their learning. The portfolio also helps students become a little more independent in their learning, since they have to decide for themselves what sort of response they will make to each item and complete this with little direct input from the teacher. Further, students have to construct their own answers: none of the portfolio items can be answered just by using material found in text books or other readings.

Feedback from students about the use of the portfolio in this way is overwhelmingly positive. They report that relating the subject matter to their own learning and experience helps them understand it better and makes it more interesting, and they like the way that the portfolio confirms that their own beliefs and experiences are an important part of their learning. They report that they do not always work on the portfolio in isolation, and some of them arrange working groups in which to discuss and refine their draft responses.

Naturally some students procrastinate and do not write their portfolio items at the intended time, and find themselves having to prepare them in the last week of semester. To some extent this defeats the purpose of the portfolio. But this will happen with any learning experience that we provide for students. Overall, most students do them on a fairly regular basis and get the learning benefits from doing so.

6.4 Conclusion

I believe that using portfolios in this way encourages most students to do what the better students probably do without needing to be told. Further, since it does place any great demands on the lecturer's time, it can be used to enhance the quality of student learning without requiring any additional resources.

7 Presentation assessment

'Being able to speak convincingly and authoritatively are useful career skills for students. One of the best ways of helping them develop such skills is to involve them in giving assessed presentations.'
Brown, et al (1996)

The following case study from the University of Southampton shows how first year language undergraduates are assessed on their presentation skills. This focus of this form of assessment is traditionally on the product of learning, i.e. the productive skills. However, undertaken in the way described below the activity gives insight too into the process of learning since the performance will be greatly improved by appropriate preparation in the following two areas: both research into the subject matter and research and practice regarding 'how' to give a successful presentation.

Case Study 3: assessment of integrated autonomous learning, Margaret Teremetz, University of Southampton

7.1 Background

The class is made up of mainly 1st year undergraduates, who gained a French A level at grade C or below. They have 3 classes per week for French language, class 1 concerned with receptive skills, class 2 with productive skills, and a further class with a native-speaker assistant.

A programme of work is prepared for every 5 - 6 weeks, to help students plan their study; this also helps harmonise the teaching of four different groups by more than one teacher. The module of work comprises authentic materials which have a particular content focus, intended to provide general interest as well as to inform students about aspects of francophone culture. Study skills and grammar are integrated into this main focus. The intention is to provide a continuum through different foci, media and modes of working.

The programme is presented to the students in grid form (see Fig 1). For the purpose of this article, only one example of integrated work is inserted, but the students' work programme is normally outlined in full; assessments are clearly stated, and if necessary detailed more fully as with the activity in question.

7.2 The activity (see fig 2 in Appendix 7 )

Students are asked to compare different aspects of different news supports, e.g. the front page of Le Monde and the front page of Libération in the form of an oral presentation, in twos or threes.

Assessment of the exercise rather than of the presentation

A typical oral / presentation assessment sheet might include:

Interactive skills / fluency
Pronunciation / intonation
Content / coverage
Range of appropriate vocabulary
Grammatical accuracy

Taken at face value, assessment of this task could seem to be focussed only on the performance. Even if it is, performance is greatly enhanced by adequate preparation out of class.

If the exercise is assessed in terms of learning outcomes, it is possible to put greater emphasis on student input, the only evident teacher input being to explain the objectives and point to sources of help - and ultimately provide a framework for reflection and feedback. The skills practised and learning outcomes and their part played in the preparation of this exercise include the following.

7.3 Cultural insight and research skills

This task fits into a module on general knowledge about France. Unlike most of the materials in the course, however, this is open-ended and so provokes a more active, and thus possibly a more effective, approach to acquiring knowledge than ready-prepared material. If a student has been taking an interest in current affairs, and in the resources on offer, they will find it easier to approach this task. Equally, if they are developing effective research skills, they will be more likely to make an appropriate selection of material, and produce a meaningful performance.

7.4 Receptive skills

Effective reading or listening skills are needed to prepare the presentation. Comprehension of the material studied necesssarily comes out in the performance.

7.5 Productive skills

Preparation of the performance is assessed by the performance itself, and will include practice in pronunciation of the key words, familiarization with the gender of key nouns and practice with the verbs likely to be used. Where glossaries, handouts and transparencies are prepared for the audience, the quality of these will be part of the assessment. This is not necessarily a uniquely oral task, therefore.

7.6 IT skills

This task provides an opportunity to use IT for both research and visual aids or handouts. The former may not be essential, but a presentation without, say, some well-presented bullet-points, may score quite low in terms of visual effect and general communication.

7.7 Communication and presentation skills

Although these will be assessed by the presentation itself, it is relevant preparation that will actually be being assessed. This will include looking for hints on making presentations on the University's Web site, and practising speaking clearly at the right pace and volume. Students will be able to reflect on previous experience as a listener: that when people read out written texts the language is difficult to understand; the value of effective body language and tone of voice; the importance of clear visual clues to help understand structure and content. The group will have discussed and experienced such things on previous occasions.

7.8 Teamwork

Before this, students will have done solo presentations. This task can extend the students' experience of, and reflection on, cooperative work. It is difficult to assess this valued aspect without a discussion with the students or a report from them, but the resulting presentation will inevitably reflect the success or otherwise of the cooperation during the preparation. Cooperation is often manifest during the presentation, in taking turns, or making notes on the board, and even in adopting an appropriate stance while the other student has the floor, and will be particularly strong if the experience of cooperative preparation has been successful.

In final discussions during the feedback period, students talk about the experience of standing up when everyone is sitting, and what it feels like when people are not paying attention. This enhanced social awareness is difficult to assess, except by a reflective post-presentation self-assessment, which would be a valuable activity if not a solid basis for marking.on a separate sheet, as with the activity in question (see later).

7.9 Conclusion

What seems at first a fairly simple task can have multiple learning outcomes. If these are made explicit to the student as inherent in the marking of the presentation, the student should work more purposefully and gain more from the experience. Sometimes the gains are unexpected; in fact the original purpose of this activity was to promote individual research, but in fact the most important perceived learning outcome for the students is the gain of confidence in speaking. Again, effective guided out-of-class preparation is at the root of this.

(See Appendix 2 : Giving oral presentations: a quick guide, and Appendix 3: Assessment criteria for oral presentations)

8 General conclusions

'One of the fundamental concerns about some traditional methods of assessment is that they fail to allow students to express the individuality of their achievement. Further concerns revolve around the fact that exam performance is in many respects an artificial skill, often seeming to encourage surface learning.' ( Brown et al, 1996)

In this handbook we have aimed to demonstrate some of the alternative forms of assessment which can offer to both learner and tutor valuable insight into the individual's cognitive processes, rather than simply revealing an ability to memorise and repeat. If this focus on the process of learning begins early on in the programme of study this means that the benefit of understanding one's own learning agenda are felt from the outset. It also allows time for the student to further develop as an effective learner, over time gaining proficiency in the range of learning skills most appropriate to his or her own context.

Appendix 1: Case Study 4 - Assessment of progress and independent learning: empirical evidence

Download Appendix 1 (rich text format, 52Kb)

Appendix 2: Giving oral presentations: a quick guide

Download Appendix 2 (rich text format, 54Kb

Appendix 3: Assessment criteria for oral presentations

Download Appendix 3 (rich text format, 51Kb)

Appendix 4: Sample learning log (Leeds Metropolitan University)

Download Appendix 4 (rich text format, 186Kb)

Appendix 5: Sample language activity worksheet (Leeds Metropolitan University)

Download Appendix 5 (rich text format, 179Kb)

Appendix 6: Case Study 3, figure 1: format of the work programme

Download Appendix 6 (rich text format, 53Kb)

Appendix 7: Case Study 3, figure 2

Download Appendix 7 (rich text format, 46Kb)

Bibliography

Ackermann, E. (1996). Perspective- taking and object construction: two keys to learning. In Y.Kafai and M.Resnick (eds), Constructionism in practice. Designing, thinking, and learning in a digital world , 25-35. Mahwah, NJ:Lawrence Erlbaum.

Atkins, M., J. Beattie and W.B. Dockrell (1993). Assessment Issues in Higher Education , Newcastle upon Tyne, Employment Department Group.

Brown, S., P. Race and B. Smith (1996). 500 Tips on Assessment, Kogan Page, London.

Brown, S., C. Rust and G. Gibbs (1994). Strategies for Diversifying Assessment in Higher Education, Oxford, Oxford Centre for Staff Development.

Fulcher, G. In preparation. "Independent learning: a longitudinal study."

Gibbs, G. & T. Habeshaw (1990). An Introduction to Assessment: Induction Pack 4, Birmingham, Standing Conference on Educational Development (SCED Paper 57).

O'Dochartaigh, P & M. Schmidt Effective Practices in Assessment in the Modern Language- An Interim Report (FDTL Assessment Project), University of Ulster at Coleraine.

O'Malley, J. M. & A.U. Chamot (1990). Learning Strategies in Second Language Acquisition. Cambridge University Press.

Oxford, R.L. (1990). Language Learning Strategies What Every Teacher Should Know. Heinle & Heinle, Boston, Massachusetts.

Pennycook, A. (1997). Cultural Alternatives and Autonomy. In P. Benson and P. Voller (eds.) Autonomy and Independence in Language Learning. London: Longman, 35 - 53.

Reid, J. (1985). Perceptual Learning Styles Preference Questionnaire. In J. Reid (ed.) Learning Styles in the EFL/ESL Classroom. Boston, Heinle & Heinle, 202 - 207.

Ross, S. (1998). Self-assessment in second language testing: a meta-analysis and analysis of experiential factors. Language Testing 15, 1, 1 - 20.

Rowntree, D. (1987). Assessing Students, Kogan Page, London.

Sinclair, B. (1999). Survey Review: recent publications on autonomy in language learning. English Language Teaching Journal 53, 4, 309 - 329.

Soo,K-S and Y-H. Ngeow (1997). Cost effective ESL instruction with interactive video: The MCALL project. [Online] Available:
(http://php.indiana.edu/~ksoo/JEMH.HTM.)

Wenden, A. (1991). Learner Strategies for Learner Autonomy. Prentice Hall, Hemel Hempstead.

Referencing this article

Below are the possible formats for citing Good Practice Guide articles. If you are writing for a journal, please check the author instructions for full details before submitting your article.

• MLA style:
  Canning, John. "Disability and Residence Abroad". Southampton, 2004. Subject Centre for Languages, Linguistics and Area Studies Guide to Good Practice. 7 October 2008. http://www.llas.ac.uk/resources/gpg/2241.
• Author (Date) style:
  Canning, J. (2004). "Disability and residence abroad." Subject Centre for Languages, Linguistics and Area Studies Good Practice Guide. Retrieved 7 October 2008, from http://www.llas.ac.uk/resources/gpg/2241.