Timescale

September, 2002 - January, 2004

Key contact(s):

John Canning
Subject Centre for Languages, Linguistics and Area Studies
j.canning@soton.ac.uk

Funded by:

Subject Centre for Languages, Linguistics and Area Studies

Successful bidders:

• University of Leeds and Heriot-Watt University
• University of Central Lancashire
• University of Paisley
• Liverpool Hope University College
• University of Lampeter and University of Gloucestershire
• University of Bristol 

Outcomes:

Languages

Project title: Peer and self-assessment in conference interpreter training

Contacts: Tony Hartley and Gracie Peng, University of Leeds; Ian Mason and Isabelle Perez, Heriot-Watt University
Download report: Peer and self-assessment in conference interpreter training (rtf, 698Kb)

Summary: This project aimed to promote learner autonomy in interpreter training by providing learners with detailed and applicable guidelines when practising peer-assessment and self-monitoring. This was done by observation of the practice of expert interpreters when conducting interpreting classes, interviews with trainers and end-users, and using these results to establish peer assessment criteria in the form of a grid for use with student trainees and the evaluation of their subsequent use of this grid with participating groups of learners at two UK HEIs.

This has been a relatively small-scale study, involving a small dataset from groups of participants in just two institutions. Findings could usefully be tested against the outcomes of more large-scale studies. In particular it would be useful to test the grid again on more advanced trainees who appear to have conceptualised a better 'hierarchy' of attributes of quality interpreting. One encouraging finding was that trainees were uniformly positive about the prototype grid. The reasons cited included completeness of coverage of criteria and lasting usefulness compared to verbal feedback.

The assessment tool presented in this research has been devised specifically for training in simultaneous conference interpreting. There is a need for studies of appropriate criteria for both consecutive and liaison interpreting.

In addition, interesting perspectives emerged from the consultation of professional interpreters. The suitability of a performance as a source of relay interpreting was one of the most frequently mentioned techniques among experts. This real-life criterion has not been included in the final grid. Constraints of time and research scale meant it was not possible to investigate this issue in this project.

The project team plan to create a computer-based implementation of the grid in hypertext format, with a precise definition of each criterion and instructions for use. This will allow trainees to make flexible use of the tool from a single interpreter workstation, and will form a key part of an architecture for distance learning of conference interpreting skills.

Project title: The portfolio as a learning tool in Languages: an effective means of embedding good learning practice or palliative for languages under threat?

Contact: Ruth Pilkington and Joanne Garner, University of Central Lancashire
Download report: The portfolio as a learning tool in Languages (rtf, 333Kb)

Summary: Portfolios have been seen as a means of encouraging autonomous language learning within languages for a number of years, and have been developed through the work of the ‘Translang’ project’, and through Language Centres across the country. They have been heralded as a means of countering reduced contact hours, developing learning skills, and encouraging reflective learning strategies.

This project exploits the opportunity given by a wide range of language subjects making use of portfolios within one institution, to investigate the quality of learning experienced by students, the learning drawn from the experience and asks initial questions about whether and how students apply the techniques acquired. The project reviews its conclusions in the light of the learning and teaching experience of other subjects and institutions using portfolios, and by current pedagogic research. The outcomes suggest the portfolio cannot be assumed to be a certain guarantee of raising skills and learning awareness, nor of generating conditions for transfer and reflection. The implication is that whilst the portfolio is valuable on a subject level, it may in learning terms not provide all the answers promised. Of greater importance is the extent to which tutors themselves are able to engage with current learning trends and to communicate these and engage students with them.

It is quite clear from this project that colleagues, even those involved with control groups, who had participated in discussions and signed up to an agreed approach reflected a wide diversity in terms of understanding and commitments to the portfolio. This comes across clearly in the issues identified by students:

• lack of clarity with regard to tasks, their purpose and nature, and the process around building them into a portfolio;
• a strong tendency to position and limit the value of portfolios to the subject discipline context, even where they recognise its value in terms of skills.

This poses an ongoing problem both to the portfolio itself as an effective learning tool, and certainly puts into question the success of it in relation to PDP. An established portfolio system cannot guarantee a skills-focussed, reflective, student-learning centred curriculum in which students will engage with a deep learning, high synthesis and cognitive approach to reflecting on their own progress and managing their own learning. Deep rooted changes in delivery are required. The implication is that staff development is the key, and that the dialogue and communication about learning with students will make demands on staff and curriculum delivery which must be addressed first for the PDP process to be implemented effectively.

Many of the issues identified within the literature review outlined in the full version of the report were demonstrated in the surveys. It is difficult to separate portfolio issues from issues of general pedagogic theory, and a question emerges as to whether within the context of constructive alignment (Biggs 1999, Jackson 2000), portfolios may just as usefully be replaced by other assessments. The value of the portfolio resides in the extent to which the process elements are emphasised and the extent to which it supports and motivates. The results of the survey suggest that the European Language Portfolio model may strengthen links between the portfolio and the subject weakening the relevance to PDP approaches.

There remain questions around transfer, around the impact of new initiatives in learning at secondary level, and around perceptions, both on a cultural level within EFL, but also as regards communication and how we engage students with the learning approaches espoused by PDP, portfolios and employability.

Project title: Ab initio language teaching in Scottish universities

Contacts: Susan Stuart and David Bowker, University of Paisley
Download report: Ab initio language teaching in Scottish universities (rtf, 648Kb)

Summary: This project reviews the provision and operation of a range of ab initio language courses in Scottish universities. Course co-ordinators and lecturers returned 31 questionnaires, giving information about 42 language courses. A fuller picture of certain courses and institutions was derived from 13 semi-structured interviews with staff at 7 institutions. It was found that current provision demonstrates a number of features highlighted in earlier research in the UK and that it is possible for students who start as beginners to exit as successful Honours graduates in the language. It may be, however, that the success of these students depends on a curriculum that is not appropriate for all students who take an ab initio course.

The project revealed a positive picture from the lecturers’ perspective. Although none of the interviewees was aware of an institutional policy concerning language provision, faculty entry in the older Scottish universities facilitated access to language study, and student interest is such that there was little need for specific marketing.

Ex-ab initio students were perceived as being just as successful at Honours level as students who entered with prior qualifications in the language. There appear to be three main reasons for this: weak students drop out and in some cases are encouraged to drop out at an early stage; the ‘survivors’ are highly motivated; qualified students lack the explicit knowledge of language structures that are expected. These factors combine to make the two groups of students indistinguishable by the time they complete their studies.

In summary:

• there is a real opportunity for students to take study a language from ab initio level at Scottish universities;
• students who take these courses in first year can progress to being successful honours students;
• the approaches to teaching and learning are quite diverse; none appears to be more successful than any other;
• provision in most cases focuses on the needs of those students who will continue their language study, possibly to the detriment of others;
• the year abroad makes a significant contribution to equalising the performances of ex ab initio students and those who have entered with a Higher or an A-level in the language;
• exiting opportunities at the end of the first semester or the first year mean this is a low risk system for both students and the institution.

Area Studies

Project title: Listening & Learning: Student & Staff Perceptions of Oral Assessment in American Studies

Contact: Stephen Perrin and Robert Busby, Liverpool Hope University College
Download report: Listening and Learning project (rtf, 412Kb)

Summary: In 1997 the American Studies department at Liverpool Hope University College introduced oral assessment for seminar participation. While it is accepted that this form of assessment has improved student participation, the formulation of appropriate assessment criteria has remained controversial. The Listening and Learning project used the Ideal *** Inventory (Norton 2001) to measure student and staff perceptions of appropriate oral assessment criteria with a view to modifying and improving future practice.

The research project revealed a number of complex issues. Student emphasis on ‘unintelligible’ colloquial speech brought up the phenomenon of the linguistically enclosed society. The majority of students at Liverpool Hope come from working class Liverpool or Irish (Northern and Southern) backgrounds and have experienced little socio-geographical mobility before their entry into Higher Education. Despite the levelling influence of the mass media, it would seem, living in a linguistically enclosed society leads to the presumption by the speaker that everybody understands what they are saying; increasing numbers of non native speaker students serves to magnify this problem.

Instructions to ‘speak properly’ during seminars would, no doubt, be seen as culturally insensitive. It may also be the case that students have not been previously exposed to environments where respectful consideration of a debate has taken place, and lack the experiences of employing different usage of language in different settings. One must assume, however, that students have developed the ability to ‘code shift’. As a consequence, encouraging students to consider the type of speech appropriate to the academic environment is also necessary.

In terms of further research, work on students’ understanding of tutor feedback would be worth consideration.

The number of students who stressed good listening skills and the need to curb ‘competitive’, and ‘dominating’ behaviour by their peers suggests the need for greater policing of seminars on the part of staff. However, a balance has to be struck to prevent students claiming inappropriate credit for non-participation on the grounds of listening skills. Moreover, it should be stressed that competitive discussion needs to be curbed rather than eradicated, in part to ensure that students are appropriately versed in different types of oral discussion and are adequately prepared to deal with differing situations, both at university and in any future employment. The new assessment criteria, however, particularly those which relate to open class discussion provide a common frame of reference which stresses interaction and listening skills as well as encouraging active participation, and thus make a meaningful contribution to self management of ‘competitive’ and ‘dominating’ behaviour on the part of some students.

A further issue with the nature of the Ideal *** Inventory was highlighted by the fact that a number of points came to the fore which were raised by only one or two students: these points were considered but ultimately thought to be largely unreflective of the general student and staff needs. The researchers are aware, however, that the adoption of such a consensus-based approach could mitigate against originality of thought.

Another aspect of the methodology which needs to be addressed is that if we accept that students are more likely to understand and act on criteria if they have helped to formulate them, it would mean that the criteria would need to be rewritten every time a module is run. Given time constraints, a five year review would form a timely opportunity to, at the least, evaluate whether there are any significant alterations or amendments suggested by at least a sample group within a degree pathway.

In conclusion, it is clear from this study that the implementation of the new criteria serves to improve student participation in the learning process, makes the assessment criteria relevant to the current student body and allows institutional, linguistic and social class issues to be incorporated into assessment methods. It addresses an emergent and largely unexplored area, employing a simple responsive and easily applied methodology which is transferable from pathway to pathway and institution to institution.

Project title: Teaching across Cultures: Anglophone Area Studies and Student Diversity in an International Context, University of Wales, Lampeter

Contact: Andrew Hassam, University of Wales, Lampeter, Clare Spencer University of Wales, Lampeter, and Ros Jennings, University of Gloucestershire
Download report: Teaching across cultures (rtf, 99Kb)

Summary: This project aimed to investigate classroom practices in Anglophone Area Studies programmes as they related to student awareness of cultural difference. It aimed to examine in particular, how programmes addressed student diversity and the relationship between the cultural composition of the learning environment and the culture and society being studied. An on-line questionnaire was designed to examine the multiple cross-cultural relationships between teachers, their students and the societies being studied and to invite Area Studies colleagues to reflect on whether their learning activities and assessment methods fostered cross-cultural awareness and engaged the identity consciousness of their students. The number of respondents was disappointingly low, which was tentatively attributed to the gap between subject-centred research and pedagogical research. In addition, Area Studies is being insufficiently recognised as a category relevant to those teaching in subject fields like American Studies, Canadian Studies and Australian Studies. The responses themselves seemed to point to a pedagogical black hole in terms of Area Studies and issues of classroom diversity and reflections by our respondents on the process of completing our questionnaire did indicate that the project had posed some interesting questions. With regard to American Studies programmes that formed our responses, there was a sense that diversity of the area itself poses some particularly interesting issues for further research related to teaching and learning strategies that engage with student diversity within the UK classroom.

A close reading of the small number of responses indicated that the majority of those who replied to the survey were either unaware of any support for pedagogical development in Areas Studies teaching (though some resources such as the LTSN and various funding initiatives from individual institutions were mentioned) or had not actively used support resources in the last two years.

Linguistics

Project Title: Teachers into researchers

Contact: Richard Kiely, Gerald Clibbon, Pauline Rea-Dickins, Catherine Walter, Helen Woodfield, University of Bristol
Download report: Teachers into researchers (rtf, 858Kb)

Summary: This research report examines the implementation of a series of innovations in a research methods course in a postgraduate programme (Masters) for Teachers of English to Speakers of Other Languages (TESOL). The specific innovations are i) a new assessment format which centred on the writing of a critique on a published research study, and ii) a series of oral presentations based on the written critique (post assessment) to peers. The approach to the research has two principal features: it develops a comprehensive account of the student experience through use of self report in the questionnaire and interview studies, and it has a longitudinal dimension, evidenced in the dissertation phase of the research. The data include a series of four questionnaire studies, interviews and an impact trail to the dissertations completed at the end of the programme. In addition to contributing to an evaluation of these innovations, the research informs wider debates relating to the skills development in research methods, and the role of such skills in the professional development of teachers.

Conclusion

Summary of findings

The evaluation of the research methods programme

• The overall hour-glass curricular approach – research methods input, critique on one published study, and a series of oral presentations based on the critiques – to teaching research methods is a valuable one.
• The focusing of the assignment on one specific research article seems to have been particularly effective in developing critical reading skills, and this in turn may have added value to the dissertation-writing process.
• The oral presentation phase deepens understanding, and has the potential to broaden conceptions of research where the presentation is communicatively effective and interactive.
• The approach contributes to the dissertation writing component of the programme in three ways: first, it assists in scoping a study; second, it guides the research design and development of data collection instruments; and third, it provides a useful organisational perspective, both in terms of what the research does, or what the final study might look like.
• The findings in relation to the two specific innovations – the critique of a published research study, and the series of oral presentations based on these written critiques – suggest that student perceive them as contributing to learning in a range of ways.

Teacher perspectives

• Strong students tended to integrate their developing understanding of research with their professional identity. This is evident in the data in terms of their approach in the oral presentation: their confidence and authority, and their grasp of research purpose and relevance derive from their teacher identities.
• Weak students tended to look at the research article as a set of prescriptions for the classroom, a perspective which may derive from limited experience as teachers, and relatively low level of academic English skills.

Learning through oral presentation

• Particular value was attached to the process of revisiting the article, the written critique and the tutor’s feedback in the context of having to present it to peers.
• The public aspect of this engagement with the critique provided its own pressures, motivations, benefits and disappointments, deriving in part from personal expectations, and partly from social factors.
• While there was awareness of the opportunities for learning in the discussion which followed the presentation many students were disappointed by the limited feedback of peers.

Further resources available on our website:

• Phase 3, to be published in April 2008
• Phase 2, published in January 2005