Mark my words: creative assessment and effective feedback in linguistics

Location map | Programme | Abstracts | Event Reports

The Subject Centre is organising a seminar to showcase and discuss innovation and good practice in the assessment of Linguistics in UK Higher Education. This seminar will explore some of the ways in which colleagues are assessing learning in Linguistics from the traditional to the technical in order to share expertise and support new ideas.

Learning outcomes

• Understanding of the role of assessment in the teaching of linguistics
• New ideas and approaches to assessment in linguistics

Workshop fee

There is no charge for employees and postgraduate students of publicly funded UK educational institutions. The fee for employees and postgraduate students of private institutions/organisations and non-UK institutions is £40.00.

Lunch will be provided. Please note we reserve the right to charge a £50.00 non-attendance fee.

Travel bursary

A travel bursary is available for this event. Closing date for applications is: 16th May 2008.

Abstracts

Session 1: Fieldwork for Success

Stimulating interest in subjects which appear marginal to the main thrust of a course of study or which are perceived as being "difficult" or in some way "different" is a challenge often faced by teachers at all levels of learning. Phonetics is a subject which is often disliked by arts and humanities students because they consider it to be too scientific and too hard – a subject in which they believe they are unlikely to excel. One strategy for promoting a closer involvement with the materials is an assessment exercise based on the notion of fieldwork.

Each student is provided with a unique piece of data – a specially selected word – which they then "investigate", week by week, applying each new step in the acquisition of phonetic theory to the increasingly minute description of how the word is spoken aloud. This technique – akin in some ways to the idea of a learning log – is also suited to blogging. The experience promotes engagement, ownership and a demonstrably improved grasp of the subject matter.

Session 2: “Do as we do not do as we say”: research-led assessment of courses in social dialectology

This paper presents the methods used to assess two undergraduate courses, one beginners and one more advanced, in language variation and change. In the beginners’ course, students go through the process (from conception, reviewing the literature, piloting, ethical data collection, transcription, analysis, presentation and write up) of conducting their own research in social dialectology. Like Labov, they conduct a rapid anonymous survey (the written version of which forms part of their assessment), before, like Wolfram, sensitively and ethically collecting recordings of conversational data from a non-standard dialect speaker (assessed…), before, like Trudgill, engaging in variationist analysis of data from the English of East Anglia. They present their research to a mini-conference of their classmates and others (assessed…) before writing up a final (assessed) ‘journal article’. Advanced students use collected data to engage in more theoretically and analytically demanding socio-phonological analysis which, for assessment, is both presented orally, and launched on a self-designed website. The courses, therefore, while placing the acquisition of sociolinguistic methodological, analytical and theoretical skills at centre-stage, incorporate a wide range of more transferable key skills. Moreover, students learn what it is like to be a practicing dialectologist, rather than (just) read about them. The presentation will finally consider the complications as well as the attractions of these methods of assessment.

Session 3: Distance teaching and assessment: developing online multimedia activities for MA phonetics and phonology students

1. The lack of face-to-face teaching in distance programmes can be felt particularly in an area such as phonetics and phonology where audio visual cues are so important. This paper will report on a project to develop multimedia online activities to enhance the provision of text-based learning materials and provide greater scope for a range of assessment methods to add variety to the traditional written assignment approach.
2. The activities were developed with our own distance MA in Applied Linguistics and TESOL students in mind but would be relevant to a range of linguistics, English language or ELT students, working either at distance or autonomously.

Session 4: Group Project Assessment

This paper presents and discusses the group research project and presentation assessment methods used for two modules on the Masters programmes in Cross Cultural Communication at Newcastle University. The modules are Professional Communication Skills and The Social Psychology of Communication. The group presentations now count for 40% of the mark on these twenty-credit modules, with an individual essay counting for the other 60%. The projects are introduced in class early in the module, with students being placed in multicultural groups, and given a detailed rubric, as well as guidance on procedure, assessment criteria and time management. A ‘peer assessment’ element is introduced at this stage, which encourages self-policing of group contributions, and offers help with project planning, management and delivery. The groups have around a month to research the topic and prepare a presentation of their findings. Presentations are then delivered to the class near the mid point of the module: this aims to encourage group bonding and peer teaching, and allows for formative feedback. Detailed grading is given to the group after the presentations: 50% of the mark is given for presentation content, criteria including coherence, exemplification, critical reflection and understanding of theory. 50% of the mark is given for presentation skills, and these include use of language, visual aids, voice and hand-out.

Session 5: Giving extra credit in half the time: managing teaching and assessment with reduced teaching hours

As part of a shake up of UG linguistics teaching in the School of Languages at the University of Salford half a dozen years ago, a set of 10-credit modules each delivered for two hours per week over one semester was converted into a set of 20-credit modules each delivered for two hours per week over one semester. In other words, credit-for-credit, the amount of contact time was halved. The linguistics curriculum team dealt with the situation in a number of various ways. In the case of relevance here, the ‘Language and mind’ module merged with the ‘Language and meaning’ module to become the ‘Language, meaning and mind’ module. My presentation describes how the module’s intended learning outcomes which were centred on a small-group presentation were addressed when, as a consequence of the halving of class time, the presentation had to be dropped.

Event Reports

Creative Assessment and Effective Feedback in Linguistics

Louise Mycock, University of Manchester

Session 1: Field Notebooks, Enquiry-Based Learning, and Assessment in Phonetics

Patricia Ashby, University of Westminster

Patricia Ashby began by discussing how to stimulate interest in a subject which students may view as being remote and difficult. She identified three needs which must be met if this problem is to be addressed:

• make the subject accessible
• enhance the learning experience
• improve results

Field Notebooks, as an example of Enquiry-Based Learning (EBL), are one way in which these needs may be met. Field Notebooks give students the opportunity to apply what they have learned in lectures and practical sessions week-by-week to data, as if they were undertaking fieldwork. As new concepts and skills are introduced in a lecture, tutorial, or laboratory session, students update their analysis of the data in the Field Notebook, enabling them to review course content and check their understanding. Assessment in at least two stages (at the mid-point and the end of the course) was recommended.

Feedback has been positive: students have a raised awareness of the progress they have made and their own responsibility for learning, the different assessment format has been welcomed, and statistics indicate encouraging upward trends in students’ marks. Though using Field Notebooks as a form of assessment requires a substantial commitment from teaching staff, in the case of the teaching of phonetics at the University of Westminster, the signs are that this investment has paid off. Patricia concluded by pointing out that Field Notebooks can be used in the teaching and assessment of a variety of other subject areas as well.

Session 2: “Do as we do not do as we say: research-led assessment of courses in social dialectology

David Britain, University of Essex

In the second session of the day, David Britain also identified accessibility as an issue in the assessment of linguistics, along with a number of institutional pressures.

He described how research-led assessment had been made a part of the sociolinguistics curriculum at the University of Essex. Two year-long courses on Methods in Sociolinguistic Dialectology were introduced, one for 2nd year and the other for final-year undergraduates. The 2nd year course encourages students to connect with the subject of linguistic variation by having them experience the theoretical, methodological and analytical stages involved in taking original research from design to publication. Students produce a research portfolio, elements of which are assessed at various points throughout the course.

In the final-year course, a different linguistic variable is examined from a variety of perspectives each week. Students work with corpora of non-standard data and are assessed at various points throughout the year. The assessed work takes a number of different forms, including a conference-style presentation of their findings and the design of a website on which their research report appears, and thus involves a range of transferable skills. The course is co-taught by a theoretical phonologist and a dialectologist – an example of interdisciplinarity in action, which serves to give students an insight into the relationships between sub-disciplines of linguistics.

Though time- and resource-intensive, research-based coursework is more interesting to mark, it enables students to develop a deeper understanding of what is involved in research, and they report a greater sense of achievement when they complete it.

Session 3: Distance teaching and assessment: developing online multimedia activities for MA phonetics and phonology students

Pamela Rogerson-Revell, University of Leicester

Pamela Rogerson-Revell outlined the aims, objectives, and outcomes of the mini-project “Leading innovation in distance teaching and assessment: developing online multimedia activities for MA phonetics and phonology students”, which was funded by the Subject Centre for Language, Linguistics and Area Studies.

The materials developed exploit open-source multimedia resources to facilitate the teaching and assessment of phonetics and phonology. One of the project’s key objectives is to produce a template for online resources which can be used or adapted by staff to create new online activities that will enhance their own courses. Pamela presented examples of activities that have been developed using this template by a variety of users with differing amounts of experience.

At the University of Leicester, online multimedia activities have been used in the summative and formative assessment of MA students on campus as well as distance learners. Formative assessment activities (usually multiple-choice or gap-fill quizzes) can increase the amount of feedback that students receive as they learn. Summative assessment involving the analysis of online speech extracts is especially useful in the case of distance learners, who might otherwise face technical or practical issues in acquiring spoken data of their own.

The advantages of these resources identified in staff and student feedback include:

• application of theoretical knowledge to authentic data
• increased choice for students
• easier access to/retrieval of language data
• increased variety of assessment methods

Sourcing and developing online multimedia activities is complex and requires significant effort, but students (and distance learners in particular) in a range of subject areas are likely to benefit from and appreciate access to online multimedia activities.

Session 4: Group project assessment

Tony Young, Newcastle University

In his session, Toby Young described how group research projects and presentations can be used to assess whether students have worked collaboratively and acquired related skills.

Students are provided with information about the assessment method and what to expect at the start of the module. Details are given of group size, the date of the presentation (after reading week), how long it should be, and the proportion of the final mark which it represents. Students are also told that marks will be awarded on a group basis and that they will be part of the audience for other groups’ presentations. A suggested timetable is supplied, along with advice on how to prepare the presentation and accompanying handout.

Each group must present a piece of research work (not necessarily original). Project topics are deliberately broad and linked to the entire module’s content. The groups usually have four weeks to prepare. After an initial supervised session, students are given time at the end of each class to discuss their project, but virtual as well as face-to-face meetings (via email, Blackboard, etc.) are encouraged.

Toby identified challenges which teaching staff have encountered and refinements that have been made, including:

• stressing the fact that working together effectively will be crucial to the success of the final presentation
• dividing students into groups
• having group members keep a record of their contributions
• providing explicit grading criteria
• giving worth to presentations by providing appropriate feedback
• having a contingency plan in place

Feedback from a number of sources (students, external examiners, etc.) has been very positive.

Session 5: Giving extra credit in half the time: managing teaching and assessment with reduced teaching hours

Paul Rowlett, University of Salford

Paul Rowlett described how intended learning outcomes which had previously been met by a small-group presentation were now addressed following changes to the teaching of linguistics to undergraduates at the University of Salford.

In the case which Paul discussed in detail, contact time was reduced and, as a consequence, there was no longer sufficient time for students to present group work in class. Instead, poster presentations are now an obligatory part of the assessment. As with the group presentation, students work together to present an issue from the primary literature within the wider context of linguistics to their peers, but now they produce a poster for assessment. They are provided with models and the criteria for marking in advance.

There are a number of similarities between the group and poster presentations. As methods of assessment, they both:

• represent diversity
• evaluate clarity and accessibility
• reflect the institution’s ‘real-world’ focus
• encourage reflective practice
• give students the opportunity to acquire transferable skills
• involve lecturers and students

One difference between the assessment of the group and poster presentations is the form which peer assessment now takes. Previously, group presentations were awarded marks separately by the lecturer and students, and the final mark was the average of the two. This method has its drawbacks though: for instance, students may mark each other more harshly than a lecturer would or give good marks to their friends. Now, two lecturers mark the poster presentations and students rank them relative to each other rather than giving marks to each group. The final mark is determined by comparing the lecturers’ marks and the students’ rankings.

Session 6: Discussion and feedback: key issues for assessment in linguistics

Chaired by Paul Rowlett, University of Salford

The final session provided an opportunity for discussion and feedback. Participants spoke about their own experiences and the day’s speakers took further questions about matters relating to topics they had covered. More general points concerning assessment in linguistics were also considered. These included:

• how to ensure that students complete coursework which is not assessed by teaching staff
• at which stage(s) and how often coursework should be assessed
• how to define success in group work
• the value of online multimedia activities as teaching tools to encourage reflection and deliver rich feedback
• resources that can be used to develop online multimedia activities

Mark my words: creative assessment and effective feedback in linguistics

Adela Gánem

The first session, “Field Notebooks, Enquiry-Based Learning, and Assessment in Phonetics” was led by Patricia Ashby. Patricia reported on an innovative form of assessment she has developed with the aim to a) encourage students to study phonetics, which is an area usually perceived as difficult, and b) improve student performance.

The assessment innovation in question is the ‘field notebook’. The theoretical underpinnings of this concept lie on ‘learning by discovery’ and ‘enquiry-based learning.’ The technique is based on the concept of helping learners become researchers in a guided and supportive environment. At the beginning of the course, each student is given a word or phrase which they will individually examine throughout the course. The main idea is to encourage students to incorporate their weekly understanding – derived from the lectures and weekly readings – to the study of their target structure. Every week, students are required to update their field records, which are then collected for assessment twice during term: ‘mid’ and ‘exit’ point assessment.

The effectiveness of this form of assessment appears to be positive judging from various perspectives:

1. Students’ feedback is encouraging and has been transformed. Prior to the ‘field notebook’ they used to complain about the amount of content they had to learn and the difficulty of the subject in general. Students’ experience of this form of assessment had led to positive remarks such as “It was nice to do something a bit different than a standard essay” and “I was amazed at the depth of knowledge I obtained through this course in such a short time.”
2. The average grades have generally improved over the years.
3. Recruitment levels for the subject have also improved.

Patricia believes this form of assessment has something to offer to other disciplines and, importantly, believes they provide a good opportunity to gain a good perspective on what students have actually learned. A crucial aspect of the ‘field notebook’ is that students are given ownership and responsibility for their learning. However, she emphasised the fact that this is a labour-intensive form of assessment, albeit in her view, well worth it.

The second session “Do as we do not do as we say”: research-led assessment of courses in social dialectology” was led by David Britain. The session shared the elements of field work and enquiry-based learning and assessment highlighted and praised by the previous colleague.

After providing a historical overview of curriculum changes in Sociolinguistics at the University of Essex, David described two new modules in this programme: Researching Language Variation and Language Variation and English Phonology. The methods of assessment for these modules reflect concerns to provide students with opportunities to acquire key transferable skills, give them the opportunity to receive early assessment, and are also guided by research-led teaching principles.

A detailed account of the procedures and activities which form the basis for the two modules was then presented before discussing assessment methods. These include a research portfolio consisting of: a survey, recording and transcription of conversational speech, contextualisation of data, some data analysis, and a 5,000 word research paper for the first module. For the second module, students are assessed by means of an analysis of data available on the Departmental corpus of conversational speech, the oral presentation of their analysis, and the construction of a website to present their research report.

Finally, David discussed future development and problems with this method of assessment. Among the most important issues highlighted was the fact that due to the cumulative nature of skills development and assessment, students who tend to miss classes or do not keep up their weekly work find it very difficult to deliver at the end of term. A second issue is that marking is more time-consuming albeit more interesting.

The final session before lunch was led by Pamela Rogerson-Revell and was entitled: “Distance teaching and assessment: developing online multimedia activities for MA phonetics and phonology students.” Pamela demonstrated some interesting multimedia on-line materials developed with a two-fold aim: on the one hand, to enhance text-based materials for students who follow their degree in a traditional way, i.e. on campus at the university and, on the other hand, to provide distance MA in Applied Linguistics and TESOL students with the opportunity to work autonomously on phonetics and phonology which have been historically difficult to deliver online.

The second part of the workshop consisted of two complementary sessions which described a course which has successfully integrated group oral presentations as part of their assessment methods at Newcastle University on the one hand and, and on the other, a course where time restrictions imposed by curriculum changes led to substitution of assessed oral presentations by assessed poster design at the University of Salford.

In the first instance, the group presentations are reported to be successful in encouraging bonding among the class and provide opportunities for participants to develop transferable skills among other benefits. During this session – and among other interesting issues - Tony Young discussed the marking grid used for the presentations. This is a very detailed tool which includes a comprehensive set of criteria upon which students are evaluated such as: Presentation Context (coherence, relevance, sing-posting, reflection, originality, etc.) and Presentation Skills (use of language, visual aids, non-verbal behaviour, voice, hand-outs, etc.). The criteria for evaluation led to a lively discussion. Once again, the incorporation of assessment practice which is not solely based on essay writing appears to be motivating and valuable for students, but requires a greater effort made by teachers and markers who also need to invest more time on marking and feedback provision.

Finally, Paul Rowlett presented a case where a restructuring of the linguistics programmes at the School of Languages at the University of Salford led to a fundamental change in the amount of contact hours per subject, which also had an impact on assessment methods. While historically students had been assessed on the basis of a small-group presentation, staff had to resort to poster design as part of the assessed components of the course in order to accommodate the changes in the curriculum. Paul considered the substitution a successful compromise and showed us a sample of posters created by his students, which led to our final workshop discussion. There appeared to be a general consensus about the value of poster design to foster skills such as ability to summarise and express core ideas in a concise way; ability to create attractive visual materials to summarise research papers; and the opportunity for students to develop creativity.

All in all, an interesting and fruitful exchange of ideas about assessment and feedback options in linguistics.

Creative assessment and effective feedback in linguistics

Nadya Yakovchuk, University of Leicester and University of Warwick, UK

Session 1: Field notebooks, enquiry-based learning, and assessment in phonetics

Patricia Ashby, University of Westminster

Patricia Ashby started off by acknowledging the fact that the subject of phonetics does not often appeal to arts and humanities students because they tend to find it too scientific, and those who opt for it do not usually do very well in assessment. She identified three needs that have to be met in order to solve this problem: making the subject user friendly and more accessible, enhancing the student learning experience and improving results (without lowering standards).

Patricia responded to these needs by introducing Field Notebooks into her teaching of phonetics to undergraduates at the University of Westminster. Students are given individual utterances (short words or phrases) of phonetic interest, and they are supposed to update the phonetic description of their utterance (e.g. by using transcription diagrams and articulatory accounts) on a weekly basis, following their weekly advancement in the acquisition of phonetics. Field Notebooks are a form of Enquiry-Based Learning and they encourage students to engage with the lecture material regularly during the whole course of study. In her 12-week study block, Patricia assesses student Field Notebooks at two stages: mid-point assessment in week 6 and exit point assessment in week 12. Although currently these Notebooks are word-processed, the work is under way to introduce blog format Notebooks.

Although Field Notebooks are labour-intensive in terms of assessment, the statistics suggest that they seem to contribute to an improved understanding of the subject knowledge and the module fail rate has dropped. Student feedback suggests that they enjoy the learning experience, find it more engaging, and feel a greater sense of achievement and responsibility for their learning.

At the end of her session Patricia pointed out that Field Notebooks are not only restricted to phonetics and can be effectively applied to other subject areas.

Session 2: ‘Do as we do not do as we say’: research-led assessment of courses in social dialectology

David Britain, University of Essex

David Britain’s presentation focused on the assessment of individual project work on undergraduate social dialectology courses at the University of Essex. This form of assessment was introduced to facilitate student appreciation of and engagement with the process of research into language variation and change, and at the same time to meet the requirements set by the University (i.e. embedding key transferable skills into the curriculum, devising early assessments, diversifying assessment methods, engaging in research-based teaching, etc.).

The project work described by David requires students, depending on their course level (second or final year), to go through various stages of the research process individually: literature search, literature review, data collection, transcription, analysis, interpretation and presentation of results. For second-year students, evidence-based assessment is used in the form of a portfolio, the evidence for which is collected throughout the course starting from week 2. Final assessment for both levels consists of an oral presentation and a text-based component (a ‘journal article’ for second-year students and a self-designed website for finalists).

Introducing project work has resulted in increased student interest in the subject, with more students taking up final year research projects in this area. As coursework is more spread out throughout the year, students engage with feedback more regularly. Their learning becomes more meaningful as they are able to connect theory with practice.

From the staff perspective, marking research work is more time-consuming (but more interesting at the same time), and developing the original curriculum initially takes a lot of time. David also mentioned ‘logistical’ problems with on-line submission of audio-data and with using large data repositories within the university system so that speech corpora collected by students can be kept and retrieved on demand, but these problems can be minimised through the digitalisation of data.

Session 3: Distance teaching and assessment: developing online multimedia activities for MA phonetics and phonology students

Pamela Rogerson-Revell, University of Leicester

Pamela Rogerson-Revell reported on a project funded by the LLAS Subject Centre which aimed at developing online multimedia activities to enhance the teaching and assessment of phonetics and phonology on the distance learning MA course. Although primarily developed with distance students in mind, these online activities can also be used for the benefit of campus-based students.

The online activities were developed with the help of the Course Genie software and included interactive activities, animated diagrams (essential in the teaching of phonetics), transcription practice, etc. The activities can also be printed out and used in the paper format. The online mode also led to re-designing assessment strategies, with numerous interactive activities used for formative assessment throughout the course (e.g. multiple-choice quizzes), and open-access online materials used as data for the final part of the timed exam for campus-based students (via Blackboard) and as part of written assignments for distance students.

Although Pamela acknowledged that finding and developing multimedia learning activities is not easy and can be very time-consuming for staff, such efforts should pay off as multimedia materials are proving to be particularly useful for distance students, the online mode allows staff to diversify assessment methods, and students appreciate close engagement with real data which allows them to put their theoretical knowledge into practice.

Session 4: Group project assessment

Tony Young, University of Newcastle

Tony Young discussed group research project work which is used on two Master’s modules, ‘Professional Communication Skills’ and ‘Social Psychology of Communication’, at the University of Newcastle. It was introduced in an attempt to re-focus assessment on a number of important Intended Learning Outcomes (i.e. teamwork, interpersonal skills, problem solving, etc.) which were not flagged through the traditional assessment methods.

Students are supposed to work in groups researching a particular topic within the subject area of the module they are following. This work results in a group presentation on the basis of which they are assessed (it accounts for 40% of the final mark for the module). Project work starts early in the module and students are normally given 4 weeks to do their research and prepare for the presentations (which take place half-way through the module). Presentations are recorded and assessed by two staff markers using a very detailed marksheet designed specifically for this purpose (50% of the mark is given for the content and 50% for presentation skills). Throughout the work on the project, self-policing of group members is encouraged (again, using a set of specially designed guidelines, which also assist in project planning and management).

The feedback received from students is largely positive and they particularly appreciate a detailed explanation of the procedure and the requirements for the project, and also the specific grading criteria used for assessment. External examiners have positively commented on the fact that this project work addresses some usually neglected Intended Learning Outcomes explicitly.

Session 5: Giving extra credit in half the time: managing teaching and assessment with reduced teaching hours

Paul Rowlett, University of Salford

Paul Rowlett talked about the change in the teaching and assessment on the new ‘Language, meaning and mind’ module (amalgamation of the ‘Language and mind’ and ‘Language and meaning’ modules) in the School of Languages at the University of Salford, which followed the re-structuring of undergraduate degrees within the School in the early 2000s.

The changes in the linguistics curriculum meant that the amount of class time on the module was halved. This reduction in contact time meant that using small-group presentations as part of the module assessment was no longer feasible. The presentations had been based on an article from primary literature and assessed by the lecturer and fellow students, with written qualitative feedback provided by the lecturer.

As assessment regimes had to be rationalised, the solution adopted on the ‘Language, meaning and mind’ module was to replace group presentations with poster presentations. Student posters still had to be based on a core article and the same assessment criteria were used. Poster assessment was carried out by two lecturers who provided a mark and by fellow students who indicated ranking. At the end of the session Paul Rowlett showed the examples of posters designed by his students.

Session 6: Discussion and feedback: key issues for assessment in linguistics

In the first half of this session, Alison Dickens, the senior academic coordinator for Linguistics in the LLAS Subject Centre, talked about the resources on assessment and feedback available on the HEA website. She also informed the participants about the LOC (Learning Object Creator) software developed by the Subject Centre and the Department of Modern Languages at the University of Southampton. LLAS runs regular workshops for those interested in learning how to use this tool. Alison emphasised that LOC is a teaching rather than a testing tool.

In the second half, the participants exchanged final thoughts and opinions on assessment in linguistics.