Learner training: From strategy awareness to actual language improvement

Authors: Andrea Wilczynski and María Fernández-Toro

© Andrea Wilczynski and María Fernández-Toro, University of Newcastle upon Tyne

Table of contents

• Introduction
• The learner training programme
• Students' strategy choices for productive skills
• A typical writing strategy: back translation
• A typical speaking strategy: creating a role-play
• The lessons to be learned
• Bibliography

This paper was originally presented at the Setting the Agenda: Languages, Linguistics and Area Studies in Higher Education conference, 24-26 June 2002.

Introduction

This paper aims to demonstrate how an observational approach to strategy research could be developed on the basis of student performance data. The data presented here was compiled at the end of the academic year (i.e. only a few weeks before this paper was given). Therefore it should be regarded as a preliminary communication rather than hard evidence of specific findings. Nevertheless, it was thought that an early glimpse into the nature of the information that can be obtained by this method could be of use to other researchers in the field, and might generate fruitful discussion at this initial stage.

Two questions will be addressed. The first one is of a pedagogical nature and examines to what extent explicit strategy training facilitates independent learning. Productive skills constitute a particular challenge when promoting independent learning. Because such skills are more likely to elicit open-ended responses from the learners, speaking and writing are regarded as the areas in which independent learners are likely to need most support in order to make up for the absence of ad-hoc teacher feedback. Hence the authors' choice to focus on speaking and writing.

The second issue is related to research. Traditionally, most strategy research studies rely on data collected by means of introspective methods, such as retrospective questionnaires, think-aloud techniques, or learner diaries. Useful as such methods may be to give us an insight into what learners perceive to be doing, it has also been argued that the information obtained can only be subjective and does not necessarily reflect the learners' actual behaviours and performance (Seliger 1983). This study proposes that an observational approach is a viable alternative and examines what kind of information can be gained from an observational study of students' performance on a strategy-training module.

The learner training programme

The Language Learning Skills module is a credit-bearing module taught at Newcastle University. It aims to promote independence by presenting students with a wide range of learning strategies, from which they try out and evaluate a selection of their choice. It runs for 12 consecutive weeks (1 hour plenary session per week). Students are normally non language-specialists and can take the course at any stage of their degree. The module is non language-specific (learners of French, Spanish and German attend the same plenary sessions), but language-specific support is available in the form of short one-to-one feedback sessions with a teacher of the relevant language (normal average: 10 minutes per week). It is also non level-specific (from beginner to post A-level).

The syllabus first covers individual learning styles and needs analysis techniques and is then structured around six major areas/skills: vocabulary, grammar, listening, reading, speaking and writing. A similar structure is followed in the course book used - DIY Techniques for Language Learners (Fernández-Toro and Jones 2001) - , where chapters 6 to 11 are each related to one of the areas/skills listed above. Each chapter begins with a general introduction to the skill in question (e.g. 'What does learning to listen in a foreign language involve'), followed by a list of 10 to 15 'DIY Techniques' (learning strategies). In line with Brown and Palincsar's recommendations (Brown and Palincsar 1982), each 'Technique' entry comprises three sections:

1. 'What is it good for?' (rationale and relevance of the strategy to specific individual needs)
2. 'How to proceed' (step-by-step directions on how to apply the strategy to a specific task)
3. 'How to assess progress' (guidance for self-assessment, taking into account the absence of a teacher)

Each week, the plenary session is devoted to one skill. The students then read the relevant chapter in their own time and choose three 'techniques' from the book. They must try each one out (this normally requires a search for suitable input material), assess their own preformance, evaluate the strategy itself and show evidence of the whole process in the form of a series of submissions, which are all formally assessed:

• By the end of the course, a student's portfolio comprises evidence of having tried out 18 different strategies (three for each of the six areas/skills).
• The last two weeks of the course are devoted to a short project in which students set themselves one specific goal (e.g. 'learn 100 words related to topic X and use them correctly in context', 'understand and summarise 6 news items from the radio', 'learn all regular and irregular verb forms for tenses X, Y and Z', and so on). They must then choose and apply strategies that are relevant to their goal, assess their level of achievement and evaluate the strategies used in the two-week study plan.
• Students are also required to produce an extended report evaluating their experience over the 12 weeks of the module.

Students' strategy choices for productive skills

A total of 17 students took the module in 2001-2002 (10 Spanish, 4 French, 3 German), of which 8 were at beginner/elementary level (pre-GCSE), 5 intermediate (post-GCSE) and 4 advanced (post A-level).

The following list shows the strategies that were chosen under the 'writing' and 'speaking' components of the module. The numbers indicate how many students chose each strategy. Complete descriptions of the strategies listed below can be found in Fernández-Toro and Jones (2001).

The strategies chosen under 'writing' were: back translation (13), summary writing (7), letters and e-mail (5), linking sentences and paragraphs (5), creating a gapped text (4), dictation (4), writing a pastiche (3), rewriting a text (3), songs and poems (2), paragraph expansion (2), keeping a diary (1), editing a transcript (1), creating a gapped text (1).

The strategies chosen under 'speaking' were: reading aloud (10), creating a role-play (9), imaginary chats (8), don't look (7), recording yourself (5), learning a song (5), listen and repeat (4), arts review (2), pronunciation drills (1), interrupting your partner (1), outspeaking your partner (1).

A typical writing strategy: back translation

This paper will examine students' performance on back translation, which was the most frequently chosen strategy for writing skills. The principle of back translation is to translate a carefully chosen text from L2 into L1, and then use the L1 as stimulus to rewrite the text back into the L2. Learners can then assess their performance by comparing their final version in the L2 against the original text. For the purpose of this module, students were required to find suitable texts in L2, apply the technique and, in addition, complete a self-assessment questionnaire (Fernández-Toro and Jones 2001: 113-117) and produce a detailed analysis of all the discrepancies found between the L2 original and their own version.

A set of operational criteria was developed in order to facilitate comparisons of student performance across the sample. The students' work was assessed separately by two experienced language teachers (the two researchers) using the same criteria; which are listed below.

1.Text length
The texts chosen by students for back translation had an average length of 171 words. Text length was not found to be related to the students' proficiency level in the target language.

2.Source of texts
Beginners and other lower level students experienced some difficulty in finding appropriate texts for their level, and chose extracts from course books or other materials supplied in their language class. Intermediate and advanced students explored the media or the internet, using predominantly authentic material.

3.Difficulty for level
The two independent raters examined the 13 cases in which students had used back translation, and rated the difficulty of the chosen source texts according to the students' proficiency levels as follows: 'about right' for the student's level: 9 cases; 'a bit too easy': 3 cases; 'a bit too difficult': 1 case.

4.Discrepancies between student's L2 version and the original
The back translations completed by students were compared to the source texts and the total number of discrepancies between the two texts was counted for each student. In order to allow comparisons across the sample, the average number of discrepancies per 50 words of original text was also calculated for each student. The findings showed a wide variation in the number of discrepancies found across the sample:

• 10 discrepancies or more: 3 cases
• between 6 and 9: 3 cases
• between 3 and 5: 6 cases
• 2 discrepancies or less: 1 case

These scores did not appear to relate to the students' proficiency level.

5.Proportion of discrepancies noticed by students
Although the book explicitly instructed learners to use the original test in order to asess their performance, only 9 out of 13 performed the comparison themselves with the following results:

• 4 students identified all the discrepancies produced
• 4 students identified two thirds of the discrepancies produced
• 1 student identified one third of the discrepancies produced.

The remaining 4 students failed to do so, and asked the tutor to correct their mistakes instead. Two of those went on to produce an excellent follow-up analysis of the teacher's correction (see next). This suggests that even these students gained some benefit from the exercise as a whole. Again, the students' success in locating mistakes did not appear to be related to their proficiency level.

6.Mistakes analysis
Only six out of 13 students produced an analysis of discrepancies as requested. Both researchers examined the work independently, and classified the analyses into 'very good', 'decent attempt', 'poor' and 'not submitted'.

Below are two examples taken from the students' own analyses of their mistakes:

Example A: French (Advanced)

• Mistake: les mères
• Type of mistake: language
• Correction: des mères
• Justification: 'trouver' is followed by 'de'

This example would be rated as a 'poor attempt' of mistake analysis: the student does not present the mistake in context, 'language' (under 'type of mistake') is far too vague a category, and the statement provided by the student under 'justification' is simply wrong.

In comparison, Example B provides a sound explanation of the mistake that was made. The student has understood her problems and made corrections accordingly. This kind of analysis would be rated as 'very good'.

Example B: Spanish (Beginners)

• Mistake: ...exclusivamente se dedica a las tareas dom...sticas:... y al cuidado de el niño
• Correction/justification: I used English word order in this case. In Spanish adverbs are mentioned after the verb not before hence it should be 'se dedica exclusivamente'.:In Spanish the preposition 'de' followed by the masculine 'el' is brought together to 'del', whereas the feminine is not. It remains 'de la'.

Students who attempted an analysis did generally well (6 rated 'very good', 1 'decent attempt', 1 'poor'). This general success could indicate that learners who are given the opportunity to choose a source text for themselves are in a better position to recognise and understand their own mistakes.

7.Use of self-assessment scales
Although self-assessment questionnaires were made available to all 13 subjects, these scales were only used by 4 students. The questionnaires allowed users to add supplementary criteria to those proposed on the checklist, and in all four cases, the subjects added one task-specific entry in order to cover 'translation'.

A typical speaking strategy: creating a role-play

The most frequently chosen strategy under 'speaking' was reading aloud, because it was the easiest one to set up independently (no interlocutor required). However, it was thought that a task that elicits open-ended speech production would be more likely to confront learners with the problems related to lack of external feedback, one of the main issues raised in this paper. Therefore this preliminary study focuses on the second most frequently chosen strategy: creating a role-play.

1.Difficulty for level
The guidelines stated clearly that the purpose of this strategy was to prepare for a new, unfamiliar situation. In spite of this, only one student used the strategy for that purpose, while the rest focused on situations that had already been covered in their courses.

Where two students at different levels were creating a role-play together, the more advanced student ended up working below her actual proficiency level in order to accommodate for her partner's limitations. Overall, 6 out of 9 students designed role-plays that were either rated as 'a bit too easy' or 'far too easy' for their level.

2.Preparatory work
Despite detailed instructions on how to create a role-play, these were not always followed by the students.

Four students produced an outline or a list of cues or steps in L1 as shown in Examples C and D below.

Example C:

• You rang earlier in week + were told to come and see the lecturer
• You left school at 16 + now want to return to education to get a job
• You think you'd like to do sth with computers
• [lecturer tells you about courses on offer]
• Ask her about intermediate computing: times, dates, are there any assignments, how is the course examined, how much does it cost
• Tell her you would like to enrol for the course
• [you need Spanish GCSE].
• You tell her you failed all your exams at school, but you will work because you want to do the course and you can speak Spanish.

Example D:

Two students prepared a full script of the dialogue in L2 beforehand, and three students failed to submit any evidence of preparatory work whatsoever.

3. Evidence of genuine interaction
Recordings of the role plays were assessed in order to establish the extent to which the learners had used on-line speech production (as opposed to reading a prepared script) and genuine interaction. In 7 out of 9 instances, they were found to use a 'reasonable amount' of genuine interaction.

4. Follow-up
Students were requested to follow up the role -play by writing a verbatim transcript of their contribution, which was to be the basis of self-assessment. The final requirement was a detailed analysis of all the mistakes found by students in their own transcripts (much like the mistakes analysis requested for the back translation exercise).

Only 3 students out of 9 produced actual transcripts (two of which also showed the necessary corrections). Three students did not produce any follow-up whatsoever. The rest (3) produced a script of the dialogue as planned before the task was recorded (as opposed to a transcript of the actual performance). The researchers found that students did not always specify clearly whether the text that they were attaching was a transcript of their actual performance, or simply a written script that had been written prior to the recording. Such confusion shows that students are not fully aware of the relevance of on-line speech production in a task of this kind.

5. Mistakes analysis
Of the three students who wrote down full transcripts of their performance, only one produced an analysis of the mistakes made, which was independently rated as 'excellent' by both researchers. Another student simply indicated the necessary corrections with no further analysis.

6. Use of self-assessment scales
All students were provided with a self-assessment questionnaire (Fernández-Toro and Jones 2001:131-136) similar to the one that was used for writing tasks. In contrast with the poor use made of self-assessment questionnaires for back translation, nearly all students (8 out of 10) used them in order to assess their role-plays. This could be due to the fact that in this task students had no way of receiving objective feedback on their performance, whereas the back translation exercise enabled them to compare their final text with the original and thus correct and analyse their own work without further support.

The lessons to be learned

Two questions were raised at the beginning of this paper. The first, pedagogical, issue related to the effectiveness of an explicit strategy-training programme in promoting the use of independent learning strategies. For the two cases that are considered in this preliminary study (back translation and role-play), the following comments can be made:

• Students can indeed learn to choose resources and strategies appropriate to their needs, but at beginner level they need additionnal guidance in order to find suitable input material.
• They can learn to apply specific strategies effectively, although the level of success may vary. Such variations do not appear to be related to individual levels of proficiency, but to other variables such as (possibly) motivation and previous learning experience. While some students were quite happy to invest considerable effort in trying out each strategy, others tended to regard the 'time-consuming' nature of certain tasks as a disqualifying feature, and rated strategies requiring less effort as 'better' strategies. Language acquisition theories such as Craik and Lockhart's levels of processing (Craik and Lockhart 1972) suggest the opposite, and there may be a need for further instruction on the long-term benefits of 'time consuming' tasks.
• Self-assessment was found to be the students' weakest area, as shown by the consistent omission of follow-up strategies. A revised version of the module would clearly need to incorporate better provision for explicit training in self-assessment strategies.

The second question related to the viability of using student performance data for observational strategy research. Although the research is still in embryonic stage, this paper proposes a number of features of students' performance that can be easily observed. Furthermore, the course book used in the module presents a list of 82 learning strategies that are clearly described in the form of precise step by step procedures. Detractors of the book could reasonably argue that the range of strategies is not as comprehensive as it could have been, or that the balance of skills should be different, or that the strategies listed under one heading ought to relate to a different skill. Nevertheless, and however imperfect it may be, the material provided does ensure that the input given to all students is identical, so that researchers can compare student performance using a standardised instrument. A recurrent problem in strategy research is precisely the lack of a standardised taxonomy of learning strategies (Rees-Miller 1993) beyond the recognised categories of metacognitive, cognitive and social-affective (Brown and Palincsar 1982; O'Malley and Chamot 1990). The present study demonstrates that it is however possible to define an operational framework for the study of learning strategies, so that an observational approach can be applied.

The next logical step ought to be a systematic study of the data obtained for each of the strategies chosen by students under each of the six skills that are covered in the module. It is hoped that such a battery of studies will give us a better understanding of the kind of of strategy training that is worth pursuing in terms of actual language achievement.

Bibliography

Brown, A.L., and A.S. Palincsar (1982). 'Inducing strategic learning from texts by means of informed, self-control training'. Topics in Learning and Learning Disabilities 2 (1):1-17.

Craik, F.I.M., and R.S. Lockhart (1972). 'Levels of processing: A framework for memory research'. Journal of Verbal Learning and Verbal Behaviour 11:671-84.

Fernández-Toro, M., and F.R. Jones (2001). D.I.Y. Techniques for Language Learners. London: CILT.

O'Malley, J.M., and A.U. Chamot (1990). Learning Strategies in Second Language Acquisition. Cambridge: Cambridge University Press.

Rees-Miller, J. (1993). 'A critical appraisal of learner-training: theoretical bases and teaching implications.' TESOL Quarterly 27 (4):679-89.

Seliger, H.W. (1983) 'The language learner as linguist: Of metaphors and realities.' Applied Linguistics 4:179-91.