The effectiveness of written feedback on writing has been extensively investigated. Although the debate concerning the most effective approach to feedback on writing still continues, giving feedback on student writing can considered an important part in the development of writing skills in a foreign language. Typically, previous research has explored the effectiveness of feedback provided using the traditional pen-and-paper method, that is, adding brief handwritten comments on a student paper, such as underlining or coding of grammatical errors, sometimes accompanied by short explanations. Naturally, text processing software provide similar features that allow commenting student texts. Non-corrective approaches to feedback, in which the students themselves try to correct the errors in their text based on clues or comments, have shown great promise in improving writing skills. In general, the more detailed the feedback, the more time it requires from the teacher to produce the feedback. A more detailed feedback and concrete examples help students to correct any errors in their texts more reliably. However, for the teacher, writing, typing and updating of the explanations is time consuming and reuse complicated. A more recent approach to providing feedback on writing is to use a text commenting tool which allows reuse of extended commentary, examples and weblinks to student texts. However, only a few studies have investigated the effectiveness of rich feedback currently allowed by such text annotation/feedback tools. This paper investigates the success rate of undergraduate and graduate engineering students in revising their own texts after receiving rich written feedback provided by the teacher using an online commenting tool. More specifically, this paper aims to determine: (1) to what extent students are able to revise their text according to teacher feedback?; (2) what kind of feedback helps students successfully revise their texts?; (3) what kind of feedback is difficult for students to revise?; (4) how the use of such a tool influences teacher workload and the quality of the given feedback? The data included in this study contains an analysis of several hundred individual teacher feedback comments and the corresponding revisions by students. The data was collected from a set of basic undergraduate engineering writing courses with a main focus on academic and scientific writing style. Author(s): Jan-Mikael Rybicki (Aalto University) & Juhana Nieminen (Aalto University)

1. Improving revision success with
written feedback using an online
commenting tool
EUROCALL 2013, Évora, Portugal
14 September 2013
Jan-Mikael Rybicki / jan-mikael.rybicki@aalto.fi
Juhana Nieminen / juhana.nieminen@aalto.fi
Aalto University Language Centre, Finland

2. Providing written feedback to students is
one of the ESL writing teacher’s most
important tasks, offering the kind of
individualised attention that is otherwise
rarely possible under normal classroom
conditions.
Hyland, F. and Hyland K. (2001). Sugaring the pill
Praise and criticism in written feedback.
Responding to student writing is one of
the most challenging aspects of the
writing instructor’s job, and it is certainly
the most time-consuming.
Ferris, D. (2007). Preparing teachers
to respond to student writing.

3. Overview
1. Previous studies on writing feedback
2. Research questions
3. Data and methods
4. Results
5. Conclusions

4. (1) Previous Studies on feedback
• Process approach (Krashen, 1984; Zamel, 1985)
• Continuing ESP and EAP traditions (e.g., Ferris, D. and
Hyland, K.)
• Feedback = pen and paper or face-to-face
conferencing
• Resources-Rich Web-Based Feedback (Milton 2006)
What are good practices?

5. Previous studies
Ferris, D. (2006). Does error feedback help student writers?
• Used error codes (15 different):
WC (word choice), SP (spelling), VT (verb tense), Art
(articles), etc.
• Indirect feedback recommended
Feedback type Correct revision Incorrect
revision
No change
Direct 2238 (88%) 107 (4%) 214 (8%)
Indirect 1759 (77%) 305 (13%) 231 (10%)
Non-log 409 (75%) 86 (16%) 52 (9%)
Incorrect 38 (62%) 14 (23%) 9 (15%)
Uncessary 164 (80%) 17 (8%) 25 (12%)

6. Previous studies
Types of students:
• Non-native 1st-year university students, mainly
Asian (Ferris 2006)
Types assignments:
• Essays, personal experiences
Engineering students
• Problems understating feedback (Taylor, S. 2010)

7. (2) Research questions

8. (2) Research questions
1. to what extent students are able to revise their text
according to teacher feedback?
2. what kind of feedback helps students successfully
revise their texts?
3. what kind of feedback is difficult for students to
revise?
4. how the use of such a tool influences teacher
workload and the quality of the given feedback?

9. (3) Data and methods

10. Data: Course information
Kie-98.1112 Writing in Technology (2 ECTS)
• Fulfills the university requirement for foreign
languages in English (written skills)
• CEFR-level: B2-C1
• Focuses on basics of professional and
academic writing style
• 26 hrs contact + 28 independent study
12 weeks

11. Data: Course participants
• Undergraduate engineering students at Aalto
University
• Primarily native Finnish speakers
Course:
Kie-98.1112
Males Females Total
H8 Spring 2011 19 1 20
H6 Spring 2012 13 4 17
H2 Autumn 2012 7 5 12
TOTAL 39 10 49

12. Data: overall assignment statistics
Course:
Kie-98.1112
Assignment type Number
of texts
Length in
words
H8 Spring 2011 Problem-solution text 20 200-300
H6 Spring 2012 Extended definition 17 200-300
Recommendation 17 500-1000
H2 Autumn 2012 Extended definition 10 200-300
Total
(of those who completed
the course)
64

13. Methods: Feedback procedure
1. Students submit their texts into
KungFu Writing (KFW)
2. Teacher adds comments to these
texts using KFW and submits them
to students
3. Students revise their texts based on
feedback and resubmit the texts
into KFW

14. Methods: Teacher interface

15. Methods: Student interface

16. Methods: Research question 1
1. to what extent students are able to revise
their text according to teacher feedback?
Revision success
was classified as
follows:
Positive
feedback

17. (4) Results

18. Results Q1: ALL comments

19. Results Q1: Error feedback only

20. Results Q1: Error feedback only
Ferris 2006
Feedback type
Correct revision Incorrect revision No change
Direct 2238 (88%) 107 (4%) 214 (8%)
Indirect 1759 (77%) 305 (13%) 231 (10%)
Non-log 409 (75%) 86 (16%) 52 (9%)

21. Research questions 2 and 3
(2) What kind of feedback helps students
successfully revise their texts?
(3) What kind of feedback is difficult for
students to revise?

22. Results Q2 and Q3
Revision success:
Yes partial success
no attempted
no ignored
no deleted

23. Results Q2 and Q3
Total feedback per category
(including positive feedback)Problem categories
Linear
progression

24. Results Q2 and Q3:
Effect of example phrases
A couple of examples
• students copy verbating
• little thinking
• ”correct” revision
Many examples or link to a website
• variation in student responses
• requires more reading and thinking
• more incorrect revisions

25. Results Q2 and Q3:
case ”SO” and ALSO”

26. Results Q2 and Q3:
case ”SO” and ALSO”

27. Results Q2 and Q3:
case ”SO” and ALSO”

28. Results Q2 and Q3:
case ”SO” and ALSO”

29. Research question 4
How the use of such a tool influences teacher
workload and the quality of the given feedback?

30. Results Q4

31. Evaluation: Research Q4
Disadvantages
(-) Takes more time than simple underlying or
use of error codes on paper
(-/+) Teacher must spend time preparing
templates and planning feedback strategies
(particularly at the beginning)

32. Evaluation: Research Q4
Advantages
(+) Takes less time than written feedback
combined with face-to-face conferencing
(+) increased quality: students have an
opportunity learn the underlying rules through
well-planned explanations and examples
 Process writing

33. (5) Conclusions

34. (5) Conclusions
• Students can revise the ”errors” based of KFW
feedback at least as well or slightly better than
using paper-pen and error code methods
• Electronic feedback allows improved
explanations
• Analysis of feedback and students writing
convenient with electronic documents

35. Future plans
• Adding more analysis data
• More detailed analysis of feedback results
• Plan to continue analyzing the feedback data
collected from master’s level writing courses
(Kie-98.1310 and Kie-98.1320)
– Mainly international students
– Longer texts

36. References
Ferris, D. (2006). Does error feedback help student writers? New evidence on the short-
and long-term effects of written error correction. In K. Hyland and F. Hyland
(Eds.). Feedback in Second Language Writing. Contexts and Issues.
Cambridge Applied Linguistics. Cambridge: Cambridge University Press, 185-
205.
Ferris, D. (2007). Prepating teachers to respond to student writing. Journal of Second
Language Writing 16: 165-193.
Goldstein, L. (2006). Feedback and revision in second language writing. In K. Hyland
and F. Hyland (Eds.). Feedback in Second Language Writing. Contexts and
Issues. Cambridge Applied Linguistics. Cambridge: Cambridge University
Press, 185-205.
Hyland, F and Hyland K. (2001). Sugaring the pill Praise and criticism in written
feedback. Journal of Second Language Writing, 10 (2001) 185–212
Krashen , S. D . (1984). Writing: Research, theory and applications . Oxford :
Pergamon Press .
Taylor, S. (2011). ‘‘I Really Don’t Know What He Meant by That’’: How Well Do
Engineering Students Understand Teachers’ Comments on Their Writing?.
Technical Communication Quarterly, 20(2), 139–166
Zamel, V. (1985). Responding to student writing. TESOL Quarterly, 19 (1), 79–101.