feedback

About a year ago, Dave Lee at Learning Circuits Blog wrote on why the Help Desk and Customer Service in a company are the best at helping employees or customers learn. His reasons included:

  • generally don't teach courses
  • answer the questions their "learners" have at the time they are in greatest need
  • assess each situation and determine the best course of action to answer the learner's need
  • provide the learner with only the answers they need to overcome the current problems
  • have a database of answers to questions that have been asked before and will likely be asked again
  • have a mechanism for escalating the response when the learner's need is of greater breath than can be resolved immediately.
  • track very specific metrics regarding performance and learner response.
  • often have a follow-up mechanism to determine whether their solution to the learner's need is still working several weeks later and to gain feedback from the learner regarding their experience with the group
  • feed overall questions and needs back to the stakeholder groups who can take action regarding the issue to mitigate the same problem in the future.
  • gather feedback from their stakeholders regarding their effectiveness.
  • push as much of the process and answers pro-actively to their learners in the form of knowledge bases and FAQ as well as anticipatory actions like alerts and job aides.

Quite a few people disagreed with him, saying that many help desks weren't all that good. Even so, note that most of his reasons are associated directly, or indirectly, with feedback: answering questions, tracking performance and responses, follow-up mechanisms, stakeholder feedback (two-way). As noted in "Flow, Games, and Learning", feedback, especially when immediate, is a crucial element of obtaining a state of flow, of intrinsic motivation, especially when that feedback is immediate, or just in time.

Note also that the learners are actively participating in a meaningful process: asking questions, reading knowledge bases, using the information toward their immediate goals.

We could continue analyzing this list and seeing learning "best practices". Isn't it interesting that parts of a company can come up with "best practices" of learning without having studied educational theory? Might it be that business survival pressures can lead to learning systems that work? And when educational systems don't work, might it be that they don't have enough pressure to change. I'm not suggesting that schools should become businesses. Our purposes are different. Still, perhaps we can learn from business "best practices."

With respect to pressure, one difference between businesses and schools is that both the help desk and the customers have a more pressing need to learn than students do, and they have reasons for learning answers to specific questions that students don't have. Take introductory biology, for example. Have you ever used the Krebs electron cycle once in your daily life? At Work? Learning in school is not just-in-time necessary learning: It's learning for possibly (or probably not) necessary future endeavors.

The structure of school "learning" works against facilitating intrinsic motivation. Although re-structuring traditional schools is unlikely, one approach would follow Roger Shank's story-tellling curriculum:

The idea behind the Story-Centered Curriculum (SCC) is that a good curriculum should consist of a story in which students play a key role (for example, VP of Information Security at a financial services company). These roles are selected to be ones that the graduate of such a program might actually do in real life or might need to know about (because he or she will manage or collaborate with someone who performs that role). Students, working in groups, are given detailed information about the simulated company they are working for together with detailed and authentic projects. Supporting materials and resources are available and experts and online mentors are available to answer questions and point students in the right direction on an as-needed basis.

The effect of the SCC model is that as students work through the story to achieve the missions the story puts forth, they learn the critical skills they need to successfully accomplish their tasks. The SCC implements true learning-by-doing, integrating all aspects of real-world tasks, as opposed to teaching skills independently, without context.

Shank gives examples of how the SCC curriculum would work for an MBA program and even for high school:

The SCC is about the elimination of courses in favor of curricula that tell a meaningful story that the student is likely to engage in again after graduation. Now, many high school students are simply preparing for college, and thus one could argue that they take trigonometry in order to take college math. The fact that this isn't really true may not matter in this case. What is important is that we identify some stories that the student might want to live in high school because they may come up again. Here are some examples: running a small business, working on a political campaign, building a house, designing a city, running an organization, being a parent, creating an invention, making a discovery, convincing an organization to do things differently. Now, these are not normally thought of as courses in high school. However, looked at closely, they would entail calculating, planning, reasoning, dealing with societal issues, basic psychology, basic economics, dealing with historical issues, communicating in written and oral fashion, teamwork, research and nearly every other subject normally taught in high school (and quite a few that are not.)

Unlike traditional curricula, such a curriculum can have clear goals that give immediate and contextualized feedback on one's learning. To read more on the story-centered curriculum, see Shank's white paper Every curriculum tells a story (pdf).

So now I'm wondering how the SCC curriculum might be adapted for first-year composition. The problem is that the SCC curriculum is for programs not individual courses and Shank notes that not even all programs fit into an SCC curriculum because they don't have well-defined career goals. Generally speaking, first-year composition doesn't have career goals because it's a general education course designed to prepare students to write in more advanced classes and eventually in their widely disparate careers. So, I need to think about this a bit. If you have any ideas about turning composition courses into stories based on career goals, email me.

I just listened to an interesting session at Computers and Writing 2007 on the role of feedback and assessment in first-year composition. Fred Kemp, Ron Baltasor, Christy Desmet, and Mike Palmquist talked about how they used online learning environments as sites for assessing learning and teaching.

Fred Kemp talked about Texas Tech University's ICON system in which

  • class time is cut in half,
  • assignments are doubled or tripled,
  • all relevant interactions are online,
  • students meet in a classroom once a week to support those interactions, and
  • grading and commentary are anonymous with two readers on drafts.

This particular system helps to make the composition program an adaptive, feedback system that gains knowledge over time and is not dependent on rotating faculty and program directors. The data collection that is built into the system has shown that some assignments generate better grades than others, thus indicating where to make changes in the program. For instance, pulling back from having intensive peer reviews (12-13 a semester) has shown a decrease in students' GPA, suggesting that their writing has worsened. Next year, they're reinstating the peer reviews, and if the GPA increases, then there will be a strong correlation for the effect of intensive peer reviews on learning to write.

Mike Palmquist talked about Colorado State University's Writing Studio, a combination instructional writing environment and online course management system. As in ICON, the system collects data on how people are using the site by tracking their activity as they log in, which can give guide the program on which areas need to be strengthened, or vice versa. One question to be answered is, "How does technology shape the teaching and learning in writing courses?"

Ron Baltasor and Christy Desmet talked about the University of Georgia's emma system that embeds meta-data via markup in documents that are uploaded to the server. They have three ongoing projects that look at errors, revision, and citations. One finding from the citation project was that good library instruction works best in conjunction with instructor prompts for citations, but that library instruction alone showed no improvement.

Although the three universities have different approaches, they all show the value of electronic systems that can provide feedback to programs for improving instruction and composition programs.

For the convenience of one location, here are my posts on error feedback, along with my sources and links if available.

My posts:

Sources:

Anderson, J. R., Fincham, J. M., & Douglass, S. (1997). The role of examples and rules in the acquisition of a cognitive skill (pdf). Journal of Experimental Psychology, 23, 932-945.

Anderson, J. R., & Schunn, C. D. (2000). Implications of the ACT-R learning theory: No magic bullets (pdf). In R. Glaser (ed.), Advances in instructional psychology: Educational design and cognitive science (pp. 1-33). Mahwah, NJ: Lawrence Erlbaum (LE).

Bitchener, J., Young, S., & Cameron, D. (2005). The effect of different types of feedback on student ESL writing. Journal of Second Language Writing, 14, 191-205.

Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78, 246-263. (See here, here, and here for synopses of this work and others by Dweck.)

Chandler, J. (2003). The effects of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing. Journal of Second Language Writing, 267-296.

Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York: Harper & Row, Publishers.

Deci, E. L., & Ryan, R. M. (2000). The "what" and "why" of goal pursuits: human needs and the self-determination of behavior (pdf). Psychological Inquiry, 11, 227-268. (See homepage for more on self-determination theory.)

DeKeyser, R. (2007). Skill acquisition theory. In B. VanPatten & J. Wiliams (eds.), Theories in Second Language Acquisition. Mahwah, NJ: LE.

Dörnyei, Z. (2001). Teaching and research motivation. Harlow, England: Pearson Education.

Ericsson, K.A., & Charness, N. (1994). Expert Performance: Its structure and acquisition. American Psychologist, 49, 725-747.

Ferris, D. (2004). The "grammar correction" debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the meantime...?) (pdf). Journal of Second Language Writing, 13, 49-62.

---- (2003). Response to student writing: Implications for second language students. Mahwah, NJ: LE.

---- (2002). Treatment of error in second language student writing. Ann Arbor: University of Michigan Press.

---- (1999). The case for grammar correction in L2 writing classes: A response to Truscott (1996). Journal of Second Language Writing, 8, 1-11.

Ferris, D., & Hedgcock, J. (2005). Teaching ESL composition: Purpose, process, and practice (2nd ed.). Mahway, NJ: LE.

Hinkel, E. (2004). Teaching academic ESL writing: Practical techniques in vocabulary and grammar. Mahwah, NJ: LE.

Hinkel, E., & Fotos, S. (eds.) (2002). New perspectives on grammar teaching in second language classrooms. Mahwah, NJ: LE.

MIles, J. (2002). Second language writing and research: The writing process and error analysis in student texts. TESL-EJ 6.

Ross, P. (2006). The expert mind. Scientific American, 295(2), 64-71.

Truscott, J. (2004). Evidence and conjective on the effects of correction: A response to Chandler (pdf). Journal of Second Language Writing, 13, 337-343.

---- (1999). The case for "The case against grammar correction in L2 writing classes. A response to Ferris (pdf). JJournal of Second Language Writing, 8, 111-122.

---- (1996). The case against error correction in L2 writing courses (pdf). Language Learning 46, 327-369.

Yates, R., & Kenkel, J. (2002). Responding to sentence-level errors in student writing. Journal of Second Language Writing, 11, 29-47.

Below are bibliography pages with downloadable articles related to the above sources (and some repetition, of course):

ACT-R Theory
Self-determination theory

If all I had to go on was the research on Error Correction in L2 Writing, I wouldn't do it. There's simply insufficient evidence to justify such an investment of time and effort.

However, research on learning, expertise, and motivation has garnered an impressive amount of empirical evidence for the positive effects of feedback that meets certain criteria. Before making suggestions on how to structure grammar feedback, let me summarize criteria on learning and motivation for guiding that feedback.

Learning

  1. Learning occurs sequentially through three stages of declarative, procedural, and automatic knowledge.
  2. Acquiring expertise in any field, including a second language, requires extensive practice.
  3. Practice is made effective through
    • accurate diagnosis of the task/rules,
    • examples and understandable explanations for the task/rules, and
    • feedback based on the examples and explanations.
  4. Effective time on task is the most important factor in learning.
  5. Learning occurs best when re-iterated at intervals.

Motivation

  1. Motivation is important because it encourages persistance on task.
  2. Motivation is promoted by
    • clear goals,
    • autonomy,
    • tasks that challenge one's competence without unduly frustrating, and
    • feedback that is immediate and informational.

Grammar Feedback Guidelines

Correctable Grammar

Grammar feedback in L2 writing should target only those items that are rule-governed and for which examples and clear explanations can be found. Subject-verb agreement is one such rule. Style is not.

Structure of Feedback

Dana Ferris (2003) breaks feedback into direct (the teacher giving the answer) and indirect (which ranges from merely noting the location of an error to using editing symbols to more explicit directions, such as use “future tense here.”). She says that direct feedback is preferred for beginning students, while indirect feedback seems to have better effects for intermediate and advanced students, likely because students must think about the errors and engage in self-editing (Ferris, 2003).

To some degree, if students think about an error, they're constructing declarative knowledge. But are they diagnosing the rule accurately and time-effectively? It would be better to first have the rules accompanied by examples and explanations that they continue to refer to. No doubt, Ferris and most instructors refer students to their grammar textbooks, but I'm thinking that students should construct their own textbooks to use grammar feedback more effectively.

Grammar notebooks: Students should maintain grammar notebooks with these examples and explanations, adding to the notebooks as new rules, examples, and explanations are covered. Extra space or pages should be available for students for revision. For instance, if an error was a case of misunderstanding, perhaps the explanation for the rule in their notebook should be revised. Or, if a rule doesn't seem to fit neatly into rules, examples, or understandings previously given, then students can revise the rule, create a new rule, make new examples, or write new understandings. In this way, students can acquire the requisite declarative knowledge, and the notebook becomes a textbook emerging out of, contributing to, and individualized to their own learning.

Goal logs: Students can keep a goal log, in which they set grammar goals and track their improvement over time. Seeing improvement is motivation, and seeing the same error repeatedly can help students target that error, review and revise their grammar notebooks accordingly, and determine strategies for reducing its occurrence.

Program-embedded feedback: Notebooks and goal logs should be used across courses in a program to provide the continuity and repetition needed of reading, writing, and revising understanding across different contexts to proceduralize grammar.

Frequency of Feedback

One problem with learning to write is that unlike sports, chess, and video games, feedback does not occur immediately or even often. Up until now, in my own classes, I generally only give grammar feedback on their major paper assignments, which means they get grammar feedback at the most every 2-3 weeks, and even that occurs several days after the paper is turned in.

If time allows, consider having students write for 5-10 minutes every class and then checking their work or perhaps checking their classmates' work. But instead of having them check for all errors, have them check for one specific error according to class needs. On days with less time, consider using a student example, perhaps from another class. Re-iteration of rules, or anything else, at spaced intervals is crucial for learning. This sort of task would work well for homework, too.

Note that while I grade the grammar component on a major paper assignment, I do not grade it on other assignments. Although the reality check of a grade is a given in most educational institutions, most feedback should be informational rather than evaluative. Otherwise, intrinsic motivation can be dampened.

Grammar Instruction

General lessons on grammar do not fit the criteria above. However, Ferris (2003; cf. Hinkel, 2004) suggests that mini-lessons may be useful if they have the following characteristics:

  1. Mini-lessons should be brief and narrowly focused …
  2. Instruction should focus on major areas of student need, rather than minor fine-tuning.
  3. Lessons should include (minimally) text-analysis activities so that students can examine the target constructions in authentic contexts and application activities so that they can apply newly covered concepts to their own writing.
  4. Instruction should also include strategy training to help students learn to avoid errors and to self-edit their work. (p. 157)

An example of a single task incorporating these guidelines and the criteria above would be one centering on the reporting of an interview (adapted from Hinkel, 2002). A mini-lesson could look at grammatical structures in interviews, such as tenses and reporting verbs. Examples would be given along with understanble explanations. Students would then analyze interviews in newspapers or magazines, focusing on tenses and reporting verbs and comparing to their examples. Next, they would interview someone and write a report of the interview. Finally, students would compare how they used tenses and reporting verbs to the grammatical findings of their earlier analyses and examples in their grammar notebooks.

The key diffferences in the original task and this one is (1) establishing declarative knowledge appropriately and (2) integrating feedback into the task via students' grammar notebooks. Many tasks in textbooks and elsewhere can be reframed to incorporate the learning and motivation criteria above.

Summary

Feedback is crucial for learning any activity, including languages. There are “no magic bullets” to accelerate learning. Rather, appropriate feedback helps students spend “effective time on task,” thus eliminating wasted time and effort.

Disclaimer: Because these suggestions are the recent result of my reviewing these theories and considering their application to error feedback, I haven't implemented them yet. This summer I intend to work on reframing the way I provide feedback and implement my new understanding in the fall semester. After doing so, I hope to provide some feedback here on how it went.

Call for feedback: If you have tried any of these approaches or others based on these theories, email me and let me know how it went, both successfully and unsuccessfully, and I'll post your experiences here.

All Error Feedback Posts in this series:
Error Feedback in L2 Writing
Error Feedback in L2 Writing: Scant Evidence
Error Feedback: Theory
Error Feedback: Skill Acquisition Theory
Error Feedback: Motivation
Error Feedback: Practice
Error Feedback: Bibliography

From the previous posts on learning theory (here and here), two crucial points were:

  1. Acquiring expertise in any field, including language, requires extensive practice.
  2. Effective time on task is the most important factor in learning.

You have to be motivated to practice extensively. Videogamers are and do. They can spend 50-100 hours on a game. Imagine a student spending 50-100 hours writing an essay! In an article on Wired, James Gee comments:

The secret of a videogame as a teaching machine ... [is] its underlying architecture. Each level dances around the outer limits of the player's abilities, seeking at every point to be hard enough to be just doable. In cognitive science, this is referred to as the regime of competence principle, which results in a feeling of simultaneous pleasure and frustration - a sensation as familiar to gamers as sore thumbs. Cognitive scientist Andy diSessa has argued that the best instruction hovers at the boundary of a student's competence. ... Also, good videogames incorporate the principle of expertise. They tend to encourage players to achieve total mastery of one level, only to challenge and undo that mastery in the next, forcing kids to adapt and evolve.

These principles of expertise and competence are also seen in two theories of motivation: self-determination theory and flow.

In self-determiantion theory, motivation is driven by three needs:

  1. autonomy,
  2. social relatedness, and
  3. competence, including informational (not evaluative) feedback on competence, that is, feedback that supports autonomy in learning as opposed to controlling it.

Flow is a state in which one is fully engaged in the task at hand (see this review of Csikszentmihaly's book Finding Flow). Another post noted that flow occurs under certain conditions, three of which are:

  1. clear goals
  2. immediate feedback
  3. tasks that challenge (without unduly frustrating) one's skills

These needs and conditions are an integral part of video games. But have they been in providing error feedback?

Probably not as directly as in other arenas. Just look at the condition of "immediate feedback." In my own first-year composition classes, error feedback is given mostly on essay drafts, which I receive every 2-3 weeks and return in 2-7 days. Can you imagine a basketball coach giving feedback only every few weeks and then a few days after the practice in question?

Although it's not as easy to incorporate these motivation principles into error feedback, I have a few ideas I'll discuss in the next post.

All Error Feedback Posts in this series:
Error Feedback in L2 Writing
Error Feedback in L2 Writing: Scant Evidence
Error Feedback: Theory
Error Feedback: Skill Acquisition Theory
Error Feedback: Motivation
Error Feedback: Practice
Error Feedback: Bibliography

As noted in the previous post on theory in error feedback, the research on expertise and ACT-R Theory suggest that learning a language is similar to learning other activities like chess, music, and math. In this post, we'll look at Skill Acquisition Theory (SAT), which is based upon the work in ACT-R.

Robert DeKeyser (2007) has a good overview of Skill Acquisition Theory. (See Theories in Second Language Acquistion.) As he notes, development has three stages: declarative, procedural, and automatic (from ACT-R Theory). Declarative knowledge refers to explicit knowledge about a topic, as in "knowing" and talking about grammar rules. Procedural knowledge is implicit knowledge that refers to behavior, such as speaking or writing a language. Of course, there are different levels of proficiency in using a language, and thus automaticity is not an "all-or-nothing affair". Automaticity occurs toward the endpoint of extensive practice, toward the point at which one has become completely fluent in a language. From the perspective of SAT, the sequence of these stages is crucial, as is the appropriate "combination of abstract rules and concrete examples" at the declarative stage.

According to DeKeyser, Skill Acquisition Theory does not explain all of language learning and apparently is most effective at beginner levels. He states that SAT works best with

  1. high-aptitutde adult learners engaged in
  2. the learning of simple structures at
  3. fairly early stages of learning
  4. in instructional contexts

It seems obvious that young children will not respond as well as adults to the use of declarative knowledge as their ability to understand rules and explanations is more limited. Conversely, as rules become more complex, they may become too difficult to understand in the form of declarative knowledge. Thus, it's possible that learning (or acquiriing) complex rules may rely more upon implicit processes. Anderson and Schunn (pdf) say something similar:

As knowledge domains become more advanced, their underlying cognitive structure tends to become more obscure. Thus, while it may remain easy to provide feedback on what the final answer is, it becomes difficult to provide feedback on the individual mental steps that lead to the final answer. Teachers often are unaware, at an explicit level, of what this knowledge is and do not know how to teach it to children.

Anderson and Schunn are pointing to the need to diagnose a task and break it down into its components in order to provide effective feedback. When we can't componentialize a task, then feedback becomes considerably less effective. Basically, we can only say then, "No, that's not right" or "Yes, that's it."

Thus, with respect to error correction, we need

  • rules that are not obscure,
  • examples of the rules, and
  • understandable explanations of those rules.

The ability to use declarative knowledge in the learning process does not accelerate acquisition. Rather, it eliminates wasted time and effort.

Before turning to practical suggestions for error correction, I'll look at motivation in my next post.

All Error Feedback Posts in this series:
Error Feedback in L2 Writing
Error Feedback in L2 Writing: Scant Evidence
Error Feedback: Theory
Error Feedback: Skill Acquisition Theory
Error Feedback: Motivation
Error Feedback: Practice
Error Feedback: Bibliography

When the evidence for error feedback is "scant", mostly what we have to go on is theory. Those who oppose error correction would likely assume a nativist framework that includes:

  1. Language acquisition (whether first or second language) differs from general learning processes.
  2. Language acquisition and general learning processes do not interact.
  3. The process of language acquisition cannot be accelerated.

Without reviewing the research, let me say that there are those, even in L1 studies, that assert that language acquistion results from general learning processes (e.g., Christiansen and Chater's working paper, "Language as Shaped by the Brain"). And alternative theories also exist in second language acqusition, such as Skill Acquistion Theory and Associate-Cognitive CREED (Construction-based, Rational, Exemplar-driven, Emergent, and Dialectic). (See Theories in Second Language Acquistion.) So, why would anyone want to take a nativist position? There are more than a few reasons. The main one seems to be the poverty of the stimulus, but in part, one reason seems to be that language acquisition is seen as inexplicably complex while other learning endeavors seem simple in comparison. I would like to suggest that the comparisons being made are simple, but not the objects, or processes, being compared.

I've mentioned Philip Ross's article "The Expert Mind" (in Scientific American, see also my post) on more than one occasion, but it's worthwhile to return to it often. It states,

The preponderance of evidence is that experts are made, not born.

Although not innate, expertise takes time to develop. In general, it takes "takes approximately a decade of heavy labor to master any field." Thus, although it doesn't take much to learn how to move the pieces in chess, becoming a grandmaster of strategy, however, takes at least a decade of intense practice. Similarly, although it doesn't take much to learn a few basic grammar rules and a small vocabulary, becoming fluent in another language--that is, acquiring the "grandmaster status" of a native speaker--takes at least 10 years of intense practice, too.

If the amount of time to acquire expertise is similar between chess, music, art, math, and language, that suggests for learning a language,

  1. the crucial element is practice rather than some language module
  2. the process cannot be accelerated.

Because the process cannot be accelerated, it may matter little whether one takes an nativist or general learning process approach to language acquistion. Note, however, that all practice is not equal. From the article,

Ericsson argues that what matters is not experience per se but "effortful study," which entails continually tackling challenges that lie just beyond one's competence. That is why it is possible for enthusiasts to spend tens of thousands of hours playing chess or golf or a musical instrument without ever advancing beyond the amateur level and why a properly trained student can overtake them in a relatively short time. It is interesting to note that time spent playing chess, even in tournaments, appears to contribute less than such study to a player's progress; the main training value of such games is to point up weaknesses for future study.

The case of enthusiasts practicing without advancing is reminiscent of fossilization, and the "effortful study" reminds me of Krashen's i+1 principle for input. One difference is "point[ing] up weakenesses for future study," a "monitor" approach that Krashen would say does not contribute to language acquisition.

Anderson and Schunn in their article "The implications of the ACT-R learning theory: no magic bullets" (pdf) make similar assertions:

For competences to be displayed over a lifetime, time on task is by far and away the most significant factor.

However, they qualify that to mean "effective time on task." Once again, not all practice is equal. In the case of ACT-R Theory, "effective time on task" is promoted through

  • the use of examples accompanied by explanation and understanding,
  • accurate diagnosis of the learning task and performance, and
  • feedback

It seems that research on expertise and ACT-R Theory would support some form of error correction. Because Skill Acquistion Theory, which draws upon ACT-R and similar theories, focuses on SLA, in my next post, I'll look at it in a little more detail.

All Error Feedback Posts in this series:
Error Feedback in L2 Writing
Error Feedback in L2 Writing: Scant Evidence
Error Feedback: Theory
Error Feedback: Skill Acquisition Theory
Error Feedback: Motivation
Error Feedback: Practice
Error Feedback: Bibliography

My earlier post Error Feedback in L2 Writing looked at Truscott's position (pdf) not to correct grammar at all.

Is Truscott right? Should we abandon grammar correction in our writing classes? One reason he gives is that the research does not support correcting grammar. Even Dana Ferris (1999), responding to Truscott, admitted that the evidence supporting error response was "scant." But she added that that was not the same as saying that it doesn't help students. She mentioned a few studies not mentioned by Truscott that do support error correction, stating,

This rush, or stampede, to judgment is especially egregious in Truscott’s review essay. Based on limited, dated, incomplete, and inconclusive evidence, he argues for eliminating a pedagogical practice that is not only highly valued by students, but on which many thoughtful teachers spend a great deal of time and mental energy because they feel that helping students to improve the accuracy of their writing is vitally important. Had Truscott used his review to ask some pointed questions about error correction and to identify some of the problems raised by the available research, he would have done teachers and scholars a valuable service. But because he went further and offered sweeping conclusions, he has potentially put students at risk--that their teachers, teacher educators, or researchers will accept his claims uncritically and adjust their practices accordingly, to the possible detriment of students’ development as writers.

I find this position less than satisfactory. If the evidence is "scant," according to someone who supports error correction, then which way to turn is little more than a coin toss. If it has little or no effect, why would anyone want to do, as Ferris puts it, "time-consuming and mostly tedious" work? Ferris adds,

I also find that the time and energy I spend sometimes does not pay off in long-term student improvement.

Although Ferris qualifies that statement with "sometimes," such a statement coming from her, a major proponent of error feedback, is discouraging. Even so, I figured if there were any research supporting error correction it would be found in Ferris's (2003) Response to student writing, which contains an excellent review of the literature on error feedback. However, either my library has misplaced it or someone decided the library didn't need it as much as they did. So, I looked at Laurel Reinking's (Linguist List 16.111) friendly review of the book. But even she had to conclude,

Although Ferris provides an insightful analysis of each study presented, because, as she complains, "the results of the ... studies ... have been conflicting and not always well designed or clearly described ..." (p. 67), her conclusion does not support the efficacy of error correction.

Not much support here. So, I turned to one of Ferris's more recent articles "The 'grammar correction' debate in L2 writing." In this article, she summarized her position:

  1. the research base on the ‘‘big question’’—does error feedback help L2 student writers?—is inadequate;
  2. the previous studies on error correction are fundamentally incomparable because of inconsistencies in design; and
  3. existing research predicts (but certainly does not conclusively prove) positive effects for written error correction.

In other words, the best we can say is that some research hints that error feedback may be helpful. Ferris concludes that what she has done is to

critique most or all of the previous research and essentially argue that we need to start from scratch. Obviously, it could be years, even decades, before we have trustworthy empirical answers to some of the questions we need to consider—so what do we (teachers and teacher educators) do in the meantime?

As Ferris herself notes, we teachers don't have "decades" to wait. We have to teach now. One possibility is to consider other theories outside of the SLA and SLW boxes in order to re-frame the error correction issue. So, more on theory in the next post.

References:

Ferris, D. (1999). The case for grammar correction in L2 writing classes: A response to Truscott (1996). Journal of Second Language Writing 8, 1-11.

Ferris, D. (2003). Response to student writing: Implications for second language students. Mahwah, NJ: Lawrence Erlbaum.

Ferris, D. (2004). The "grammar correction" debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the meantime...?) Journal of Second Language Writing 13, 49-62.

All Error Feedback Posts in this series:
Error Feedback in L2 Writing
Error Feedback in L2 Writing: Scant Evidence
Error Feedback: Theory
Error Feedback: Skill Acquisition Theory
Error Feedback: Motivation
Error Feedback: Practice
Error Feedback: Bibliography

Do you ever wonder how your English language learners (ELLs) to improve their grammar? For myself, when I look at their papers in first-year composition, I'm struck by the number of errors in grammar, not simply problems of prepositions and the articles a and the, but problems of subject-verb agreement, incorrect use of verb tenses, run-on sentences, sentence fragments, and many more.

Language learning is a long, arduous process, and it's not realistic to expect that ELLs will become nativelike in less than 10 years, much less the one semester I have them in my course. Just consider that Ulla Connor, a leading scholar in contrastive rhetoric and professor of English at Indiana University, stated in her book Contrastive Rhetoric (1996) that she still “tends to use [articles and prepositions] inappropriately” 20 years after receiving her doctorate and teaching in the U.S. (p. 4).

So, how can we help our students improve their grammar? Or can we? Some believe that we can't. Innatists, such as Krashen, hold that language acquisition differs from language learning, that the two have no interface, and so grammar instruction does not aid language "acquisition," only "learning." Still, even Krashen (2004) admits that some grammar knowledge can be useful for advanced learners in editing. This makes sense as writing, unlike speaking, allows time for monitoring. How to help advanced learners acquire this knowledge remains problematic, however.

Another anti-grammar-correction proponent is John Truscott. His (in)famous article "The case against grammar correction in L2 writing classes" (pdf) reviewed the literature on error feedback and asserted:

Do not correct grammar.

Truscott's main reasons for abandoning grammar correction included:

  1. Research has not shown grammar correction to be effective.
  2. Language acquisition is a gradual process that cannot be accelerated through the "transfer" of grammar knowledge.
  3. The time students spend on understanding grammar correction and applying it could be spent more productively on other activities, such as improving organization and logic.
  4. Teachers may do a poor job of recognizing and correcting errors.

Truscott's position is controversial, of course. So, we'll look at the reasons and positions on error correction in more detail over a series of posts, with the next post on the paucity of evidence.

All Error Feedback Posts in this series:
Error Feedback in L2 Writing
Error Feedback in L2 Writing: Scant Evidence
Error Feedback: Theory
Error Feedback: Skill Acquisition Theory
Error Feedback: Motivation
Error Feedback: Practice
Error Feedback: Bibliography

It is not that students cannot learn, it is that they do not wish to.
—Mihaly Csikszentmihalyi

Have you ever noticed that second thoughts are often better than first ones? In my previous post, my first thoughts were to tie engagment to autonomy and time on task. Two days later, on my desk staring at me was psychologist Mihaly Csikszentmihalyi's book "Flow: The Psychology of Optimal Experience". "Flow" fits the notion of engagement better. From the book, flow is

the state in whch people are so involved in an activity that nothing else seems to matter; the experience is so enjoyable that people will do it even at great cost, for the sheer sake of doing it.

Most of us have had that "involved" moment happen, when we concentrated our attention so intensely on solving a problem, reading a book, climbing a mountain, on some task, that we lost track of time and when we became aware of our surroundings, a few hours or more had passed by as if they were minutes. Such "flow", according to Csikszentmihalyi, is "optimal experience" that leads to happiness and creativity.

Flow occurs when certain conditions are met, four of which are

  1. clear goals
  2. immediate feedback
  3. focused attention
  4. tasks that challenge (without frustrating) one's skills

Csikszentmihalyi's diagram of item #4 (this representation was created by Wesley Fryer) shows why flow should be taken into consideration when designing class tasks.

If a task is not challenging enough, boredom sets in, while too great a challenge results in anxiety, and both cases result in task, and thus learning, avoidance. As one's skills increase, then the challenge must also increase for one to remain in a state of flow. Because flow is an enjoyable experience, one continues to increase the challenge level (as from A1 to A4 and so on), and consequently continues to improve one's skills because doing so is necessary to stay in a flow state. Thus, we see the importance of "engaging" students in school. From the book,

flow activities lead to growth and discovery. One cannot enjoy doing the same thing at the same level for long. We grow either bored or frustrated; and then the desire to enjoy ourselves again pushes us to stretch our skills, or to discover new opportunities for using them.

Of course, easier said than done, especially when one's students (mine, for example) often hold full-time jobs while being full-time plus students. Too much work and too little time constantly puts my students in states of frustration. Even when not, flow states are not a regular occurrence in life; according to Csikszentmihalyi (quoted by Jamie Chamberlin in the APA Monitor),

'A typical day is full of anxiety and boredom,' says Csikszentmihalyi. 'Flow experiences provide the flashes of intense living against this dull background.'

So, it's unlikely that in formal learning contexts that states of flow will be become the norm every day all day. After all, not all tasks are enjoyable, but they might be necessary, just as grading is a necessary but tedious part of teaching. Still, it seems more and more that students are being turned off by classroom learning. As Csikszentmihalyi asserts,

It is not that students cannot learn, it is that they do not wish to.

They don't, because they don't see the relevance of school learning. The relevance of math, for example, remains hidden until it is needed in a real world context such as engineering. As Csikszentmihalyi notes, in former times, young children learned in real settings, whether it was hunting, farming, or some trade. They could see first hand the relevance of their activity. Similarly, students playing sports and music see the relevance of any associated instruction. More importantly, goals are clear and feedback is immediate, as in chess when a piece has been taken.

Now, I still agree with Artichoke that student satisfaction/enjoyment is not a reliable measure of learning and that much talk about "engagement" is more jumping on a not-too-well-thought-out, feel-good bandwagon than anything else, but I want to look at this one point:

And engagement, despite Prensky’s slickly marketable “engage me or enrage me” stuff, engagement is not a self report measure of wonderment and awe but rather a reflection of the determined and persistent focus that a learner needs to promote learning.

As Brabazon notes in her provocative book Digital Hemlock “To read remember, understand, synthesise and interpret knowledge is often drudgery. To learn with effectiveness requires repetition, practice and failure.”’ (p9)

Why should repetition, practice and failure be equated with drudgery? James Paul Gee (pdf) notes how in good video games,

mastery is consolidated through repetition (with variation), only to be challenged again.

Yet gamers play hours on end, repeating the same moves over and over. And this is true, too, of playing sports, music, and chess. Yet, one seldom hears of the repetition in these arenas as drudgery, perhaps hard, perhaps demanding, but not drudgery. In fact, flow can be achieved in something as apparently boring as working on an assembly line. In his book, Csikszentmihalyi recounts the example of Rico:

The task he has to perform on each unit that passes in front of his station should take forty-three seconds to perform—the same exact operation almost six hundred times in a working day. Most people would grow tired of such work very soon. But Rico has been at this job for over five years, and he still enjoys it. The reason is that he approaches his task in the same way an Olympic athlete approaches his event: How can I beat my record? Like the runner who trains for years to shave a few seconds off his best performance on the track, Rico has trained himself to better his time on the assembly line. With the painstaking care of a surgeon, he has worked out a private routine for how to use his tools, how to do his moves. After five years, his best average for a day has been twenty-eight seconds per unit. ... when he is working at top performance the experience is so enthralling that it is almost painful for him to slow down. "It's better than anything else," Rico says. "It's a whole lot better than watching TV." Rico know that very soon he will reach the limit beyond which he will no longer be able to improve his performance at his job. So twice a week he takes evening courses in electronics. When he has his diploma he will seek a more complex job, one that presumably he will confront with the same enthusiasm he has shown so far. (pp. 39-40)

Apparently, it is not repetition or practice per se that is drudgery. Perhaps, in school learning, repetition is not accompanied by variation, and so becomes drudgery. Perhaps, school repetition too often lacks the clear goals and immediate feedback of video games, sports, music, and chess, and so becomes drudgery. We need to find ways of integrating repetition and practice into school learning without them becoming drudgerous.

In the comments section, Artichoke stated,

the problem does seem to lie in the many differing meanings we attribute to "engagement".

If we can design learning conditions conducive to flow, perhaps our students will wish to learn.

Definitely. But the everyday meaning of engagement seems congruent with the academic concept of flow. In fact, one of Csikszentmihalyi's books is titled "Finding Flow: The Psychology of Engagement with Everyday Life". Perhaps informing our understanding of engagement with the research on flow can help us move forward in "engaging" our students. Flow, again, requires clear goals, immediate feedback, challenging tasks, and variation in those tasks. If we can design learning conditions conducive to flow, perhaps our students will wish to learn.

Related posts and articles:

I've partially fixed my webdesign problem with Internet Explorer. The sidebar can be seen. However, on IE 6, a supposedly transparent spacer covers the title, and on IE 5.2, the sidebar has dropped down. I have no more time right now. Maybe in December?

Working on my own design instead of just using the templates provided wasn't easy, especially as my HTML knowledge is limited, while my CSS knowledge is even more limited. So, why did I go through frustration instead of just using the templates? Where'd the motivation come from? Both the notion of flow and self-determination theory can shed some light here.

From the perspective of flow, I had clear goals, immediate feedback, focused attention, a sense of control (at least when I was successful), and a merging of action and awareness (see Flow, Games, and Learning and Flow Theory): As I tinkered with the design, I could quickly see what worked and what didn't. And the time flew by as I focused on the task at hand.

In self-determination theory, three needs for intrinsic motivation are autonomy, competence, and social relatedness. I'm not quite sure how social relatedness applies here, but it's obvious that I had some competence (or I couldn't have finished what I had begun) and that I was fully autonomous in choosing what I wanted to do, how I went about it, and what I accepted as the final (for now) product. Despite the frustration, solving the puzzle of creating my own design was fun and satisfying.

Now I wonder, How often do our students enjoy the puzzle of learning? Looking back at my own undergraduate days, I'm not sure I enjoyed learning as much then as I do now. First off, the amount of cramming required for a high GPA required for going to graduate school simply took the fun out of learning. Note the word "required," a staple of educational institutions, which precludes much of autonomy. Of course, there were other factors, such as lack of time: I had to work my way through school. Lack of time affects the ability to develop competence--as noted above by my problem with Internet Explorer.

So, how do we go about creating environments that promote flow and self-determination in our classes? More on that later.

A little while ago, my seven-year-old son asserted on doing his homework,

I'm so smart. I have everything in my brain.

However, about ten minutes later when I asked him to tie his own shoelaces, he said,

I can't. I know the first part, but I don't know the second part. Is it the thumb or two fingers?

His comments reminded me of the book The Embodied Mind by Varela, Thompson, and Rosch, which posits a fundamental circularity between cognition and experience. There is no disembodied mind directing our actions: All knowledge is enacted via experience.

Eleanor Rosch gave a talk at the American Psychological Association a few years ago titled "What Buddhist Meditation has to Tell Psychology About the Mind". She ended her talk with some quotes from the autobiography of Peter Ralston, a world martial arts champion:

One: The starting point: ambition, focused intention “As a teenager I wanted to be the best fighter in the world. Period!”

Two: Recognition of the unsatisfactoriness of the ordinary conscious way of doing things. (May come with success) “Around that time, I would go to classes and fight black belts and win, but still feel like I lost…Something wasn’t right…. I was winning from natural ability, but I wasn’t winning because I really understood anything…”

Three: Finding the unbiased mind beyond fear and desire. Opening perceptions. Appreciation. “It was in that situation that I first learned to drop fear of getting hit, or of winning or losing… What that did was open up my perception to what was really happening. I just saw a fist coming and I’d move…When I’d get worried about it, I’d get stuck somewhere and get hit… It’s a beautiful secret, an exacting and tremendous feedback.”

Four: Expansion of the knowing field. Also some change in sense of time. “…abilities like being able to read somebody’s disposition accurately started to come. The moment they would think to hit me I would stop them. That’s it. Handled. I just kept finishing everything before it got started.”

Five: Actions from awareness; simply knowing what to do and it’s always appropriate “New abilities started to arise… I didn’t have to be cognizant of any movement on their part, psychic or otherwise, to know what to do. I just knew. That blew me away. I didn’t have to perceive a thing…very simple, very simple.”

Six: Comes full circle; transformation of the original ambition and intention “I decided that if I were to continue to do this, I wanted to start contributing what I did and what I knew in a much larger way. I wanted to transform the martial arts in the world into a place for the development of the human being, and of honesty.”

Quite a bit of what Ralston says is similar to Mihaly Csikszentmihalyi's work on flow, a process of total enagement in an activity for its own sake with the result that one feels a sense of satisfaction and loses track of time. Flow has eight dimensions, not all of which must be operating at once (from EduTech Wiki):

Clear goals and immediate feedback
Equilibrium between the level of challenge and personal skill
Merging of action and awareness
Focussed concentration
Sense of potential control
Loss of self-consciousness
Time distortion
Autotelic or self-rewarding experience

It seems obvious that Ralston often enjoyed the state of flow. Many athletes do, as do video gamers, gardeners, and others. According to Csikszentmihalyi (quoted by Jamie Chamberlin in the APA Monitor), however, flow is not typical:

'A typical day is full of anxiety and boredom,' says Csikszentmihalyi. 'Flow experiences provide the flashes of intense living against this dull background.'

Elsewhere, Csikszentmihalyi wrote,

It is not that students cannot learn, it is that they do not wish to.

And they don't, because school is seldom a place of "intense living." Of course, work isn't, either, but that's not the point. If we wish for students to enjoy learning, then it would help to design our classes so that they are more conducive for states of flow to occur.

Sometimes, the system just works against states of flow. For instance, my ESL students are expected to reach levels of English that, although possible, are often more than challenging due to obligations constraining their study time, such as working 20, 30, and 40 hours a week. In addition to working full time, most of my night students (and some of my day students) are married (or single) with children.

Still, another condition for flow is clear goals and immediate feedback. As I look at my composition syllabus, those goals are probably not clear enough to my students, and feedback is usually delayed. It shouldn't be too difficult to make the goals clearer, but it's more difficult to give immediate feedback on essays. I usually grade them on the weekend, and so there's a 5- to 7-day delay.

What would be interesting would to develop a software tutor for writing that could provide immediate feedback and guidance. John Anderson et al. has an interesting article "Cognitive Tutors: Lessons Learned". The article discusses different tutors (algebra, geometry, LISP) used to facilitate student learning and mentions a few problems:

Students' own attitudes to the tutor classrooms are quite positive to the point of creating minor discipline problems. Students skip other classes to do extra work on the tutor, refuse to leave the class when the period is over, and come in early.

Isn't it terrible when motivation becomes a problem? A tutor application for writing would likely be harder to create than it is for math. Math has right and wrong answers, and the wrong answers can fall into different types of errors for which a tutor can be programmed to respond. Writing is fuzzier than math. It's not right or wrong: it's more or less effective. But if it could be done, it would have the advantage of many of the conditions for flow.

Another possibility would be to create video games in which writing plays a major role. James Gee, a professor at the University of Wisconsin, in his book "What video games have to teach us about learning and literacy" strongly supports using games in education. Christine Simmons ("Video games seen as way to train, learn") reports that the Federation of American Scientists (FAS) "has developed three 3-D video games to be used for training and education," two of which are for firefighting and immunology. On the latter one:

"Immune Attack," places players on a tiny vessel that can travel inside the human body. The game aims to educate high school, college and graduate-level students in immunology. The goal is to find and attack dangerous bacteria, said Kay Howell, vice president for information technologies at the FAS.

Shaffer et al. have a paper on "Video Games and the Future of Learning". As they note:

The American Army, a longtime leader in simulations, is building games like Full Spectrum Warrior and America’s Army —games that introduce civilians to military ideology. Several homeland security games are under development, as are a range of games for health education, from games to help kids with cancer better treat themselves, to simulations to help doctors perform surgery more effectively. Companies are developing games for learning history (Making History), engineering (Time Engineers), and the mathematics of design (Homes of Our Own).

I doubt that most of my students would be interested in a game designed simply to write better. But what if writing were a crucial element in the game? Perhaps games for journalists, business managers, lawyers, and others for whom writing is an integral part of the job? Or perhaps redesign existing games to put the focus on writing? I have more questions than answers. But Shaffer et al. comment on the implicit learning theory behind video games:

Video games thus make it possible to “learn by doing” on a grand scale—but not just by doing any old thing, wandering around in a rich computer environment to learn without any guidance. These forms of learning, associated with progressive pedagogies, are bad theories of learning. Learners are novices. Leaving them to float in rich experiences with no guidance only triggers the very real human penchant for finding creative but spurious patterns and generalizations. The fruitful patterns or generalizations in any domain are the ones that are best recognized by those who already know how to look at the domain and know how complex variables in the domain interrelate with each other. And this is precisely what the learner does not yet know. In Full Spectrum Warrior, in contrast, the player is immersed in activity, values, and ways of seeing. But the player is guided and supported by the knowledge built into the virtual soldiers and the weapons, equipment, and environments in the game. Players are not left free to invent everything for themselves. To succeed in the game, they must live by—and ultimately master—the epistemic frame of military doctrine.

So, we need a game in which students "live by—and ultimately master—the epistemic frame of" a rhetorician. Hmm. I think I would enjoy, playing that game.

On a final note, educators, myself included, often try to ease students' way into materials as much as possible, thus sometimes (often?) "dumbing down" their learning. In an article on Wired, James Gee comments:

The secret of a videogame as a teaching machine isn't its immersive 3-D graphics, but its underlying architecture. Each level dances around the outer limits of the player's abilities, seeking at every point to be hard enough to be just doable. In cognitive science, this is referred to as the regime of competence principle, which results in a feeling of simultaneous pleasure and frustration - a sensation as familiar to gamers as sore thumbs. Cognitive scientist Andy diSessa has argued that the best instruction hovers at the boundary of a student's competence. Most schools, however, seek to avoid invoking feelings of both pleasure and frustration, blind to the fact that these emotions can be extremely useful when it comes to teaching kids.

Also, good videogames incorporate the principle of expertise. They tend to encourage players to achieve total mastery of one level, only to challenge and undo that mastery in the next, forcing kids to adapt and evolve. This carefully choreographed dialectic has been identified by learning theorists as the best way to achieve expertise in any field. This doesn't happen much in our routine-driven schools, where "good" students are often just good at "doing school."

How did videogames become such successful models of effective learning? Game coders aren't trained as cognitive scientists. It's a simple case of free-market economics: If a title doesn't teach players how to play it well, it won't sell well. Game companies don't rake in $6.9 billion a year by dumbing down the material - aficionados condemn short and easy games like Half Life: Blue Shift and Devil May Cry 2. Designers respond by making harder and more complex games that require mastery of sophisticated worlds and as many as 50 to 100 hours to complete.

"Hover[ing] at the boundary of a student's competence" and challenging students "to adapt and evolve" with immediate feedback put players in a state of flow. Hmm. Would it be possible to design an entire course as a video game?

As you may have noticed, I've been working on the design of this blog: mostly color changes but also a fluid design for the content side. At first, I started trying to wrap the posts around at the bottom of the sidebar. I did my research, read the tutorials, but couldn't figure it out. I emailed Mark Bernstein, the designer of Tinderbox, this weblog's software application, suggesting that the feature be incorporated into later versions. He responded (Mark's generosity with his time is unbelievable!), offered to do it for me, and, one hour later, sent me my weblog file re-coded with the fluid design (a switch from the earlier wrapping style). And I continued with changing the colors, which is not a straightforward process for someone who is colorblind. (I use the Color Generator and patient friends.)

What's this got to do with learning with examples? Well, I've learned by observing what Mark did. Previously, I would duplicate an entire file to have a practice file; Mark simply added a new CSS note. Previously, I would export an entire document to see how it looked in html. While in Boston, I noticed that Mark just used the Preview button. And from the code he sent, I began to understand the difference between "float" and "absolute". In trying to re-design this blog,I spent two full days acquiring quite a bit of frustration but little understanding, as opposed to taking a few minutes to look at Mark's re-coding to learn where I had gone wrong.

Such incidental learning via examples underscores John Anderson's ACT-R learning theory. Anderson, a professor of psychology and computer science at Carnegie Mellon University, is the one who first posited two types of knowledge: declarative and procedural. I've posted on Anderson before (see "Lies teachers tell?" and "Forget IQ. Just Work Hard!".

Anderson and Schunn in their article "The implications of the ACT-R learning theory: no magic bullets" (pdf) write:

There are no magical properties conveyed upon a knowledge structure just because it was self-generated. If all things were equal it would be preferable to have children learn by generating the knowledge (due to the redundant encoding). However, because of difficulties of generation and dangers of misgeneration, things are not always equal and it can be preferable to tell the knowledge.

...

Thus, ACT-R's theory of procedural learning claims that procedural skills are acquired by making references to past problem solutions while actively trying to solve new problems. Thus, it is both a theory of learning by doing and a theory of learning by example.

Simply providing the learner with examples is not sufficient to guarantee learning in the ACT-R theory. The sufficiency of the production rules acquired depends on the understanding of the example.

Anderson and Schunn add, "For competences to be displayed over a lifetime, time on task is by far and away the most significant factor." That is, learners must practice a lot. The problem is one can practice the wrong skills, in which case "practice makes imperfect." In other words, learners need feedback and explicit guidance, often in the form of examples, to make their practice effective.

But how can examples be so effective? Perhaps because human beings learn mostly through imitating. Vilayanur Ramachandran, Director of the Center for Brain and Cognition, posits that imitation via mirror neurons is the driving force of human evolution:

With knowledge of these neurons, you have the basis for understanding a host of very enigmatic aspects of the human mind: "mind reading" empathy, imitation learning, and even the evolution of language. Anytime you watch someone else doing something (or even starting to do something), the corresponding mirror neuron might fire in your brain, thereby allowing you to "read" and understand another's intentions, and thus to develop a sophisticated "theory of other minds."

An earlier post "Be Happy, and Learn!", commented on Kathy Sierra's post "Angry/negative people can be bad for your brain" on the effect of mirror neurons on one's emotional state:

There is now strong evidence to suggest that humans have the same type of "mirror neurons" found in monkeys. It's what these neurons do that's amazing--they activate in the same way when you're watching someone else do something as they do when you're doing it yourself! This mirroring process/capability is thought to be behind our ability to empathize, but you can imagine the role these neurons have played in keeping us alive as a species. We learn from watching others. We learn from imitating (mirroring) others. The potential problem, though, is that these neurons go happily about their business of imitating others without our conscious intention.

Note, however, that this imitation is an unconscious process. I'm not quite sure of the relationship between consciously understanding and using examples and imitating those examples. Perhaps understanding comes through imitation + practice.

Although we wouldn't want to limit ourselves to learning by imitation, the fact that imitation is such a strong component of learning should give us pause when we read statements that denigrate imitation and position it in opposition to creativity.

Paul Butler argues for re-introducing imitation into composition in his article "Imitation as Freedom: (Re)Forming Student Writing":

For many years now, the use of imitation in the composition classroom has been waning. As Connors points out, articles on imitation, sentence combining, and generative rhetoric have steadily declined and have been almost nonexistent since 1995. Yet in composition classrooms all over the country, as we adopt various process techniques, we still hold our students accountable for the fundamental elements of good writing: organization, coherence, unity, and clarity, among others. Lisa Delpit has pointed out that our expectations are sometimes “hidden,” that they remain invisible to students as we encourage them to explore their ideas and work within the process model of teaching. Delpit’s argument, though intended to address the situation of minority students, also applies to students in composition classes around the country. Indeed, it seems the height of hypocrisy to use strictly process techniques when we expect high quality “products” from our students’ writing.

Along these lines of using examples and imitation, I commented previously on They say/I say : the moves that matter in academic writing, a book by Gerald Graff and Cathy Birkenstein that uses templates to help students see and be able to make the rhetorical moves of academia.

I think that most of us forget how often we use and appreciate examples when we enter new territory. For instance, when writing my first book review, I looked at dozens of other book reviews to understand this genre's requirements. I imagine if someday I write a grant proposal, I'll do the same, too. And, I imagine that most writers follow suit. If we learn this way, then why wouldn't our students do so, too? Why do we expect them to start from scratch when we don't? And with respect to EFL/ESL students who don't have a strong L2 cultural foundation for learning L2 writing by "osmosis," the case for making explicit the implicit is even more essential.

None of this is an argument for rote memorization of models. Rather, it's an acknowledgement that if we are wired for imitation, for learning with examples, then why not take advantage of our "wiring" when designing class activities?

John Liang, Timothy Grove, Sydney Rice, and I presented papers at TESOL 2006 on the theme of "Moving Toward Self-Assessment in L2 Writing."

John Liang began with an "Overview of Self-Assessment in the Second Language Classroom." His overview handout here (.doc) also has a good bibliography on self-assessment.

Next, I talked about using "Course-Embedded Assessment" (.doc) to help students learn to assess their writing. Generally speaking, course-embedded assessment refers to program- or institution-wide assessment embedded in general education courses in order to focus the curriculum on student learning. In my classes, I've incorporated the program rubric for assessing L2 writing in all aspects of my first-year composition courses--from modeling, using it to guide my feedback, having students use it to guide their feedback to others, and to guide their own self-evaluation--so that it becomes part of their mind-framework for looking at writing rather than remaining fragmented information and forgotten as soon as the semester ends.

Sydney's paper looked at "Focused Self-Assessment" (.doc) presenting three basic steps for students to become self-editors:

1. Provide input and examples of both effective and ineffective language use.
2. Involve students in peer review and peer editing, as well as self-editing.
3. Provide students with the key for productive self-editing.

Her approach uses "methodical and uncomplicated" rubrics, an approach that makes it clear and gives to students the tools for editing and revising their writing. Here are her other handouts (all are .doc): Summary, Overhead figures.

Timothy Grove discussed "Showcase Portfolios" (.doc) for helping students become better self-assessors. When students have to select and present their best work, they begin to learn how to evaluate their work.

John Liang ended the colloquium talking on "Toward a Three-Step Pedagogy for Fostering Self-Assessment in a Second Language Writing Classroom" (.doc). The three steps are:

Stage 1: Extensive teacher modeling
Stage 2: Teacher assessment with guided and independent peer assessment
Stage 3: Peer assessment leading to guided and independent self-assessment

One point John mentioned that occurred in all of our talks was the need for rubrics or something that would give structure to the students as they began to learn to assess their learning.

I've been reading up on course-embedded assessment, which I mentioned in an earlier post, and am wondering about the implications for my own classes. As a member of the ESL Program, I use the program's rubric for assessing my students and also to help focus them on areas in which they need improvement. Apparently, however, the rubric's criteria are somewhat elusive for my students. Actually, there are two elements out of the ten that I have to think about, too. If I want my students to better understand criteria of good writing, I've been considering ways to incorporate the rubric in other ways than simply their final grade.

One way I have recently incorporated the rubric is to use it as the basis for my feedback on rough drafts. Another is to have students use it to guide their reviews of classmates' papers. A third way I'm considering is having students write a paragraph to hand in along with a following essay on what aspects of the rubric did they work on to improve the present essay with respect to the previous one.

For students to grasp these criteria that I've been working with for ten plus years, consciously and unconsciously, using them 3-4 times a semester, once per paper is not enough. They need to spend time with them, to reflect on them, and to use them throughout the course on a variety of assignments.

For assessment to be formative, it should be embedded pervasively throughout the course so that the students continually receive feedback and so that they internalize its criteria. Such course-embedded assessment seems common sense to me.

Course-embedded assessment: What's it all about? one might wonder, thinking that all assessment is somehow embedded in course content. But that is only one aspect of it. Course-embedded assessment also refers to program- or institution-wide assessment that is embedded in all courses in order to focus attention on student learning. Donald Farmer, an architect of course-embedded assessment at King's College in Pennsylvania, writes:

Although many factors contribute to successful student learning, there are two factors that appear to be vital links connecting specific levels of achievement with anticipated learning outcomes. One is to transform students from being passive to being active learners and the other is to make assessment of learning an integral part of the teaching-learning equation. Assessment can play a critical role in developing students as learners if assessment is understood to be formative as well as summative. Assessment best serves as a strategy for improving student learning when it becomes an integral part of the teaching-learning equation by providing continual feedback on academic performance to students. This can be achieved most effectively by designing an assessment model in course work and intended to be both diagnostic and supportive of the development of students as learners. Assessment encourages faculty to focus on the actual learning taking place for students, and to recognize that teaching and assessment are strategies, not goals. (p. 199)

In other words, once an institution forms goals for student learning and develops criteria to measure how student learning outcomes meets those criteria, then colleges, departments, and instructors can develop curricula and activities to help students become active learners, and use assessment to provide feedback both to students and to the institution on how well students are meeting those goals.

Because outcomes and assessment are now discipline- and institution-oriented, curricula can be designed to focus on the development of skills across years and disciplines. For example, when students graduate, what sort of critical thinking skills does an institution want them to have? Then, how should freshman-level courses begin developing critical thinking skills, sophomore-level further that development, and so on? An integrated curriculum can help students better internalize critical thinking by (1) ensuring that it's a goal of all courses and (2) overcoming the compartmentalization and fragmentation of knowledge that occurs when skills are not transferred across years and courses. And flexibility is built in because although the goals are institution-wide, the curricula to obtain those goals are determined by individual instructors and departments.

Tim Frederick (via Bud and Nancy) are discussing the "lies" teachers tell their students, one of which seems to be saying "this is an important book." They make some good points, which I'll come back to, but first I want to look at some of the assumptions being made.

According to Tim, this is called a lie because: "How did we become so arrogant as to think we had the right to say which books were important to read and which aren't? "

I'm not sure we should consider arrogance as a form of lying, and I'm not sure that it's rights that are the issue. Shouldn't it be responsibility? That is, teachers have the responsibility (and are accountable to parents and society) for selecting those books that will best enable students to learn. Actually, depending on the grade level and subject, school administrators often do the choosing of books for the school's curricula, books that must meet a state's criteria, as determined by state departments of education.

Tim adds:

What disturbs me most is that when we say this, we take a little power away from students AND hurt their critical thinking. Shouldn't they decide what's important and why? That can be empowering, as well as exercise the critical thinking muscle of evaluating. They would have to be able to justify their reasons for thinking a book is important and we can share how other people define "important". Students can further evaluate others' criteria for "importance". How many perfectly good lessons surrounding this are thrown away when we decide what's important?

Part of this argument is a value judgment of "empowering" students, of appealing to egalitarian values. In the classroom, however, such an appeal should be secondary to principles of learning. No research on learning is cited in these claims, nor is any evidence given to support that "empowering" students will help them learn better. To be fair, Bud just wrote a few paragraphs, not an academic essay. However, with such strong claims, I'd like to see a little evidence.

Another assumption without evidence is that saying "This is an important book" somehow "hurt[s] their crtical thinking." Actually, this assumption is a shift from the perspective of teachers wanting students to read "good" books to a position on the value of "critical thinking," as if these positions were exclusive. Of course, I can imagine teachers who pontificate without inviting students into the discussion, but that's not at issue here.

There is no getting away from the teacher's responsibility. Consider Bud's last sentence, "How many perfectly good lessons surrounding this are thrown away when we decide what's important?" Who decides what are "perfectly good lessons"? If we carry this perspective to its conclusion, then we should have the children evaluating the criteria for "perfectly good lessons" and the criteria for good teaching. In fact, we should listen to the commplace saying that one learns best by teaching, and we should just have the children do the teaching, too. Then what would the teachers do?

Now looking at the positives of Bud's argument, It does make sense that students need to learn and evaluate "how other people define 'important'" and also develop critical thinking. The issue is how to do this. Perhaps we can draw from ACT-R learning theory. Anderson and Schunn (2000) write,

There are no magical properties conveyed upon a knowledge structure just because it was self-generated. If all things were equal it would be preferable to have children learn by generating the knowledge (due to the redundant encoding). However, because of difficulties of generation and dangers of misgeneration, things are not always equal and it can be preferable to tell the knowledge.

...

Thus, ACT-R's theory of procedural learning claims that procedural skills are acquired by making references to past problem solutions while actively trying to solve new problems. Thus, it is both a theory of learning by doing and a theory of learning by example.

Simply providing the learner with examples is not sufficient to guarantee learning in the ACT-R theory. The sufficiency of the production rules acquired depends on the understanding of the example.

Anderson and Schunn add, "For competences to be displayed over a lifetime, time on task is by far and away the most significant factor." That is, learners must practice a lot, whether critical thinking or other skills. The problem is one can practice the wrong skills, in which case "practice makes imperfect." In other words, learners need feedback and at times explicit guidance to make their practice effective. Of course, they can get that when they choose their own books. And now we're back where we started: How does the teacher choosing a book hurt students?

Reference:

Anderson, John R., & Schunn, Christian D. (2000). The implications of the ACT-R learning theory: no magic bullets. In R. Glaser (Ed.), Advances in instructional psychology (Vol. 5). Mahwah, NJ: Erlbaum.