January

The journal Science has an interesting article Computers as Writing Instructors, an article that stirred up a conversation on the WPA listserv. Some of the concern relates to what Richard Haswell, a professor emeritus of English at Texas A&M University, Corpus Christi, stated in the article:

One peril, says Haswell, who has studied both traditional and electronic measures of writing, is that the programs pick up quantifiable indicators of good writing--average sentence length, for instance--yet ignore qualities such as whether an essay is factually accurate, clear, or concise, or whether it includes an element of wit or cleverness. "Those are all qualities that can't be measured by computer," he says.

When I read such statements, I wonder if supervisors worry about architects using computers to create and modify designs because computers can't measure the aesthetic qualities of the design. The computer is a tool. Of course, any tool can be abused. And if all teachers did were to use the program for assessing student writing and never offered their own feedback, that would be a problem. Still, no one seems to worry about architects using computers.

Flow
One thing I see as good about such tools, if they work (which is a requirement, of course), is that they incorporate conditions of flow, a state of intrinsic motivation, such as:

  1. immediate feedback
  2. clear goals
  3. focused attention
  4. tasks that challenge (without frustrating) one's skills

Motivation is crucial in engaging students to spend time on their writing, to work at improving it. (For more on motivation and flow, see Engagement and Flow.)

Immediate Feedback
Although learning and instruction may meet conditions 2-4, seldom is immediate feedback given in composition classes. In one semester, students might write from three to six essays, depending on the instructor, which means that feedback on essays is given every two to three weeks. In addition, the feedback of peer reviews generally takes place hours after the last version, unless a student pulled an all-nighter for an 8:00 am class. In this case, most of the feedback will be seen through a haze. The feedback of instructors usually occurs days later after they have looked at all of them.

The importance of immediate feedback with cognitive tutors has been demonstrated in teaching LISP, algebra, and geometry. In their abstract, Anderson et al. write,

Early evaluations of these tutors usually but not always showed significant achievement gains. Best case evaluations showed that students could achieve at least the same level of proficiency as conventional instruction in one-third of the time.

Those "best case evaluations" are in the lab where there are no distractions, but even in real classrooms, Anderson and Schunn (pdf) have found achievement gains equal to one letter grade. Learning is directly due to time on task, that is, practice. (Of course, practicing the wrong tasks leads to mislearning.) Thus, providing immediate feedback helps to eliminate wasted time in trying to figure out how to do something, which in turn, decreases the time required to learn a particular activity.

Now, writing is a fuzzier than math. Math usually has a correct answer, while writing doesn't. But perhaps by limiting one's focus to particular aspects of writing, such as coherence, cognitive tutors like WriteToLearn may be of help to students in developing their writing.

Interaction
Alex Reid, however, questions interacting with computers instead of with other students:

The Science article explains that these computer programs are necessary because teachers cannot read and respond to as much student writing as the students should be doing; so the machine reads them instead. Hmmm.... what other possibilities could there be I wonder?  .... Maybe the other students? Maybe the could be reading each others' work? Maybe they could even actually be writing to one another? Maybe they could be using these networks to write to other students around the world? Maybe they could be composing texts that were addressed to other humans rather than to machines and which might actually have some real meaning and value?

I think that interaction with others is important for learning, too, but that does not necessitate an either-or dichotomy of interacting with students and others versus interacting with computers. In fact, using a computer doesn't necessarily mean that students are not interacting with others. Anderson et al. wrote,

When students are in the laboratory, they are working one-on-one with the machines, but that hardly means they are working in isolation. There is a constant banter of conversation going on in the classroom in which different students compare their progress and help one another. ... An effective teacher is quite active in such a classroom, circulating about the class and providing help to students who cannot get the help they need from either the tutor or their peers. (p. 200)

In addition, it would seem to be useful for students to have such a program at home when they are alone, according to Anderson and Schunn, because of "difficulties of [self-]generation and dangers of misgeneration." In other words, much time can be wasted in writing to others and also mislearned, with respect to learning specific aspects of writing.

Meaning and Value
As noted above, Reid's thrust is on the "meaning and value" of student writing. However, meaning and value shouldn't be limited to writing to people. It's interesting that just as we don't question architects using computers to aid in creating aesthetically pleasing buildings, neither do we question coaches who have their players practice drills over and over and over to perfect their skills. No one says, These drills don't have meaning. And no one asks, Why don't you just let them play games that have meaning instead of mindless drills? No one does because it's understood that honing one's skills is valuable for playing the game well. And skills like coherence are crucial to writing well.

Meaning and value are relative. What meaning and value do videogames have? Isn't it primarily just for pleasure, part of which derives from improving one's skills. And for that pleasure, people, especially youngsters, can play for hours on end, as can athletes. Supposedly, ex-NBA star Larry Bird felt shooting "200 free throws before school, every day" had meaning and value. From the article, Jenkins' students apparently found the writing tutor meaningful and valuable, as indicated by their improvement in writing:

Jenkins suspects that English language learners (ELL)—educationese for children who speak another language at home—may be among those who can benefit the most from using writing-instruction software. Last year, 92% of his ELL students passed the writing portion of the state assessment test, he says, compared with 31% of his ELL students before he started using the software. That percentage is also well above the statewide ELL rate of 58%.

That's a tremendous difference. Of course, there is a danger of limiting writing to what a standardized test can measure, and of dumbing down instruction, which is well-documented in George Hillocks' book The Testing Trap: How State Writing Assessments Control Learning.

Having said that, although writing with a purpose to others, just like practice, can help to improve one's writing, such an approach has its limits. And to move beyond those limits requires studied practice (see The Expert Mind in Scientific American). And if some computer program can help in that regard, great!

Motivation
As noted above, cognitive tutors, if designed appropriately, can motivate students to spend more time on task, which is the most important factor in learning. Anderson et al. wrote,

Students' own attitudes to the tutor classrooms are quite positive, to the point of creating minor discipline problems. Students skip other classes to do extra work on the tutor, refuse to leave the class when the period is over, and come in early.

How often does that happen in our classes? Students coming in early, not wanting to leave at the period's end, and preferring to do our homework instead of others'?

In their conclusion, Anderson et al. mention an anecdote:

The student, frustrated by restrictive access to the LISP tutor, deliberately induced a 2-day suspension by swearing at a teacher. He used those 2 days to dial into the school computer from his home and complete the lesson material on the LISP tutor. (p. 204)

And the Science article says that Jenkins found similar results with his students:

Maria had more confidence in her writing abilities--and passed the writing portion of the state assessment test. "It's not a cure-all, but what a difference it's made in what the kids have shown they can do," says Jenkins, who began using the software last year.

As Anderson et al. assert, "learning achievement is a very empowering experience," and one that has "meaning and value" to the students.

Values
So, why wouldn't compositionists applaud the use of computers as tutors? Asao Inoue, in his review of the book Machine Scoring of Student Essays: Truth and Consequences stated,

More importantly, most in the present collection do not acknowledge or address (accept [sic] arguably Haswell, Anson, and Broad) a core premise of the book, that what is at issue is a paradox of technology. We already use and need technologies of assessment, yet we are fighting against certain kinds of technologies because they take us in different directions, shape our practices, assumptions, student arrangements, and working conditions in ways we do not value enough to pursue.

This particular technology is too quickly dismissed. Not because it may not work but because present practices and assumptions have attained canonical status rather than being critically re-examined. Of course, we shouldn't uncritically accept new technology, either. But if it meets my values of motivating students to work on their writing and actually helps to improve their writing, then I'm interested in learning more about it.



Just finished re-reading Mark Buchanan's book Nexus: Small Worlds and the Groundbreaking Theory of Networks. The notion of "small worlds" comes from Stanley Milgram's letter experiments on how people are interconnected in the U.S. He sent out letters to randomly selected people in different cities with instructions to send them to some other individual. If they didn't know that person, then they would forward it to someone they felt might know the person. Milgram found that the letters made it to their destination with an average chain length of 5-6 people. (For a critique of problems with Milgram's study see Could It Be a Big World? by Judith Kleinberg and her shorter followup Six Degrees of Separation: An Urban Myth? (via Rebecca Skloot)).

Whether or not the six degrees of separation is accurate, the small-world phenomenon apparently is found in many different network systems—for example, food webs, cellular metabolism, the Internet, language, and so on—and the organizing principles of social networks are apparently the same as those of other small worlds. Buchanan wrote,

This feature has a specific mathematical signature: the power-law or fat-tail pattern for the distribution of elements according to how many links they have. And this signature turns out to be nearly identical from one kind of network to the next.

What we see then is a kind of natural order that for mysterious reasons seems to well up in networks of all kinds and that does so despite the complexities of their individual histories (p. 91).

In this power-law pattern, a few nodes are highly connected, resulting in clusters, while the majority have only a few connections. We can see that in the classroom, too. The teacher is a hub, connecting to all the students, and among the students, some have more connections with classmates than others do. In a small class, this might not play a significant role, but in large lecture classes—I remember my introductory chemistry course with 400 students—it might. And the teacher is not the only hub nor always the biggest one in a particular class.

Some connections between people are obviously stronger than others. For instance, my connections, or ties, to faculty in other departments are weaker than those with my colleagues in composition. I interact with other faculty infrequently while I talk with composition faculty almost every day, for longer periods of time, and in more depth on our common subject, composition. Strong ties result from interaction over time and affect (e.g., trust, respect, and friendship). So, another aspect of the small world phenomenon is the notion of strong and weak ties formulated by Mark Granovetter.

Both types of ties are important. Strong ties play a role in motivation, support, and identity. Weak ties have a role, too. Granovetter's seminal article "The Strength of Weak Ties" (pdf) (see also The Strength of Weak Ties: A NetworkTheory Revisted (pdf), written 10 years after the seminal article) showed that weak ties acted as bridges to information and sources different from one's networks of strong ties. In small classes, strong ties would be dominant among classmates while weak ties would connect to others outside the classroom. Of course, students have other networks (for example, family and friend networks, which would consist of strong ties) outside the classroom and so would have strong ties outside the classroom, too. Rather, I am thinking of ties connecting to networks that would have information of use to the class's subject matter.

So, I've been thinking how to take advantage of weak ties to enhance learning in my composition courses. One way is to have each student establish a blog on a topic of personal interest and to subscribe to at least five other blogs writing on the same topic. However, with students writing on different topics, it would be unlikely that their weak ties would be bringing in information and resources of interest to the entire class. So, in addition, students would also be posting on the rhetoric, both textual and visual, contained in those other blogs, leading to interaction with their classmates on rhetorical similarities and differences between blogs and blog subjects, and between blogs and other genres the class would use or come across. If you have any ideas or suggestions on taking advantage of the "strength of weak ties" in learning, email me. I'll collate them them in a later post.



Two recent articles are asserting that traditional methods of certifying and selecting teachers do not work well and that alternative methods may help.

The 'Certified' Teacher Myth

Like all unions, teachers unions have a vested interest in restricting the labor supply to reduce job competition. Traditional state certification rules help to limit the supply of "certified" teachers. But a new study suggests that such requirements also hinder student learning.

Harvard researchers Paul Peterson and Daniel Nadler compared states that have genuine alternative certification with those that have it in name only. And they found that between 2003 and 2007 students in states with a real alternative pathway to teaching gained more on the National Assessment of Educational Progress (a federal standardized test) than did students in other states.

The authors conclude that strict certification standards hinder teaching competence. It's not clear why doing so would achieve that effect. Yet, as mentioned in What Works in Teaching, one study found that TFA teachers outperformed experienced, certified teachers. And another recent study found a somewhat similar finding: Alternative route teachers who took an intensive course on teaching outperformed experienced, traditionally certified teachers in some subjects (not all), with math again having the greatest differences.

The authors state that those states with genuine alternative certification have more minorities teaching, and assert that minority students benefit from having minority teachers. I'm guessing again, but what would make sense to me is that alternative route teachers have experience in their subject matter that enhances their instruction. Even so, the results of these intensive courses call into question the present methods in schools of education of preparing teachers for the classroom.


Most Likely to Succeed: How do we hire when we can't tell who's right for the job?

This article (via Stephen Downes) compares selecting future teachers to predicting who will become a star quarterback in the NFL. With respect to the NFL, prediction has more failures than sucesses. Yet they have easily identifiable criteria for selection, years of statistics to refer to from when the player was in high school, and then in college, and videos of their performance over time on the field. In contrast, for future teachers, the criteria are more vague, there are no years of statistics, and no videos of their performance over time. However, even if there were such evidence, we still wouldn't be able to predict who would be a good teacher any more than they can pick a quarterback:

The problem with picking quarterbacks is that Chase Daniel’s performance can’t be predicted. The job he’s being groomed for is so particular and specialized that there is no way to know who will succeed at it and who won’t. In fact, Berri and Simmons found no connection between where a quarterback was taken in the draft—that is, how highly he was rated on the basis of his college performance—and how well he played in the pros.

Of course, the difference between good teachers and not-so-good teachers has implications for what students learn:

Eric Hanushek, an economist at Stanford, estimates that the students of a very bad teacher will learn, on average, half a year’s worth of material in one school year. The students in the class of a very good teacher will learn a year and a half’s worth of material. That difference amounts to a year’s worth of learning in a single year. Teacher effects dwarf school effects: your child is actually better off in a “bad” school with an excellent teacher than in an excellent school with a bad teacher. Teacher effects are also much stronger than class-size effects. You’d have to cut the average class almost in half to get the same boost that you’d get if you switched from an average teacher to a teacher in the eighty-fifth percentile. And remember that a good teacher costs as much as an average one, whereas halving class size would require that you build twice as many classrooms and hire twice as many teachers.

And certification and degree level doesn't make a difference in teaching quality, either:

A group of researchers—Thomas J. Kane, an economist at Harvard’s school of education; Douglas Staiger, an economist at Dartmouth; and Robert Gordon, a policy analyst at the Center for American Progress—have investigated whether it helps to have a teacher who has earned a teaching certification or a master’s degree. Both are expensive, time-consuming credentials that almost every district expects teachers to acquire; neither makes a difference in the classroom. Test scores, graduate degrees, and certifications—as much as they appear related to teaching prowess—turn out to be about as useful in predicting success as having a quarterback throw footballs into a bunch of garbage cans.

That graduate degrees have an effect on teaching ability seems to call into question an earlier post stating that a thorough knowledge of subject matter was one characteristic of outstanding college teachers. But not necessarily. We would need to see what sorts of graduate degrees are being considered, whether there is a difference between a masters degree in education and one in the subject matter. And also how well one did in the graduate level subject matter courses.

From another perspective, I'm reminded of my first year teaching English in Istanbul to students admitted into Marmara University, an English-medium institution. Before they took courses in their majors, they had to take an intense, six-hour-a-day course for eight months to learn English. It's really not possible, but that's what the students had to do. Anyway, I had just finished my master's in Teaching English as a Second Language (ESL) (plus I was certified in science and biology at the secondary level) in which I was introduced to a variety of theoretical courses, including a few that covered methods of teaching ESL. But no practice. I found myself flying by the seat of my pants, using very little of my graduate education. Apparently, education separated from contextualized practice is of little help, and soon forgotten.

Actually, it makes quite a bit of sense. Doctors have four years of medical education, and then at least three years of intense internship under the supervision of experienced doctors—specialists considerably more. Would anyone really want to undergo an operation by a doctor who knew the book procedures backwards and forwards but had no experience in surgery? Engineers, after graduating, go into the workplace surrounded by more "practiced" engineers and learn through a combination of doing, observing, collaborating, and being supervised. And so on for other disciplines. But teachers, after their education, which although it includes a semester or two internship, go into the classroom doing alone—generally not observing other teachers or team teaching—and receiving limited supervision.

Learning follows a power-law relationship:
Anderson and Schunn in their article "The implications of the ACT-R learning theory: No magic bullets" (pdf) state that there are three learning processes governed by power laws:

1. Power Law of Learning. As a particular skill is practiced there is a gradual and systematic improvement in performance which corresponds to a power law. ...

2. Power Law of Forgetting. As time passes performance degrades, also according to a power function. ...

3. Multiplicative Effect of Practice and Retention. Most important, the Base-Level Equation implies a relationship between the combined variables of amount of practice and duration over which the information must be maintained. ...

This implies performance continuously improves with practice ... and continuously degrades with retention interval .... Most significantly the two factors multiply which means that increasing practice is a way to preserve the knowledge from the ravages of time.

Naturally, learning and practice need to be on target, as Albert Ip comments:

My daughter's swimming coach puts it very well: "Practice makes your stroke permanent. If you practise bad technique, you just become a more efficient bad swimmer with the bad stroke. It is even more difficult to unlearn the bad strokes."

With that caveat in mind, it's obvious that doctors and engineers follow up their book education with considerable practice in the presence of others, observing others, and receiving feedback from supervisors who see their work on a frequent basis. Other factors being equal, their environment supports learning, practice, and retention. Teachers, on the other hand, generally work alone in an environment that doesn't support collaboration, frequent feedback, or observation of others. Even if their education courses were terrific, the Power Law of Forgetting ensures that the content of all but the most recent ones is likely to be forgotten. It certainly was in my case in Turkey. And what if they don't forget, are they implementing it correctly? Or practicing "bad technique"? Without targeted feedback, they may simply become "more efficient bad" teachers.

As opposed to credentials, the most important element in good teaching, according to this article, was feedback:

Of all the teacher elements analyzed by the Virginia group, feedback—a direct, personal response by a teacher to a specific statement by a student—seems to be most closely linked to academic success. ... [Not simply] "Yes-no feedback ... which provides almost no information to the kid in terms of learning."

In quite a few ways, the necessity of feedback, especially immediate feedback, makes sense (although see Harold Jarche's post noting the importance of the when and how of feedback). It's necessary for flow to take place, and it's an important part of developing procedural knowledge (according to ACT-R Theory). However, the ability to give appropriate and immediate feedback in the classroom cannot be measured before one begins to teach—thus, the problem in ascertaining who will be "good" teachers on the basis of credentials. Perhaps what is needed is ongoing professional development that focuses on giving feedback. As Downes comments,

there seems to be nothing that prevents us from either teaching these strategies to new teachers, or evaluating them in teachers put up for tenure.

Perhaps instead of taking two years of education courses, students might replace them with

  • one more year of subject matter courses,
  • a one-year internship in a work environment appropriate to their major, and
  • an intensive summer course right before teaching.

Once teaching, they would receive

  • a year of close mentoring with respect to feedback and other elements in that course, thus contextualizing their education and not letting it be forgotten,
  • professional development that includes ongoing feedback and collaboration throughout the school year, and
  • professoinal internships in their discipline either during the summer or perhaps a semester internship every four or so years.

Of course, I'm just speculating. But the fact that alternative route teachers can outperform experienced traditional route teachers, especially in math and the sciences, indicates that, at the least, we need to understand

  • why alternative route teachers who undergo these particular training programs are outperforming experienced teachers in some fields and
  • how traditional teacher training can be improved.

Somewhat related posts:
Just-in-time Learning
Engagement and Flow
Learning with Examples



Better Learning with Sites and Sounds (by Andy Guess from InsideHIgherEd)

One qualitative study ... found that students who create and edit documents using Web-based collaboration tools include more complex visual media in their assignments — and come away with a better understanding in the process. Another ongoing experiment finds, with statistical significance, that instructors can be more effective in grading students’ work if they record their comments directly into documents as audio.

Perhaps the first finding in this article sheds some light on the finding from the previous post in which the NSSE found that online learning resulted in "deep" learning. Using "more complex visual media," that is, leads to thinking about what one is learning in ways that offline learning doesn't. And using that media depends, in part, on how easy it is to use them.

The second finding is interesting, too, because it's not the same as giving them an audio file that is separate from the document. According to Ice, one of the researchers in the article, separating the audio file results in little additional learning. I'm guessing that having the audio play as you read focuses the listener more on those areas of writing that need work and why. To use this method, however, requires Adobe Acrobat Pro, about a $200 expenditure. (Students can read and listen with the free Acrobat Reader.) Still, read the Virtual Canuck's experience and enthusiasm for Marking with Voice Tools.



E-mails 'hurt IQ more than pot' (via Bruce Hoppe)

Workers distracted by phone calls, e-mails and text messages suffer a greater loss of IQ than a person smoking marijuana, a British study shows.

Fascinating comparison! I don't know about the effect on IQ, but distractions take away time from being, or becoming, competent at anything. (See The Expert Mind and Twitter, or How to Fritter your Life Away.) And again, it underscores the negative effects of multitasking on learning. (See Myths of the Digital Generation, Part I, Part II, and Cont'd.)


Scientific Journal to Authors: Publish in Wikipedia or Perish (via Stephen Downes)

Every day, hundreds of articles appear in academic journals and very little of this information is available to the public. Now, RNA Biology has decided to ask every author who submits an article to a newly created section of the journal about families of RNA molecules to also submit a Wikipedia page that summarizes the work. As Nature reports, this is the first time an academic journal has forced its authors to disseminate information this way. The initiative is a collaboration between the journal and the RNA family database (Rfam) consortium led by the Wellcome Trust Sanger Institute.

This is a great move toward making knowledge available to all. I hope other journals will follow suit.


iTunes Study Podcasts

Clicking on the above link opens up iTunes and takes you to Wired Study Tips, podcasts on test preparation, study skills, and time management from Continuing and Professional Studies at Texas A&M University.


Update:

How the City Hurts Your Brain

Now scientists have begun to examine how the city affects the brain, and the results are chastening. Just being in an urban environment, they have found, impairs our basic mental processes. After spending a few minutes on a crowded city street, the brain is less able to hold things in memory, and suffers from reduced self-control. While it's long been recognized that city life is exhausting -- that's why Picasso left Paris -- this new research suggests that cities actually dull our thinking, sometimes dramatically so.

"The mind is a limited machine,"says Marc Berman, a psychologist at the University of Michigan and lead author of a new study that measured the cognitive deficits caused by a short urban walk. "And we're beginning to understand the different ways that a city can exceed those limitations." ...

In a study published last month, Berman outfitted undergraduates at the University of Michigan with GPS receivers. Some of the students took a stroll in an arboretum, while others walked around the busy streets of downtown Ann Arbor.

The subjects were then run through a battery of psychological tests. People who had walked through the city were in a worse mood and scored significantly lower on a test of attention and working memory, which involved repeating a series of numbers backwards.

I wonder how urban effects compare with multitasking.

However, on the plus side:

Given the myriad mental problems that are exacerbated by city life, from an inability to pay attention to a lack of self-control, the question remains: Why do cities continue to grow? And why, even in the electronic age, do they endure as wellsprings of intellectual life?

Recent research by scientists at the Santa Fe Institute used a set of complex mathematical algorithms to demonstrate that the very same urban features that trigger lapses in attention and memory -- the crowded streets, the crushing density of people -- also correlate with measures of innovation, as strangers interact with one another in unpredictable ways. It is the "concentration of social interactions" that is largely responsible for urban creativity, according to the scientists.