Convergence: Theory into Practice

Peer Reviewing the Peer Review Game

Following the development meeting, we sent that script out for blind peer review. It might be interesting for readers who have never participated in a publisher’s review to know how seriously we take the feedback we receive from reviewers. It is written up, mulled over, analyzed, discussed, and incorporated. Peer reviewers are important partners in the development process. Below is an excerpt from an internal memo summarizing our reviewer’s feedback.

May 7, 2006

To: Ryan Moeller, KS, NC, JF, DW, EA, HW, KH

From: KW, LK, EB

RE: Peer Factor: Episode One, Giving Feedback: Questionnaire Summary

The Respondents

Our 11 respondents are experienced composition teachers from 2- and 4-year colleges across the country. Most conduct peer review in class, with students in groups of three or four. Two respondents conduct whole-class, instructor-led sessions in addition to or instead of small-group peer review. Most respondents use peer review response sheets—some distribute these before class, others during. All respondents ask their students to complete a written response to their peers’ work. Several respondents ask their students to read their papers aloud in peer review sessions. One respondent uses Comment [Comment is Bedford/St. Martin’s peer review tool].

General Assessment and Likelihood of Using the Game

Respondents described the game as an accessible introduction to peer review, a useful model for peer review strategies and language, a low-risk arena in which to practice the peer review process, and a constructive starter for class discussions about peer review techniques. As one respondent said, “I think Peer Factor fills a well-needed niche, and does it well.”

Most respondents reported that they would be “very likely” to use the game.

Best Practices

Most respondents thought the best practices were well modeled in the game. As one reviewer said, “I suspect I could have listed several of the best practices in a quiz after reading the script for the game, even without having been a teacher of the subject first.”

Referring to Best Practice #3, Talk the Peer Review Talk, one respondent suggested beefing up the game script with even more peer review words and phrases, especially those related to audience, tone, stance, bias, coherence, structure, thesis, and clarity.

Three respondents suggested that we revisit and clarify the first best practice, which now reads:

As a peer reviewer, your first task is not to evaluate the essay but to see it clearly. Before you begin to make judgments, first be sure you can describe what you see.

While confirming that students do need to verbalize what the writer is saying before responding to how well the writing works, these respondents expressed concern that, as worded, this best practice would reinforce students’ habit of giving vague, uncritical comments.

Two respondents suggested adding a best practice on audience—one that would remind students to respond as readers as well as critics (by pointing to parts of an essay that are amusing or affecting, for example). Two other respondents suggested adding a best practice on students’ responsibility to each other—one that would remind students to respond as fellow writers as well as readers and critics.

We also asked reviewers to evaluate the paper assignment, the characters, the paper topics, as well as each individual response in the script. They told us where we did and did not strike the right balance, challenging but not overwhelming. They evaluated the student papers, the dialogue, and the game play scenarios.