8. Evaluating Writing

 

While many writing instructors enjoy reading and commenting on student drafts, most struggle with putting a grade them. Evaluating writing with grades and other markers of success or failure often feels difficult, even painful, and sometimes reductive. How best to assess student writing and provide appropriate feedback to support grades?

As we saw in Module 5, there are many ways to offer useful feedback to your students. Research shows that in order for feedback to be effective, and to help students develop the skill of self-evaluation, the criteria for success must be fair, consistent, public, clear, and responsible.

We focused on formative feedback in Module 5, intended to help students revise their work before it is graded. Summative feedback, in contrast, evaluates the quality of a “finished” product. Like formative feedback, summative feedback should address the strengths and weaknesses in a paper, but it typically explains the connection between those strengths and weaknesses and the grade the paper has received. Comments generally do not include advice about how the student might revise this specific paper, although they might include advice about how the student could improve his or her work in the future based on their work in the current assignment.


Considerations for Providing Summative Feedback ^

Higher Order vs. Lower Order Concerns: Higher order concerns are typically conceptual and structural. Do the ideas in the paper make sense? Are claims supported with evidence? Do the paragraphs follow a logical order? Lower order concerns have less to do with meaning than with “correctness,” i.e., grammar, style, and formatting.

While both higher order and lower order concerns are important, instructors can inadvertently send contradictory messages when they try to address both at the same time. For example, questions and comments that suggest that a paragraph needs to be entirely rethought and rewritten conflict with sentence-level markups that suggest that the sentences already present in the paragraphs will remain where they are once they are cleaned up.

Lower order concerns may require special treatment for writers who use English as their second language (L2 writers.) For more information on responding to lower order concerns in the work of L2 writers, see the “Providing Grades and Feedback to Multilingual Writers” resource and its affiliated supplements.

Drawing Connections Between the “Big Picture” and the Details: One common approach to commenting on student work is to offer praise, criticism, and questions in the margins of a paper, and then to provide a more general overview of what the paper does well and what it does poorly in a head note or end note. Ideally, this method should allow students to understand not only what the paper’s strengths and weaknesses are, but also where specific examples of these occur in the text.

However, it can be surprisingly difficult for students to understand the connections between overview notes and marginal comments unless an instructor spells those connections out explicitly. For example, an instructor might ask questions in the margins that indicate the need for evidence to support key claims, and might explain in his or her overview note that the paper’s greatest weakness is its failure to provide evidence. Yet, students who are less experienced in thinking about papers in terms of evidence and claims will not necessarily understand that marginal questions such as “Why do you think so?” indicate places where evidence needs to be provided. It can be extremely helpful for an instructor to include a sentence in his or her end comment that says something like, “In my marginal comments, I’ve tried to show you where your claims lacked sufficient evidence to support them by asking why you think the things you say are true.”

Tone: Students should feel that their instructors are interested in what they say and how they say it. Cultivating a conversational tone and indicating that you understand and appreciate what the student attempted to accomplish—whether or not the student actually achieved his or her goals—can go a long way toward helping the student accept your evaluation rather than responding defensively.

Remember, however, that the tone in the marginal comments should be consistent with the head/end note, and that tone should help make sense of the grade. For example, if an instructor is effusive in the marginal comments but gives the student a C on the paper, the student may feel confused about why he/she received the grade or how to improve on the next paper.


Ways of Assessing Papers ^

Grading Rubrics

As we saw in Module 5, rubrics can be useful for giving both formative and summative feedback. There are two basic types of rubric: analytic and holistic. An analytic rubric divides student work into component parts—e.g., tasks that make up the whole, or individual criteria such as ideas, organization, voice, mechanics, etc.—with descriptions of what high-, mid-, and low-level quality work looks like for each component (see “Sample Analytic Rubric” supplement). This type of rubric often takes the form of a table, with the tasks/criterion being judged along the left column, and assessment numbers or language along the top row.

A holistic rubric provides guidelines for various levels of achievement in the work overall, rather than by categories, tasks, or component parts (see “Sample Holistic Rubric” supplement). Both kinds of rubrics can help you save time while improving the consistency and clarity of your feedback. A holistic rubric is often structured as a commenting form with sentence- or paragraph-length descriptions of different levels of competencies, rather than a table. Detailed advice for creating and implementing rubrics of both kinds is available in Supplement 1: “Designing and Using Rubrics.”

When creating your assignment on Canvas, you can also include rubrics in the Assignments page. It’s a handy way to let students know–even before they write the essay–how you will be evaluating the piece.

 


Assessing Learning Over the Semester ^

Even more difficult than grading individual papers is assessing learning over the course of the semester. It’s easy enough to average grades and weight them appropriately, but how do you know whether a student has mastered skills in writing over a period of time?

There are many different approaches that instructors use to answer that question, and the trick is to figure out the best approach for you, as you consider your own and/or your department’s philosophy of assessment. You must also consider the criteria established by the professor teaching the course.

A few methods of making students aware of their learning throughout the semester include:

  1. Referring to earlier drafts and graded papers as you comment on their current assignment. This process may require you to have a copy of their earlier paper as you read their current one. Comments such as, “Here you provide a much more detailed description of the scene” or “This thesis is much more complex and provocative than in paper 1” allow students to see their development over time.
  2. Tracking skills and growth on a formal grid. You  may want to keep track of student writing development on a grid or spreadsheet that allows you to make brief comments as students master concepts or skills.
  3. Providing final notes to students at the end of the term. Some instructors find that writing a letter summarizing the student’s growth useful both for the student and for themselves as they make their final grading decisions.
  4. Assigning a reflection paper that asks students to compare/contrast their first writing assignment to a later one. This need not only focus on the final product; you should ask them questions about the process as well. (Cultivating student reflection on their writing is valuable at all stages. This Sweetland resource on Cultivating Reflection and Metacognition offers a range of strategies and exercises to use throughout the semester.)

One way of encouraging students to reflect on their own learning is to have them fill out their own writing assessment grid at the end of the semester. This self-assessment grid, designed by Paul Barron, aims to help students track their own progress over the course of the semester. As you can see, students track not only the particular writing skills but also “four dimensions of learning,” which include confidence, understanding, application, and problem solving.


Keeping It Real: The Psychology of Grading  ^

You have a stack of papers. It’s the weekend. You told your students you would have them back by Monday.

It’s easy to slip into frustration mode: the stack of papers seems to grow every time you look at it, and the idea of grading becomes less and less appealing. You may procrastinate. Now it’s Sunday night; your papers are not graded, and you approach them with cranky disdain.

It’s easy to put off grading for other, seemingly more rewarding tasks, or to dread the stack, at least initially, but most instructors find that once they get started, the anxiety and negativity dissipate and they experience real satisfaction, even pleasure, from the process. Rob Jenkins, in The Chronicle of Higher Education, provides some tips for approaching your grading, such as:

  • Don’t grade all at once. Many instructors like to grade a few papers a day, rather than doing them, say, the night before you told students you would hand them back.
  • Schedule grading time in your calendar. This will force you to keep on track, especially as your week gets busy.
  • Have a realistic return policy. Don’t tell students you will get papers back when you know you are going to a conference over the weekend. Students are generally happier to know that you will take a little longer to return papers if you tell them in advance. They are less forgiving of a missed deadline.
  • Limit your grading time on each essay. Don’t dwell on each essay for hours. Read with a clock nearby, or even a timer.

Remember, too, that the more you grade, the more you will figure out how you work best. It’s a lot like writing. Many instructors like to grade a few papers a day; some swear by grading them all at once, in one or two long grading sessions. Obviously, there is no formula. It does make sense to stay positive. And, if it helps, have caffeine nearby.

Canvas Tip: SpeedGrader ^

For those of you just learning Canvas, you should find the Speedgrader option useful to provide feedback and grades. When students submit their assignment, on the right-hand side of the screen you will see a prompt for SpeedGrader. Once there, you will find the student’s submitted text, where you can submit marginal comments easily. (There is a “comments” tab at the top of the page. Highlight what you want to focus on, then write your comments.) On the right, you will see you can also submit the grade, as well as “Assignment Comments.” Note, however, that you cannot revise your summative comments once you submit them.

Another tip: If you want to return grades and comments to all your students at the same time, there is a “Mute Assignment” tab at the top of the page. After you mute the assignment, students will not see what you have written until you “unmute” it. Some instructors find it helpful to grade all their assignments first before giving them back to students; others like to give their papers back as they grade them. In either case, it’s good to know you have the option to give them back all at once.

Additional Resources: ^

For one more example of a useful rubric, see Razaei and Lovorn, Using Rubrics.

When grading, scoring guides will address issues of student motivation. See White, Edward M. “Issues in Grading Writing and Using Score Guides.” Assigning, Responding, Evaluating: A Teacher’s Guide. 4th ed. Boston: Bedford/St. Martin’s, 2007. 73-85. Print.