University Writing CenterGrading papers is probably the most time consuming task faced by professors, and writing assignments are undoubtedly the most difficult to evaluate. How much should cogent arguments and accurate facts count? What about organization and style? Are grammatical errors important? Word choice? Should I consider effort and improvement? Are long papers better than short ones? What about creative thinking? How much do I take off for incorrect documentation?
Making clear decisions about your criteria before you begin to grade papers will help make the grading process quick, fair, and accurate. Ideally, these criteria should be presented in a framework that you can share and discuss with your students as part of the assignment. In fact, using a version of your grading criteria as a cover sheet for the paper can help you score the paper and provide useful feedback to the student at the same time.
“Holistic” scoring guides (also known as “rubrics”) typically divide the papers into a set number of levels, usually five or six, and describe the characteristics of papers at each level. The scoring guides used for the English Placement Test (EPT), or our own Graduation Writing Test (GWT), are typical examples. The holistic scoring process involves weighing the various factors described in the scoring guide and assigning a score, effectively sorting the papers into piles of similar quality. This type of scoring rubric is often used in large-scale essay tests. See Figure 1 for an abbreviated version of the EPT Scoring Guide.
“Analytic” scoring guides are designed to rate papers on separate criteria. Where holistic scoring assigns a single score that represents a weighing of all the different factors addressed by the scoring guide, an analytic score might assign a high score to one factor, such as “thesis,” and a lower score to another, such as “grammar and mechanics.” Analytic scoring is slower than holistic, but provides more feedback and is better for diagnosing problems with the writing.
To create your own scoring guide, start by listing the features of the paper you expect in response to the assignment. This will include such things as appropriate format and genre, what kinds of data should be collected, what kind of analysis is expected, documentation style, length, audience, etc. Then think of the kind of paper that makes you proud that you taught that student. Describe the qualities that make that paper superior to most that you get. This is the “6” on a typical holistic rubric.
For example, Geri-Ann Galanti in the Anthropology Department at CSULA says that an “A” paper for her Anthropology 250 course has the following characteristics:
1) Follows directions and completes all aspects of the assignment, 2) interviews more than the minimum number of people required, 3) is well-written and well-organized, making it easy for the reader, and 4) does an in-depth analysis of the data.
Galanti says, “If the student does the first 3 things on my list above, they will generally receive a ‘B.’ To get an ‘A,’ however, I must see evidence that they can go beyond mere description and comparison—that they can analyze data.” They must be able to “explain why people respond in certain ways; how it relates to other aspects of their culture, etc.” According to Galanti, “inability to do in-depth analysis is the major problem preventing papers from receiving ‘A’ grades.”
If Galanti were turning this description into an analytical scoring guide, she might want to give data analysis more weight in relation to the other factors she mentions.
The next step is to describe the characteristics of the average paper, the one you would accept as a minimum passing response to your assignment. In the example above, Galanti would probably say that this paper follows the directions, completes most aspects of the assignment, interviews the minimum number of people, is readable although awkward in some places, and does a rudimentary analysis of the data. This is the “4.”
Galanti has already defined depth of analysis as the primary difference between a good response and a superior response. The “B” paper might also be less consistently well-written. This is the “5.”
If we were creating a holistic rubric with a six-point scale, at this point we have the top three scores defined. The next step is to think of the marginal paper, the one that you might send to the writing center for improvement before accepting. This is an important score point, because this is a student who probably has the capability to write a passing paper, but needs some guidance. The scoring guide, and your comments, can help the student learn.
A student at this level may not have completely understood the directions, or may simply have quoted theories and procedures from the book and presented data without analysis. This is the “3.”
The “2” paper usually has many different types of problems combined. The “1” is completely inadequate, or incomplete.
Many professors ask what to do about the language problems of non-native speakers of English. A paper that is a “5” in all other aspects may be a “4” or lower if language problems make it difficult to read and understand. Generally, if the errors in the paper “confuse and/or distract” the reader, the paper shouldn’t pass. A well-designed scoring guide helps balance all of these factors. Of course, a visit to the writing center would help the student learn to proofread for the errors.
To turn a holistic rubric into an analytic one you can take the descriptions of the characteristics of the superior paper and put a numeric scale next to them. This type of rubric was first developed by Paul Diederich in Measuring Growth in English in 1974. See Figure 2 for an example of an analytic scoring guide for English 096.
Circling numbers in the various analytic categories automatically tells the student what the strengths and weaknesses of the paper are, and helps keep the instructor’s judgments consistent and fair.
More techniques for evaluating writing assignments can be found in Effective Grading: A Tool For Learning and Assessment by Barbara E. Walvoord and Virginia Johnson Anderson. The Faculty Center for Professional Development has copies to loan. More complete versions of the scoring guides included here are available on the UWC website.
If you have a scoring guide or grading rubric that you are willing to share with other faculty, please email it to John Edlund at jredlund@csupomona.edu or fax it to the UWC at 909-869-5350.
