Weekly Teaching Note
November 16 , 2009
Testing for Sanity in Larger Classes
This Weekly Teaching Note, focusing on testing in larger classes, is the third about coping with suddenly increased class size.
Ken Bain, author of What the Best College Teachers Do, observes, “Testing and grading are not incidental acts that come at the end of teaching but [are] powerful aspects of education that have an enormous influence on the entire enterprise of helping and encouraging students to learn.” (Bain, 2004:150.)
It's disheartening to realize that you have 150 students (in each section) and you can't possibly give them anything but a multiple choice test. How can you turn that mundane activity into a positive learning experience?
Knowledge survey*. This powerful tool provides guidance for both students and faculty about what is important in the course. A knowledge survey is a long list (usually 50-200 items) list in which the course learning objectives are framed as detailed questions that test mastery of the objectives. Students do not answer the items while taking the survey (unlike a pre/post test), but respond with a rating of their own confidence that they can respond competently to each query. Knowledge survey items may range from simple factual knowledge to evaluation of substantial open-ended questions. Knowledge surveys can present complex open-ended kinds of problems or issues, and they can assess skills as well as content knowledge. See Nuhfer and Knipp (2003) at http://www.isu.edu/ctl/facultydev/KnowS_files/KnowS.htm for a full description of knowledge surveys.
Why use knowledge surveys in large classes?
- Well-crafted knowledge surveys save time because when students ask what's important, you can simply refer them to the knowledge survey.
- Students don't have to guess what to study, meaning that they can spend their study time more efficiently.
- Knowledge survey items practically beg to be turned into multiple choice questions. By writing a knowledge survey, you have your tests half-written before the quarter begins.
Multiple choice question design. The key to writing questions that test knowledge and skills is to start early. Waiting until the last minute to write a multiple choice test will almost inevitably result in low-level, factual recall questions because they are much easier to write.
The hardest part of writing a good multiple choice question is coming up with the “wrong” choices, or the distracters. Crafting distracters from common misconceptions is the easiest way to write high-quality questions. If you're not sure what the common misconceptions are, an easy way to come up with some is to ask a question in class, and then ask students to write down why they answered what they did on index cards (hat tip to Ertan Salik for this idea).
For excellent information on exactly how to craft a good MC question, see Dewey (2004) at http://www.psywww.com/selfquiz/aboutq.htm .
For information on the mechanics of crafting MC questions, see Kehoe (date unknown) at http://www.testscoring.vt.edu/memo04.html .
Writing in large classes. Next week's teaching note will focus on strategies to encourage writing in larger classes.
*Some readers may remember Ed Nuhfer, our keynote speaker at the April 2009 Provost's Symposium on Faculty Teaching. He developed and extensively tested the knowledge survey concept.
References
Bain, K. 2004. What the best college teachers do. Harvard University Press. Cambridge, MA.
Dewey, R. 2004. Writing multiple choice items which require comprehension. PsychWeb. http://www.psywww.com/selfquiz/aboutq.htm , verified 13 Nov. 2009.
Kehoe, J. Date unknown. Testing memo 4: Constructing multiple choice tests part 1. Virginia Tech Testing & Data Services. http://www.testscoring.vt.edu/memo04.html , verfieid 13 Nov. 2009.
Nuhfer, E., and D. Knipp. 2003. The knowledge survey: A tool for all reasons. To Improve the Academy 21:-59-78.
