Evaluation

Evaluation

Testing
Proctoring
Grading

Testing

Testing and grading are necessary ingredients of teaching and learning. Tests give teachers a way of measuring student learning. Grading gives students feedback on their academic performance.
(See the section on University Policies & Guidelines on academic dishonesty.)

Testing

  • What to test: Learning objectives, major topics, assignments
  • How to test: Frequently, cumulatively, clearly
  • Types of questions: Multiple choice, short answer, essay, problem sets, completions, matching
  • Total test: Knowledge requirements, level of thought, cognitive performance levels

What To Test

  • Important learning objectives and course goals. Your exams should test learning objectives and instructional goals that you have emphasized to students throughout the course.If your course goal is to analyze a marketing situation and recommend alternative solutions, don’t ask students to perform simple memorization, plug-and-chug problems. You will give a mixed message and students will learn to just memorize, not to analyze.
  • Major topics of discussion. Test topics that were discussed in detail during class.You generally devote class time to discussion of material only if you consider it important to your course goals.
  • Assigned out-of-class readings or projects. Testing out-of-class readings and projects that are important but have not been thoroughly covered in class emphasizes their importance to students.

How To Test

  • Test Frequently. Research has shown that frequent testing with a quick turn-around on feedback enhances student learning and allows students to learn from their mistakes.
  • Test Cumulatively. Frequent (shorter) tests and quizzes give students good feedback, but students also need to be given tests in larger, more cumulative chunks. Students who have to study for cumulative tests remember the material longer and are better able to integrate the new knowledge with old.
  • Write explicit directions. Go over the directions with students prior to starting the test.
  • Clearly state the questions. Remove as many sources of ambiguity as you can and be careful not to introduce new or difficult terminology into your test questions.Sometimes what is clear to you may not be clear to your students, so you might ask a colleague to read your questions. During the test, if a question statement seems confusing to some students, clarify the question for the whole class.
  • Take the test yourself. In addition to checking test construction, this technique helps you to generate your answer key. This is an effective way to check for typos or problem questions. Also time yourself and check for proper length of test. A general guideline is that for a 50 minute test, you should need no longer than 20 minutes.

Types of Test Questions

  • Multiple choice. It is difficult to construct good questions, but they are quicker to grade. Appropriate for measuring both recall of information and discipline vocabulary.
  • Short answer. These questions are easy to grade, but questions need to be clearly worded. It may be necessary to accept alternatively worded answers. This form of question evaluates information recall and analytic skills.
  • Essay. Such tests are hardest to grade, but have the best educational value. These questions challenge students to organize, evaluate, and integrate ideas, as well as demonstrate logical thinking and writing skills.
  • Numerical or logical problems. Grading difficulty depends on the complexity of the problem. Problems test understanding of the material and the ability to apply it.
  • Completion questions. It may be necessary to accept reasonable alternative answers. Completion questions test for recognition of key terms and concepts.
  • Matching. Give enough answers so students can’t guess by elimination. This form of test evaluates recognition of relationships between pairs of words or between words and definitions.

Total Test

Classify questions according to knowledge requirements. You may find that some questions involve only rote memorization and that with a small change, you can produce a question which requires a higher level of thought.

Knowledge Requirement Level of Thought
Lower Memory Recall or recognition of
information
Comprehension Deciphering symbols or technical language
Application Discovery of relationships among facts, generalizations, definitions, values and skills
Analysis Breaking down a problem with
conscious use of defined forms of thinking
Synthesis Solving problems by creative
thinking. Integrating concepts into new ideas
Higher Evaluation Judging according to rational standards

This system, developed by Benjamin Bloom, is one of the most widely used and is based on the notion of a hierarchy of thought processes. Each category requires more complex thinking than the one preceding it, and also builds on or incorporates the preceding types of thought in order to proceed to the “higher” levels. This in itself suggests a teaching strategy: In the early stages of a topic, there should be more emphasis on lower, more basic thinking processes; as this is mastered, students will be able to move up the hierarchy toward more complex ways of dealing with the material.

Descriptions of the Major Categories in the Cognitive Domain

  1. Memorization is defined as the remembering of previously learned material. This may involve the recall of a wide range of material, from specific facts to complete theories, but all that is required is the recall of the appropriate information. Memorization represents the lowest level of learning outcomes.Sample Performance Statements
    • State names of 4 European rulers during World War I.
    • Describe how to titrate.
    • Define deviance.
  2. Comprehension is defined as the ability to grasp the meaning of material. This may be shown by translating material from one form to another (words to numbers), by interpreting material (explaining or summarizing), and by estimating future trends (predicting consequences or effects). These learning outcomes go one step beyond the simple remembering of material, and represent the lowest level of understanding.Sample Performance Statements
    • Summarize Plato’s views on good and evil.
    • Describe the composition of the nucleus of an isotope from its position in the periodic table.
    • Restate the concept of the superego in your own words.
  3. Application refers to the ability to use learned material in new and concrete situations. This may include the application of rules, methods, concepts, principles, laws, and theories. Learning outcomes in this area require a higher level of understanding than those under comprehension.Sample Performance Statements
    • Calculate the volume of a sample of gas under any conditions.
    • Write a coherent paragraph, as given by the basic rules, to express an idea.
    • On the basis of a description, classify a society as embodying organic or mechanical solidarity.
  4. Analysis refers to the ability to break down material into component parts so that the organizational structure may be understood. This may include the identification of the parts, analysis of the relationships between parts, and/or recognition of the organizational principles involved. Learning outcomes here represent a higher intellectual level than comprehension and application because they require an understanding of both the content and the structural form of the material.Sample Performance Statements
    • Identify the ions of an unknown using a chemical qualitative analysis scheme.
    • Identify the key themes in a short story and show how they are interrelated in producing the outcome.
    • Identify several central values in a non-western cultural system and show how they are expressed in economic and religious activities.
  5. Synthesis refers to the ability to put parts together to form a new whole. This may involve the production of a unique communication (theme or speech), a plan of operations (research proposal), or a set of abstract relations (scheme for classifying information). Learning outcomes in this area stress creative behaviors, with major emphasis on the formulation of new patterns or structures.Sample Performance Statements
    • Devise a scheme for the synthesis of an organic compound from a given starting material.
    • Working from original documents, construct an account of how European intervention affected the Civil War.
    • Design an experiment for establishing in what ways certain members of an ecological system are dependent on one another.
  6. Evaluation is concerned with the critical ability to judge the value of material (a statement, novel, poem, research report) for a given purpose. The judgments are to be based on definite criteria. These may be based on internal criteria (organization) or external criteria (relevance to the purpose) and the student may determine the criteria or be given them. Learning outcomes in this area are highest in the cognitive hierarchy because they contain elements of all the other categories, plus conscious value judgments based on clearly defined criteria.Sample Performance Statements
    • Choose the most efficient scheme of synthesis for an organic compound from among several schemes.
    • Evaluate the quality of several poems on the basis of internal structure and harmony of imagery.
    • Compare the adequacy of 3 theories of personality in explaining a specified type of behavior.

Proctoring

Proctoring Duties & Tips (prepared by Amy Cass, Kelly McHugh & Justin Alms 8/15/2006)
A proctor is an individual who is responsible for monitoring students while taking exams and ensuring proper exam protocol and etiquette.

Before Agreeing to proctor an exam:

  • Know how long the test will last. Make sure you are available to proctor the entire exam.
  • Ask the course instructor what is expected of you and what policies and protocol you will have to enforce.

Before the exam is distributed:

  • Some courses might ask you to identify each student by photo ID.
  • Make students aware of the test policies and protocol. Verbally restating these policies at the beginning of the exam will help avoid misunderstandings.
  • If closed book exam, verify that PDA computers, reference books, notes, backpacks, purses, hats, etc are not opened or used in the test room.
  • Affirm that the test environment is a quiet area with adequate space and comfort for taking the exam.
  • Try to space student out in effort to reduce temptation to cheat and to give yourself enough room to move around the class.
  • Check with the course instructor to make sure testing accommodations have been made for students with documented disabilities.

During the exam:

  • Try not to have the students leave the room except for emergencies.
  • Do not simply read and sit at the front of the classroom. Walk up and down the aisles and scan the room to discourage cheating.
  • If you suspect a student is cheating, first watch them closely and take notes. Then hover near them a minute. This discreet gesture may warn the student and discourage this behavior. Try not to publicly embarrass the student.
  • Attempt to answer student questions in a minimally disruptive way. In a small or cramped room, having students come to the front of the room to ask the proctor questions may be best.
  • Keep students aware of how much time they have left by either keeping a ‘countdown’ on the board or giving verbal warning such as “the exam is about halfway over” or “there are 10 minutes left.”
  • For large enrollment classes, when the allotted time for the exam has ended, tell students to finish writing their last sentence and pass the exam to one side of the room for collection.

Grading

Getting back a test is really stressful — especially when the average is 30 and I got a 20.
— Junior Business Major

 

Be fair and reasonable and maintain grading standards which you can defend if challenged. In your syllabus, be very clear about your grading policies and criteria. It is also helpful to specify the types of tests, quizzes, and assignments along with their point distributions. You may also want to include dates of tests and due dates of projects and papers. The more details you give about your grading criteria, the fewer hassles you will receive from students throughout the semester.

General Grading Tips

  • Construct answer key
  • Assign points in advance
Specific Grading Tips

  • Papers — Good feedback enhances learning
  • Essay exams — One question at a time
  • Problem sets, short answer, multiple choice
  • Analyze answers to determine question clarity

General Grading Tips

  • Construct the answer key prior to giving the test.

The best way of doing this is by taking the test yourself (as stated in previous section).

  • Assign points and partial credit based on your key prior to looking at student tests. Include the point distribution on a test so that students can budget their time accordingly.

Specific Grading Tips

Papers

  • Make written comments. There is nothing more arbitrary to a student than a paper passed back with just a grade on it and either no comments or just perfunctory ones. When grading papers, write comments judiciously and legibly. Do not obliterate the text: use the back or append a note. Try to say enough so that the student has a reasonably good chance of doing better next time. If you find that you are saying similar things to several students, prepare a handout on whatever the students are stumbling over; for example, how to write a review, or how to develop an argument.
  • Judge on content, organization, and presentation. Often it is useful to the student if you evaluate the paper in each of these areas and assign a mark on the basis of some combination of these factors. Also, some teachers find it helpful to ask students to write papers twice. The first draft is submitted and subjected to constructive criticism on both content and style. The second draft is graded and usually shows some kind of improvement that is quite satisfying to student and teacher alike.
  • Work to maintain objectivity. Both papers and essay exams involve a lot of subjective judgment. The following suggestions may help with the problem of maintaining consistency. You are more likely to be stringent with the first few papers you read than with the rest, and less likely to be careful about comments and such when you are tired. To avoid such problems, read a few papers before you actually start grading to get an idea of the range of quality, and stop grading when you get tired or start to undergo personality changes due to boredom. When you start again, read over the last couple of papers you graded to make sure you were fair.

Essay Exams

  • Plan ways to divide grading with other TAs. Usually the problem here is how to wade through all those booklets while remaining both consistent and sane. When there are a number of TAs assigned to a course, the course supervisor can divide the workload. If each TA has had a section and all of you have covered the same basic material, then you may prefer to mark the exams of the students in your own section. The problem here, of course, is that objectivity may be hard to achieve since you may feel close to, or even partial toward, your own students. Grading question-by-question rather than student-by-student may help. This will allow you to give credit for material that you presented in section and it will give you feedback on whether the ideas you have emphasized have actually registered. At the same time, you should be guided by a grading standard that has been mutually agreed upon by all TAs and the supervising faculty.

If each TA has dealt with specialized topics in lecture and section, then it is probably better to split the exam questions up so that each TA grades questions about what he or she taught. Dividing the exam questions this way ensures that each question will be marked consistently across the class. However, reading 200 answers to the same question one after the other has its drawbacks: it can affect your mental health and your grading range. This is less likely if you pace yourself, mark questions that you are interested in, and switch questions every once in a while.

  • Get together to resolve difficulties. When the exams have been marked, get together with the other TAs to discuss and resolve any problems you have encountered. Then add up the total scores, check your addition (this saves a lot of trouble later), and plot the distribution. Now you are ready to present the results to the supervising professor or to assign the grades yourself.

Problem Sets, Short Answer Questions, and Multiple Choice

  • Divide Exam Grading.Although these tests usually take longer to make up than the others, they are easier to grade. But problems can still arise. For the same reasons as those mentioned above, it is often a good idea to divide the exam questions among the TAs. Consistency is more likely and deviations easier to spot.
  • Be prepared for alternative answers. You may think that you have written the perfect question with only one correct answer, but you must always be prepared for alternative answers. In the case of multiple choice questions, for example, if the students are doing worse than chance on a particular question, it is likely that the question was poorly worded. In this case you must either give credit for more than one answer or toss the question out (for example, by giving everyone credit).

Grading can be a constructive process both for you and your students. It can give them the opportunity to improve their knowledge and writing skills, and it can give you feedback on your teaching and evaluation methods. By being consistent and fair, you can minimize the inevitably unpleasant aspects of passing judgment on someone’s efforts.

Print Friendly

Comments are closed