Thursday, January 11, 2007

CATs for All Occasions

(Note: CATs that require more time and/or energy for 1) faculty to prepare, 2) students to respond to the assessment, or 3) faculty to analyze the data collected are in red italics.)

Techniques for Assessing Course-Related Knowledge and Skills Assessing Prior Knowledge, Recall, and Understanding

Minute Paper - Instructor asks students to respond in two or three minutes to either of the following questions: "What was the most important thing you learned during this class?" or "What important questions remains unanswered?" Used to provide a quick and extremely simple way to collect written feedback on student learning.

Muddiest Point - Technique consists of asking students to jot down a quick response to one question: "What was the muddiest point in (fill in the blank)?" with the focus on the lecture, a discussion, a homework assignment, a play, or a film. Used to provide information on what students find least clear or most confusing about a particular lesson or topic.

Focused Listing - Focuses students' attention on a single important term, name, or concept from a particular lesson or class session and directs them to list several ideas that are closely related to that "focus point." Used to determine what learners recall as the most important points related to a particular topic.

Empty Outlines - The instructor provides students with an empty or partially completed outline of an in-class presentation or homework assignment and gives them a limited amount of time to fill in the blank spaces. Used to help faculty find out how well students have "caught" the important points of a lecture, reading, or audiovisual presentation.

Background Knowledge Probe - Short, simple questionnaires prepared by instructors for use at the beginning of a course, at the start of a new unit or lesson, or prior to introducing an important new topic. Used to help teachers determine the most effective starting point for a given lesson and the most appropriate level at which to begin new instruction.

Misconception/Preconception Check - Technique used for gathering information on prior knowledge or beliefs that may hinder or block further learning.

Memory Matrix - A simple two-dimensional diagram, a rectangle divided into rows and columns used to organize information and illustrate relationships. Assesses students' recall of important course content and their skill at quickly organizing that information into categories provided by the instructor.

Assessing Skill in Syntheses and Critical Thinking

One-Sentence Summary - Students answer the questions "Who does what to whom, when, where, how, and why?" about a given topic, and then synthesize those answers into a single informative, grammatical, and long summary sentence.

Approximate Analogies - Students complete the second half of an analogy for which the instructor has supplied the first half. This allows teachers to find out whether their students understand the relationship between the two concepts or terms given as the first part of the analogy.

Word Journal - Students first summarize a short text in a single word, and second, the student writes a paragraph or two explaining why he chose that particular word to summarize the text. This technique helps faculty assess and improve the students' ability to read carefully and deeply and the students' skill at explaining and defending, in just a few more words, their choice for a single summary word.

Concept Maps - Drawings or diagrams showing the mental connections that students make between a major concept the instructor focuses on and other concepts they have learned. This provides an observable and assessable record of the students' conceptual schema-the patterns of associations they make in relation to a given focal concept.

Invented Dialogues - Students synthesize their knowledge of issues, personalities, and historical periods into the form of a carefully structured, illustrative conversation. This provides information on students' ability to capture the essence of other people's personalities and styles of expression - as well as on their understanding of theories, controversies, and the opinions of others.

Annotated Portfolios - Contain a very limited number of selected examples of a student's creative work, supplemented by the student's own commentary on the significance of those examples.

Assessing Skill in Problem Solving

Problem Recognition Tasks - Students are provided with a few examples of common problem types and are asked to recognize and identify the particular type of problem each example represents. Faculty are able to assess how well students can recognize various problem types, the first step in matching problem type to solution method.

What's the Principle? - Students are provided with a few problems and are asked to state the principle that best applies to each problem. Instructors assess students' ability to associate specific problems with the general principles used to solve them.

Documented Problem Solutions - Prompts students to keep track of the steps they take in solving a problem. This assesses how students solve problems and how well students understand and can describe their problem-solving methods.

Audio- and Videotaped Protocols - Students are recorded talking and working through the process of solving a problem. Faculty assess in detail how and how well students solve problems.

Assessing Skill in Analysis and Critical Thinking

Categorizing Grid - Students sort information into appropriate conceptual categories. This provides faculty with feedback to determine quickly whether, how, and how well students understand "what goes with what."

Pro and Con Grid - Students list pros and cons of an issue. This provides information on the depth and breadth of a student's ability to analyze and on their capacity for objectivity.
Defining Features Matrix - Students are required to categorize concepts according to the presence (+) or absence (-) of important defining features. This provides data on their analytic reading and thinking skills.

Content, Form, and Function Outlines - Students analyze the "what" (content), "how" (form), and "why" (function) of a particular message. This technique elicits information on the students' skills at separating and analyzing the informational content, the form, and the communicative function of a lesson or message.

Analytic Memos - Students write a one- or two-page analysis of a specific problem or issue. Used to assess students' skill at communicating their analyses in a clear and concise manner.

Assessing Skill in Application and Performance

Applications Cards - Students write down at least one possible, real-world application for an important principle, generalization, theory, or procedure that they just learned. This lets faculty know how well students understand the possible applications of what students have learned.

Directed Paraphrasing - Students paraphrase part of a lesson for a specific audience and purpose, using their own words. Feedback is provided on students' ability to summarize and restate important information or concepts in their own words; it allows faculty to assess how well students have understood and internalized that learning.

Student-Generated Test Questions - Students are asked to develop test questions from material they have been taught. Teachers see what their students consider the most important or memorable content, what they understand as fair and useful test questions, and how well they can answer the questions they have posed.

Human Tableau or Class Modeling - Groups of students create "living" scenes or model processes to show what they know. Students demonstrate their ability to apply what they know by performing it.

Paper or Project Prospectus - A prospectus is a brief, structured first-draft plan for a term paper or term project. The Paper Prospectus prompts students to thin through elements of the assignment, such as the topic, purpose, intended audience, major questions to be answered, basic organization, and time and resources required. The Project Prospectus focuses on tasks to be accomplished, skills to be improved, and products to be developed.

Techniques for Assessing Learner Attitudes, Values, and Self-Awareness Assessing Students' Awareness of Their Attitudes and Values

Classroom Opinion Polls - Students are asked to raise their hands to indicate agreement or disagreement with a particular statement. Faculty discover student opinions about course-related issues.

Course-Related Self-Confidence Surveys - Students answer a few simple questions aimed at getting a rough measure of the students' self-confidence in relation to a specific skill or ability. Faculty assess their students' level of confidence in their ability to learn the relevant skills and material and can more effectively structure assignments that will build confidence in relation to specific tasks.

Double-Entry Journals - Students begin by noting the ideas, assertions, and arguments in their assigned course readings they find most meaningful and/or controversial. The second entry explains the personal significance of the passage selected and responds to that passage. Detailed feedback is provided on how students read, analyze, and respond to assigned texts.

Profiles of Admirable Individuals - Students are required to write a brief, focused profile of an individual - in a field related to the course - whose values, skills, or actions they greatly admire. This technique helps faculty understand the images and values students associate with the best practice and practitioners in the discipline under study.

Everyday Ethical Dilemmas - Students are presented with an abbreviated case study that poses an ethical problem related to the discipline or profession they are studying and must respond briefly and anonymously to these cases. Students identify, clarify, and connect their values by responding to course-related issues and problems that they are likely to encounter. Faculty get honest reactions and information on what students' values are and how they apply them to realistic dilemmas.

Assessing Students' Self-Awareness as Learners

Interest/Knowledge/Skills Checklist - Students rate their interest in various topics, and assess their levels of skill or knowledge in those topics, by indicating the appropriate responses on a checklist which has been created by the teacher. These checklists inform teachers of their students' level of interest in course topics and their assessment of the skills and knowledge needed for and/or developed through the course.

Goal Ranking and Matching - Students list a few learning goals they hope to achieve through the course and rank the relative importance of those goals.. This assesses the "degree of fit" between the students' personal learning goals and teachers' course-specific instructional goals, and between the teachers' and students' ranking of the relative importance and difficulty of the goals.

Focused Autobiographical Sketches - Students are directed to write a one- or two- page autobiographical sketch focused on a single successful learning experience in their past - an experience relevant to learning in the particular course in which the assessment technique is used. This provides information the students' self-concept and self- awareness as learners within a specific field.

Self-Assessment of Ways of Learning - Students describe their general approaches to learning, or their learning styles, by comparing themselves with several different profiles and choosing those that, in their opinion, most closely resemble them. This provides teachers with a simple way to assess students' learning styles or preferences for ways of learning.

Techniques for Assessing Learner Reactions for Instruction Assessing Learner Reactions to Teachers and Teaching

Chain Notes - Students write immediate, spontaneous reactions to questions given by the teacher while the class is in progress. This feedback gives the teacher a "sounding" of the students' level of engagement and involvement during lecture.

Electronic Mail Feedback - Students respond anonymously by E-mail to a question posed by the teacher to the class. This provides a simple, immediate channel through which faculty can pose questions about the class and students can respond to them.

Teacher-Designed Feedback Forms - Students answer questions on feedback forms which contain anywhere from three to seven questions in multiple-choice, Likert-scale, or short fill-in answer formats. These forms allow faculty to quickly and easily analyze data and use the results to make informed and timely adjustments in their teaching.

Assessing Course-Related Learning and Study Skills, Strategies, and Behaviors

Punctuated Lectures - Students and teachers go through five steps: listen, stop, reflect, write, and give feedback. Students listen to lecture. The teacher stops the action and students reflect on what they were doing during the presentation and how their behavior while listening may have helped or hindered their understanding of that information. They then write down any insights they have gained and they give feedback to the teacher in the form of short, anonymous notes. This technique provides immediate, on-the-spot feedback on how students are learning from a lecture or demonstration and lets teachers and students know what may be distracting. And students are encouraged to become self-monitoring listeners, and in the process, more aware and more effective learners.

Productive Study-Time Logs - Students keep a record of how much time they spend studying for a particular class, when they study, and how productively they study at various times of the day or night. This allows faculty to assess the amount and quality of out-of-class time all their students are spending preparing for class, and to share that information with students.

Process Analysis - Students keep records of the actual steps they take in carrying out a representative assignment and comment on the conclusions they draw about their approaches to that assignment. This technique gives students and teachers explicit, detailed information on the ways in which students carry out assignments and shows faculty which elements of the process are most difficult for students and, consequently, where teachers need to offer more instruction and direction.

Diagnostic Learning Logs - Students keep records of each class or assignment and write one list of the main points covered that they understood and a second list of points that were unclear. Faculty are provided with information and insight into their students' awareness of and skill at identifying their own strengths and weaknesses as learners.

Original Source: Classroom Assessment Techniques, 2nd edition (1993) – Angelo and Cross