Assessment and Outcomes
Teaching - Learning Fellows, Summer 2003 Learning Outcome Project
Greetings and Salutations! My name is Mike Mana. I am a physiological psychologist and an Assistant Professor in the Psychology Department at Western. I received my Ph.D. in Psychology from the University of British Columbia in 1990. I then spent 5 years as a Research Fellow in the Neuroscience Department at the University of Pittsburgh, and 4 years as an Assistant Professor in the Biology Department at Chatham College in Pittsburgh, before coming to Western in 1999.
My interest in the process and product of pedagogy dates back to my earliest TA experiences at University of British Columbia; I have always loved the give-and-take atmosphere of a college classroom and the “buzz” that comes from sharing (as opposed to simply talking about) my interest in brain and behavior. My teaching philosophy emphasizes the notion that students learn best when their participation in the classroom is encouraged and expected…in the selection of topics included in the course syllabus; in the direction that a class discussion takes; in electronic chat rooms devoted to topics pertinent to the class; and in the application of their growing knowledge to issues of personal and/or public importance and interest.
My interest in the TOLO project is based, in part, on a desire to be able to better gauge the success of my teaching by careful assessment of what my students learn. It is always interesting . . . and humbling . . . to play “This is what I said, but this is what you remember?” with students, especially good ones. In this regard, I am very grateful for the opportunity to interact with two outstanding undergraduate students on the TOLO project, Kyle Nelson and Meghan Manaois.
My name is Kyle Nelson and I am currently am a junior at Western Washington University. I am originally from Tumwater, Washington where I graduated from Black Hills High School. I plan to graduate in two years with a Psychology Major and a Biology Minor; my long-term goal is to continue with graduate training in psychology with the eventual goal of opening a camp for teenagers with low self-confidence. I have taken an Introductory Physiological Psychology course from Dr. Mana, and currently work in his research lab. I became interested in the TOLO project because I was interested in the opportunity to impact the nature of the courses offered in the Biopsychology area. I feel that the lower-level classes level need to help students better prepare for more upper-level courses, while upper-level classes need to better prepare students for real life applications of their knowledge and skills. Through our work with the TOLO process this summer, I believe that we have moved closer to these goals, making the courses more interesting and also more applicable.
My name is Meghan Manaois and I am a recent graduate of Western Washington University. My involvement in the TOLO program stemmed from an interest in furthering the connection between student and professor; further, the opportunity to refine the syllabus for a class that was so influential in determining my future career plans has been quite a privilege for me. Having worked on the TOLO project, I believe that future students in Psychology 320 will be better informed about the area of physiological psychology in general, and better prepared if they choose a career in this area. The program, in general, seemed helpful in structuring a class to fit everyone's needs.
I am also very happy to have interacted with the other members of the TOLO group this summer. From the outset, I was interested in developing, adapting, adopting, borrowing or outright stealing assessment approaches that would test the gamut of student knowledge (rote detail to broad themes and connections), be interesting and challenging to the student, applicable to their future academic and/or professional endeavors…and perhaps equally important in a time of shrinking budgets and increasing class size, entail little additional faculty effort. Working with the other members of the TOLO group provided ample stimulation in all of these areas…thanks to one and all!
The course that we have chosen to highlight on the TOLO website is Psychology 320: Topics in Physiological Psychology (pdf file opens in new window). Designed for students with more than a casual interest in the area of brain and behavior, Psychology 320 is the second course in a sequence of courses that begins with Psychology 220 (Introduction to Physiological Psychology) and ends with a number of 400-level seminar courses on specific issues in the area of physiological psychology. Psychology 320 provides a focused and detailed understanding of brain/behavior issues that were introduced in Psychology 220. Its main goals are to provide students with a more detailed understanding of the biological bases of behavior; to familiarize students with the different research questions asked, and approaches used, by physiological psychologists; and to develop skills required to effectively evaluate, and communicate about, research in this area.
To meet these objectives, the course has traditionally included 2 multiple-format exams (multiple-multiple choice; modified true-false; fill-in-the-blank; identification; and/or short answer/essay); a series of “target article critiques” in which students learn to critically read primary research literature, and an end-of-term poster session in which they present an original research paper to the other members of the class. In our evaluation of the course and its assessment tools, my student colleagues and I decided to focus on the teaching objectives and learning outcomes associated with the poster presentation.
Psychology 320 focuses on various topics and issues that fall under the guise of physiological psychology. Its main goals are to provide a strong basic background in brain/behavior relationships; to familiarize students with different areas of research in physiological psychology; and to develop the skills required to evaluate, and communicate about, research in this area.
To this end, student in Psychology 320 have traditionally been assigned an end-of-term poster presentation in which they must read and evaluate an original research paper in an area of their choosing, and then design and present a poster describing this research. Students are prepared for their poster presentation in several ways:
- Approximately every 2 weeks, a class is devoted to the “shredding” of an original research paper. In each class, a single paper (distributed 1 week earlier) is read and analyzed in terms of the hypotheses that were tested; the research design, techniques, and analyses employed; its strengths and weaknesses; and the “future research ideas” that were generated. Students must involve themselves in an in-class discussion as well as submit a 2-page critique of the article under consideration.
- In the middle of the term, 1-2 classes are devoted to the nuts and bolts of designing and putting together a scientific poster, using the Poster Guidelines of the Society for Neuroscience.
The inclusion of a poster's preparation and presentation in Psychology 320 is intended to assess students' ability to read and understand research in physiological psychology, to integrate this information into a cogent whole, and to communicate this new understanding to their peers. We believe…and student evaluations support the idea…that the planning, preparation and presentation skills are practical ones for students who plan to go on to a graduate program in the behavioral and brain sciences, as well as students who will leave Western with a B.A. in psychology and join the workforce in some area completely unrelated to psychology.
Student evaluations indicated that the poster sessions were one of the most enjoyed, and valuable, assignments in Psychology 320. However, many students felt they lacked the background necessary to confidently speak in the think-on-your-feet, Q-&-A atmosphere of a poster session. In addition, many students missed the opportunity to develop their writing skills in an upper-level course. After careful consideration, my student colleagues and I decided that the completion of both an end-of-term paper AND an end-of-term poster presentation was cruel and unusual punishment. Instead, we elected to change the course requirements to include a mid-term paper and end-of-term poster, on a single topic. We hope that the novel (at least to us!) integration of a midterm paper with an end-of-term poster in our class assessment plan will provide a vehicle that will facilitate students' abilities to gain both breadth and depth of knowledge in a favored area of physiological psychology, while developing technical writing and presentation skills that will be useful to future academic and nonacademic endeavors.
In large part, the knowledge base and skills outlined in the learning objectives for Psychology 320 are slowly acquired during each class period over the entire quarter. With that said, my student colleagues and I have selected several key classes and assessment tools included in the syllabus that are particularly relevant to the intended teaching objectives and learning outcomes required for the paper-and-poster assignment that we have focused on for the TOLO website.
Critical Thinking/Public Speaking Skills. My student colleagues and I believe that the first formal (i.e., not happening every class) teaching objective/learning outcome to be assessed that is relevant to the end-of-quarter poster presentation will occur during the first “article critique”. In this class, we single out an original research paper and explore its strengths and weaknesses. Most students in Psychology 320 have just begin to appreciate that much of what is reported in science…especially the behavioral and brain sciences…is not dualistic (aka the Perry Scheme's notion of right v. wrong) but instead is multiplistic (there are several possible answers to a question) and/or relativistic (the answer depends on the conditions). They are intimidated by the challenge of CRITICALLY reading the research of a “published scientist” and by the notion that they should question EVERY important claim made in that published article.
To get students past this hump early…to whet their appetites for the critical dialogue that should increasingly become a part of their academic life…I will schedule the first “paper shred” as early as possible in the quarter (i.e., within the first 2 weeks of class). In this way, students will begin to speak publicly about their own critical thinking as early as possible in the course. In addition, they will begin to appreciate the vagaries of scientific research and the advantages and pitfalls inherent to different approaches to physiological psychology research. Some students will leap at the chance to engage in this dialogue; other students will have to be “bird-seeded” with leading questions that encourage their participation. Regardless, each student will be expected to contribute to this 90-min discussion in some way.
Following this class, each student will receive feedback (written comments and a grade) about their contributions to the class discussion and about their 2-page critique of the research article. The first article critique will be worth ½ of subsequent critiques, so that students will not feel penalized in their first attempts to engage in this new academic skill.
Critical Thinking/Novel Synthesis of Existing Knowledge/Technical Writing Skills. The second phase of our teaching objectives/learning outcomes to be assessed will begin at the start of the 3rd week of class, when students will use Blackboard to submit a 1-page outline of their midterm paper. This outline will describe the general area they have selected to write about and lay out the key sections of their paper; references for the articles that each student has gathered to date will also be included. It is our hope that this phase of the process will allow students to accumulate and integrate the background knowledge in their topic area that will allow them to more comfortably handle the Q-&-A Environment of the end-of-term poster session.
These outlines will reviewed for general organization, clarity of thought, breadth v. depth of topic, and quality of references; comments will be promptly returned (within 1 week) and students will be given until the end of the 5th week of classes (about 10 days) to complete and submit their mid-term paper. If they choose to, students will be able to submit a draft of their paper for further comments and revision.
Critical Thinking/Focused Description/Technical Presentation Skills. The third phase of our teaching objectives/learning outcomes will prepare students specifically for their poster presentations. During this class, which will occur during the 6th week of the quarter, I will talk to students about the differences between a poster presentation and the term papers that they have just completed, using the guidelines of the Society for Neuroscience as a starting point. The need to focus on a single piece of original research, brevity of text, importance of graphical data presentation, text size, poster layout, and other topics will be discussed using (mostly) model posters from previous classes to highlight key points. I will use Blackboard to provide students with links to sites that provide further instruction on the poster-making process and to provide a forum for students to seek/offer help on their poster presentation.
Following these classes, students will use PowerPoint, Word or a similar program to prepare a mock-up of their poster, including text and key figures, for submission to me via Blackboard by the end of the 7th week of class. Comments will be returned during the 8th week, with comments focused on the key points outlined above.
Think on Your Feet/Poster Preparation/Public Speaking and Presentation Skills. The fourth and final phase of teaching objective/learning outcome assessment comes during the final week of class when each student is responsible for a 5-10 min presentation of their poster. The forum mimics a real scientific meeting…during the in-class presentations, posters line the walls of the classroom, coffee is available so that brains are properly stimulated, and each poster presenter is allowed to “tell their story”, with the caveat being that they can be subjected to a “Q-&-A period” where classmates ask questions about the research under consideration at any point in time.
Assessment of the midterm paper and the end-of-term poster session will be done separately, so that two grades will be assigned. Midterm papers will be graded according to a rubric provided to students at the beginning of the quarter and available at the Blackboard site for the class. Each poster presenter will be graded on the quality of their poster according to a rubric that will be distributed during the class focusing on poster preparation and presentation and available at the Blackboard site for the class. In addition, each poster will be anonymously evaluated by other students in the class, using a rubric distributed to each student at the start of each poster session. Students who are not presenting will be graded on the quality of their participation in the Q-&-A sessions (thoughtfulness of question; clarity of question). At the end of the poster session, each student will receive a single grade based upon the design and content of their poster (40%), their presentation of the poster (30% my grade + 10% summary of peer evaluations), and their interactions during other poster presentations during the Q&A session (20%).
At the end of the quarter, students will use Blackboard to complete a Student Assessment of Learning Outcomes (SALG) which will provide feedback on their perception of the success of the combined term paper/poster presentation.