Award-winning educator questions UVic’s testing techniques

Tweet about this on TwitterShare on FacebookShare on Reddit

While UVic conducted surveys to determine why people choose UVic, one parent and educator has questioned UVic’s use of multiple-choice testing. UVic’s webpage states, “Instructors at UVic have become recognized innovators in developing and identifying teaching practices which best support student engagement and success.” It is a statement that Michael Maser, educator and father of a UVic undergrad, may not always agree with. The question, “Why UVic?,” asked through the survey created by The UVic Difference project, has a different tone for Maser, now more than ever.

“It’s no longer appropriate to serve administrative needs over and above the learning interests of students,” said Maser. He has been questioning the university’s testing methods, specifically in the faculty of sciences, for the past year. After getting in touch with the dean of sciences, Dr. Robert Lipson, and receiving what he felt was a “patronizing response,” Maser has tried contacting UVic’s president, Jamie Cassels, and aims to get UVic’s process of student evaluation changed.

Maser’s daughter, Robin, was enrolled in UVic’s theatre program during her first year, but tried out a few science courses at Camosun. “It was a smaller class and the teacher got to know their names,” said Maser. As welcoming as Camosun was, she looked through the UVic course calendar and Maser says it “was enticing to her.  Ecology courses . . . she wasn’t going to get that at Camosun.”

“I didn’t know much about the science faculty,” said Maser. “The website said ‘come for the best quality education that you can get’ sort of thing, not that any of them don’t say that.” UVic’s reputation for its education may be one of the university’s many draws for new students. The UVic Difference project asked students, staff, faculty, and alumni why they came and what they think of UVic  by online survey between Feb. 3 and 18. The questions ranged from, “How satisfied are you with the academic experience?” to, “How important is it for UVic to be a research-inspired learning environment?”

The last time a study like this was conducted was 10 years ago. “UVic has a great reputation generally,” said UVic Difference Project Manager Bruce Kilpatrick, by email, “but we’d like to understand more about how the university’s perceived with various audiences in order to sharpen it up.”

Maser, who is an educator himself, has some thoughts of his own on how UVic could sharpen up. He has been given the Prime Minister’s Award for teaching excellence, as well as an award of merit from the B.C. Ministry of Education. He is dissatisfied with multiple-choice testing that his daughter encountered at UVic. “I think the university needs to offer more personalized testing,” said Maser. Some of The UVic Difference survey questions pertained to personalized experiences. “I think they need to break down class sizes into smaller cells . . . They need to do whatever is necessary to make this a meaningful experience for learners . . . I don’t have all the answers, but they need to at least consider them.” Maser added, “They need to shove multiple choice questions off the table.”

The dean of the Faculty of Science, Lipson, said how he felt about multiple-choice questions over e-mail correspondence. “Multiple choice exams, when developed correctly (and that is every instructor’s goal), provide an effective mechanism to test a wide range of detailed material in a clear, comprehensive manner in a reasonable time frame (one-hour midterm, three-hour final exam).”

UVic has recently invested in software that Dean Lipson said will “allow instructors who use multiple choice testing to determine which questions do a good job of measuring what was intended to be measured; which items are too easy or too hard or don’t discriminate well; and items that are redundant.” The name of the software is Remark Optical Mark Reader. It offers statistics on which questions were answered right and wrong, giving feedback after a test is written, on which questions were fair and which weren’t.

Multiple-choice questions are a more cost effective way of evaluating students. The all-too-familiar Scantron sheet can be graded quicker than a written exam. But Maser said, “It’s indefensible that Dean Lipson told me, last spring, that they can’t afford more personalized experiences.” When asked if some money should be pulled from other areas, like research, Maser said, “Research universities, they confuse their priorities. When those priorities of offering learning services to the young people, to their students, gets muddled up with research priorities—an exclusive focus on research to the detriment of learning experiences for young people—then there is a problem . . . Considering what kids are shelling out for tuition.”

So what is the role of research at a university? The topic was brought up many times during The UVic Difference survey. It asked people how important research was, as well as comparing perceptions of UVic’s quality of research to those of other schools in Canada. “The competition among the 100-plus universities in Canada for students, faculty, staff, donors, partners, and funders, etc., is becoming increasingly fierce,” wrote Kilpatrick.

Dean Lipson writes, “Our faculty members compete at the national level for Federal funding from the Natural Sciences and Engineering Research Council (NSERC), the Social Sciences and Humanities Research Council (SSHRC), and the Canadian Institutes of Health Research (CIHR). They also compete for funds from the Canadian Foundation of Innovation and from industry.”

Research, according to Dean Lipson, “is an absolutely critical thread in the fabric of a university. It is this integration of teaching and research, which I think enhances the student experience and provides unique contexts to the curriculum, and therefore, distinguishes universities from other types of post-secondary institutions.”

Cutting research funding in order to provide more personalized learning experiences may not be a possibility. When asked if research should receive a cut in order to afford more comprehensive testing methods, Dean Lipson said, “The bulk of the research funding is obtained from external sources, which is in addition to the budget allocated for teaching, not at the expense of teaching . . ..” He also wrote, “Our current budget issues are driven entirely by other external factors (such as a cut to UVic’s funding by the B.C. government). Conflating research with testing methods is, in my opinion, not meaningful.”

Maser and Lipson both feel that students should have the ability to review their tests after receiving a grade. Maser mentioned that his daughter, as a UVic Science student, wasn’t able to review a test after receiving her grade; although, he couldn’t recall the class in which this happened. Dean Lipson said that, “In my opinion no student should be denied, and, in my experience and to the best of my knowledge as dean, I am not aware of any student ever being denied the opportunity to check their bubble sheet to check what they did or did not get wrong on a multiple choice exam.” In many cases, class time is given to review the exam, or students are offered the option to review their exams during office hours or with a teaching assistant.

According to the Times Higher Education ranking, UVic is 11th globally and first in Canada for schools under 50 years old. UVic is making a name for itself on an international stage. “We’re not going to rest on our laurels,” said Kilpatrick. The question of “why UVic?” will continue to be asked for many years to come.

Tweet about this on TwitterShare on FacebookShare on Reddit
  • Ryan Ziegler

    I can say unequivocally that “Yes” is the correct answer to at least the first two questions.

    – Currently, you may view exams as far back as summer 2013: http://www.uvic.ca/current-students/home/academics/examinations/viewing.php

    – I haven’t personally visited a professor to review multiple choice exams. The option, however, has been extended by professors in lower and upper level courses. typically following midterms with relatively low averages. Presumably, whether a professor openly offers to review exams of any sort will vary based on the professor. That being said, I’ve never had any professor unable to see me outside of their scheduled office hours to discuss my exam marks. Moreover, TAs are usually available to discuss the material.

    – I have had multiple professors in lower and upper level courses in natural sciences, social sciences, and humanities, acknowledge “flawed” MC, short answer, and essay questions on the basis that they were either worded too vaguely or too specifically, or asked for content that was related to the course but ultimately outside its purview, or asked for too much information given time constraints, or marked too harshly by their marking assistant, etc. In response, these profs have either scaled up the class average to correspond with past/expected averages; dropped questions entirely; treated the questions as bonus questions; etc. Moreover, certain profs spend entire lectures discussing exam marks, taking questions, and clarifying what sort of answers they expected. Obviously a flip-side exists, but I’ve seen it considerably less often.

    – I have no idea whether this is the case. I have had a roughly analogous experience where someone pointed out a logical inconsistency with an MCQ that allowed 2/4 answers to be correct. After some deliberation the professor allowed both answers to stand, and changed the marks to reflect this.

  • michaelmaser

    My Qs for Dean Lipson
    I am the parent profiled in this article and I’d like to correct that my daughter took one course at Camosun, not several.
    Also, my daughter has experienced 3-hour multiple choice exams as the culminating learning-evaluation experiences in several of her science courses, not just one. And when the exams were done, it was on to a break (Christmas or summer) and new courses.
    Otherwise, I am intrigued by Dean Lipson’s response that students have opportunities to review their multiple choice exams and results after the exams (and courses) are complete. The questions I would next like Dean Lipson to answer, then, are the following:
    – to his knowledge, has any student ever done that, or gone to a professor to review a multiple choice exam?
    – does he know if faculty identify flawed questions, or questions that generated the most wrong answers and really seek to determine the cause(s) of this, and clarify this with students?
    – are students able to re-submit their MC exam ‘bubble-sheets’ for revision and grade re-assessment if and when questions are found to be flawed, which is a common characteristic of MC testing?
    I am skeptical that “Yes” is be the correct answer to any of these questions. As well, I would like to know, given what I wrote above about the timing of these MC exams, when, exactly are students supposed to approach their profs or TAs about exams that may have happened weeks or even months, previously.
    Over to you, Dr. Lipson.
    – Michael Maser