In August of 2014, after five years of study and drafting, the ABA added a new set of assessment requirements to the accreditation standards. Starting with the incoming class in the fall of 2016, ABA accredited law schools are required to “establish and publish” learning outcomes (Standard 301) addressed to a general description of the purpose and focus of legal education (Standard 302), and use both “summative and formative” assessment methods to “measure and improve” student learning and provide meaningful feedback to students (Standard 314).
Although fairly common in undergraduate education, the term “formative assessment” is fairly new to most of the legal academy. Clinical and legal writing faculties have employed these assessment methods for years, but many other faculty members are new to it.
I am a member of the Board of CALI, the Center for Computer Aided Legal Instruction. In that capacity I have recently been involved in discussions around what law schools might do to meet the new ABA requirements pertaining to assessment.
One of the topics that have come up in those discussions is whether it is possible for multiple-choice tests to be formative assessment. If it were, then the addition of formative assessment to the standards would be a fairly easy thing to add to the curriculum. To answer that question, we have to first understand what formative assessment is.
The first thing to note is that it is something qualitatively different from summative assessment. The typical summative assessment is the final exam at the end of a law school course. These are particularly common to the first year courses, but are not exclusive to them, and often appear in “casebook” courses in the second and third years of law school. So if formative assessment is something different, what is it? A recent article by Joni Larson distinguishes summative and formative assessment as follows:
“Formative assessments are designed primarily to improve learning…. [while] Summative assessments are designed primarily to judge learning.” (Emphasis added).
An essential feature of formative assessment is individualized feedback provided by the teacher beyond merely a test score that measures how much the student fell short of an arbitrary standard. The feedback provided should detail what good performance looks like and explain why it is better than what the student did. Such feedback provides tools to the student that help them self-assess going forward, and has been found to foster higher levels of engagement and motivation in learning the material.
Formative assessment should be designed to monitor student learning during the course, but also to allow the teacher to monitor and modify his or her teaching practices during the pendency of course. So there is an element of formation for the teacher as well as the student.
These definitions may be somewhat helpful, but perhaps a couple of examples might help to put the definitions in the context of law school teaching. I have a colleague who teaches an upper-level large course (60ish students) and he gives a midterm that is mostly short and long answer essay format. There is a great deal of feedback given on each exam, together with a grade, which amounts to 25% of the grade for the course. Then, the professor invites each student to appeal their grade, a challenge that many accept. The appeal must be in a prescribed written format, the professor reviews each appeal, and then meets with each student to give them the result of their appeal and explain it. I believe this is the best of what we mean (in legal education) by formative assessment, and I think it is the sort of assessment the ABA is hoping to see when they visit schools and examine how well the school is meeting the requirements of Standard 314. (Not in all courses, and not all the time... but that's a question for another blog post.)
In contrast, a colleague at another school described an exercise in first year contracts in which she has them prepare multiple choice questions (and proposed answers) for an optional test, and submit them to her in advance of the test, which uses versions of those questions (after her edit). This is certainly an interesting exercise, and probably fun for the students. But it is not clear what formative learning the students get from the professor in that exercise (other than from their question appearing on the optional test with or without modification). And it is not clear how the professor uses this exercise for her own learning and adjustment of the course while she is teaching it. So this example does not have either of the two essential hallmarks of being a formative exercise.
Multiple-choice questions can be formative, but they have to accomplish more than merely to judge the level of learning that has been accomplished. So when a multiple-choice test provides the student after the test with detailed explanations for the correct answers (as well as the incorrect answers) it can provide some guidance for improvement of comprehension of course content. But - particularly when testing skills – multiple-choice questions are most effectively formative when there is some individualized feedback provided by the professor to the student, and the professor uses the information gleaned from the assessment to adjust the teaching and learning activities in the course, ideally while the course is still underway.
For additional information on this subject, with more examples of formative assessment in law school, link to Part II of this blog post here.