I’ve been gone for a while, working on a book manuscript (on sexual consent). But I’m ready to think about something else for a few days, so… here I am.
The University of Chicago Law School has just released its AI Strategy Statement. It is very, very good. In this post, I want to say a few words about it.
The Berkeley Way
First, though, a bit of context. To say that AI has “disrupted” legal education would be a colossal - nay, hilarious - understatement. The fact that law students are able to generate legal memos, papers, exam answers, etc. with a prompt has given rise to all kinds of hand-wringing about how we should go about teaching and assessing them going forward. (I have written about this here.) The main concern, at bottom, is over “cognitive offloading” - students using AI to do the thinking for them. This can be a concern for practicing lawyers too, but it is of far greater concern with students, who have never acquired basic legal skills in the first place. A recent article in Nature, discussing the effect of AI on medical education, has used the term “never-skilling”.
One way to deal with this problem is The Berkeley Way. Berkeley Law’s new policy, widely publicized, essentially creates a default rule prohibiting students from using AI in any way when producing for-credit work. The rule is only a default, meaning that there is leeway for individual instructors to permit AI use for pedagogical reasons. But it is pretty all-encompassing (in spirit, at least).
The Berkeley Way has been subjected to a great deal of criticism. For one thing, it is essentially unenforceable. There is, at present, no software that can reliably determine whether a student has used AI, and many universities forbid the use of AI-detection software by professors. And though some people are exceedingly confident in their ability to sort AI slop from salt-of-the-earth human writing - to the point, indeed, where they are prepared to issue open death threats against those who use AI in their writing - I confess that I am not so confident. One may have suspicions, of course, but they are unlikely to be actionable suspicions. More likely, one would try to use a range of existing AI-detection sites to buttress or dispel one’s private suspicions, and then (if supported by the results) look for problems that are actionable - e.g., plagiarism. But in any case, the enforceability challenges are very real.
Now, I am not necessarily against law schools and professors issuing rules that are unenforceable - I may not be able to stop a law student from doing something, but that does not mean that I should simply stand idly by and quietly let that student do what is (in my pedagogical judgement) contrary to his or her best interests as a student. Sure, that student might roll his or her eyes (behind my back), but so what? I will have done my duty. A sanctionless rule can still be perfectly coherent and appropriate as a rule.
In this context, however, the lack of an enforcement mechanism is a real problem. Many or most law schools have a ‘curve’. (U of Saskatchewan doesn’t, though we do have grading guidelines.) Students are competing for jobs against each other, and the competition starts early - in the first half of 1L, when students are at their most confused and most desperate to keep their heads above water. If students believe that even some of their peers are using AI to get ahead, it will be extremely difficult for them to resist using it themselves. After all, it’s not just bad people who cheat. A person who hates the idea of cheating may nonetheless think that he or she would be a sucker to sacrifice that Bay Street job for the sake of principle. To dissuade students from thinking this way, there needs to be a credible threat of enforcement and sanction.
But unenforceability may not be the biggest problem with The Berkeley Way. Arguably, the bigger problem is that, gosh, maybe we want law students to use AI in some ways. Lawyers are, of course, increasingly using AI in the production of memos, briefs, and the like. Law firms increasingly expect new hires to be familiar with AI. Perhaps it would be a good thing not just to prohibit law students from using AI tout court, but to provide some guidance in how they should use AI. And, hey, there are good reasons to think that law students have much to learn from AI. Consider, for example, this recent study out of Stanford Law School, which suggested that many law professors believe that AI-generated answers to student questions are better than the answers provided by law professors. Or consider this extremely interesting paper by Nicholas Bednar, David Cleveland, Alvin Erbsen, and Daniel Schwarcz, which suggests that students who used AI to synthesize an area of the law in the preparation of a legal memorandum actually had a better grasp of that area of law than those who did not. (Schwarcz has discussed the latter study on the podcast Ipse Dixit here.)
In truth, I don’t read the Berkeley policy as outright forbidding many of these uses of AI. Again, it sets a default rule - and a default rule about the research and production of graded material. Dean Chemerinsky has observed that Berkeley law students are taught to use AI in their 1L course on Legal Research and Writing. So The Berkeley Way has received some undeserved opprobrium. For my purposes, though, I want to stress that the reaction to The Berkeley Way has been driven by the deep-seated intuition - from members of the profession and the legal academy - that an outright prohibition on AI would not just be hopeless but pedagogically and professionally counterproductive.
Lurking in the background, too, is a question that I raised some months back: why have law schools in the first place? If students can learn much about the law from AI - at least, everything that law schools are capable of teaching at scale - then why bother having law schools? Similar questions have been raised by Jordan Furlong and Michael Madison.
The University of Chicago Law School’s AI Strategy Statement: The 1L Curriculum
Enter the University of Chicago Law School’s AI Strategy Statement [‘the Statement’]. Early on, it makes clear that “[w]e need to ensure that our students actually learn to think critically, strategically, and independently without relying on AI; but we also must face the reality that AI tools are already widely available to our students, and our graduates will be expected to be prepared to use them in legal practice.” To that end, the “strategic vision” emphasizes:
1. Developing AI-resilient pedagogy and assessment;
2. Elevating the “essential human” skills that distinguish excellent lawyers; and
3. Teaching the responsible, effective, and ethical use of AI.
The Statement emphasizes the need to ensure that students invest cognitive work in the material. So: “We thus need to ensure that our students do not rely on AI-provided shortcuts that help them produce easy answers but stunt intellectual growth.” And: “This requires developing AI-resilient pedagogy, by which we mean modes of classroom interaction and performance evaluation that reward students’ effortful and sustained engagement with the material, and discourage the offloading of work to AI tools.” These concerns are especially (but not exclusively) live in 1L, where “[s]tudent expertise in judging the quality of AI output is at its nadir.” With these concerns in mind:
Across all 1L sections, we will prohibit the use of electronic devices such as laptops, tablets, and phones in the classroom. …. Additionally, examinations will be in-class without access to the internet, electronic files, or apps. And most of all, we will continue our longstanding tradition of emphasizing the Socratic Method as part of these courses. This coordinated approach reflects our experience, and an emerging scholarly consensus, that active, in-person engagement is conducive to learning. Reliance on devices to take notes or assist with answering questions tends to inhibit reflection and reasoning.
The move to in-class examinations without access to the internet or electronic files is no surprise. I expect that most law schools have already done this by now.
Much more striking is the prohibition on the use of electronic devices in the classroom.1 That, folks, is a seismic change in legal education. And it is what we should all be doing. Indeed, I would say that this change is long overdue. We have known for years that many students are unable to cope with the distractions that ready access to the internet provides - never mind AI. (Yes, Student-in-the-Back-Row, I know that you’re playing poker/browsing Etsy/checking Facebook/playing Wordle. I have seen you, albeit during peer teaching reviews when I sit behind you.) We have also known for years that students benefit from having to take notes by hand - that the need to intellectually work with the material as they hear it, rather than function as glorified stenographers, helps them retain it. The fact that students now have access to AI just gives us a good reason to do something we ought to have done a long time ago.
Over at Above the Law, Joe Patrice has observed that the laptop ban actually doesn’t solve an AI problem at all, since no AI chatbot could possibly react quickly enough to satisfy a professor using Socratic Method. I agree with the second half of that proposition, but not the first. The Chicago Statement does, as I will discuss in a moment, place a heavy emphasis on Socratic Method, but it doesn’t say that that’s all that will happen in the classroom. Students may be asked to think about fact patterns, or asked to consider (as a group) how a given case or doctrine would apply to a particular scenario. (We often do this in my Evidence class.) A student with access to AI could simply outsource the questions to the chatbot. Worse, the student could articulate that answer to the rest of the class, cutting short the deliberation that others might otherwise have done.
I also anticipate mischief. Imagine the student who decides to use AI to pepper the professor with questions. Now, professors are already familiar with the traditional version of this student - the smarty-pants who wants to use the lecture-time as if it was office hours, hijacking the class. But that student is at least engaged in a human-to-human debate/conversation. There are bad/clueless actors, sure, but I can’t really begrudge a genuinely keen, interested student for wanting to pick my brain with questions and follow up. If anything, I want more of those students. I don’t, however, want a classroom of students with no genuine interest in the material embroiling me in endless debates with Claude. The very thought is crushing.
With respect to the Socratic Method, I can only say: if your school does this already, and the students haven’t rebelled, then I guess keep doing it. Bobby Chesney, Dean of Law at the University of Texas at Austin - and someone who has been watching AI-related issues for the past decade - has also strenuously argued for more Socratic Method as a way to combat AI-fueled cognitive offloading. (His memo is also very much worth your time.)
I love the idea of the Socratic Method. But I do not believe that I have ever seen proper Socratic Method in the wild - not as an undergraduate student; not as a law or graduate student; not as a peer teaching reviewer. I don’t use it, because I have never seen it done, have few instincts about how to do it elegantly and efficiently, and because there are things I need to say in my lectures that I worry I wouldn’t have time to say if I felt the need to engage individually with every student during scarce lecture time. I also worry that it could easily go wrong. Like every law prof from my generation, I’ve seen The Paper Chase. And I also know that the Socratic Method can too easily become an excuse for bullying or haranguing students. Law profs are as vulnerable to the seductions of power as anyone else.
On a more institutional level, I have real doubts that many law schools - at least in Canada - are remotely willing to require Socratic Method. If anything, the institutional winds are all blowing against Socratic Method. One must remember that law schools, like the rest of the modern university, have long been in thrall to The Pedagogy Mafia Experts. And The Pedagogy Experts have insisted for years that every student has his or her own ‘learning style’, which is often not compatible with Socratic Method. The idea of forcing students to submit to a semi-public grilling is, to say the least, in deep tension with long-running currents in legal education. These are tied into worries about student anxiety and mental health. I’m not saying that these concerns are valid or compelling, but it would be a shock to the system for law schools to re-embrace Socratic Method en masse. Rolling back the clock that far would take more than a policy shift for many law schools; it would require a wholesale shift in culture.
Moving on. There is a special approach carved out for 1L Legal Research and Writing. The Statement takes the position that students coming out of 1L must have some ability to use AI in their legal research and writing - for the simple reason that law firms will expect summer students to have it. That being the case, they should be trained to use it. At the same time, there remain concerns that 1L students will lack the ability “to review, assess, and improve the output of AI tools.” They don’t know what they don’t know. To learn to use AI in a responsible, ethical fashion, students will (ironically) need extensive “human interaction” - involving close supervision and individualized feedback.
Our approach will treat writing without AI as the foundation and will layer writing with AI onto it. Throughout the year, students will write without AI, while also using AI for research, revision, iterating on drafts, and preparation for oral argument. Students and their instructors will review together both their writing and their use of AI. In this way, students will develop their own writing skills independent of generative AI tools while also developing their ability to supervise AI and critique its output.
This sounds reasonable - great even. It also sounds quite resource-intensive. At my law school, we have one or two sessional instructors to teach LRW to the entire 1L class (of about 130). Providing this kind of ongoing 1:1 supervision and individualized feedback would require far more resourcing than many (smaller, regional) law schools can afford to invest in their 1L LRW courses. Maybe I’m wrong.
The Upper-Year Curriculum
Beyond 1L, there will be no absolute prohibition on the use of electronic devices in classrooms, and no absolute requirement for Socratic Method or “in-class, no-access exams”, but this will be a default rule. The Statement emphasizes that many existing modes of assessment are AI-resilient: midterm exams (presumably in-class, no-access), group projects (really?), oral presentations, and peer-to-peer feedback. And the Statement encourages faculty to use AI where it may provide new pedagogical opportunities - e.g., “custom chatbots that serve as study aids” (so: the sort of use contemplated by the Stanford Law School and Bednar et al studies I referenced earlier) and “AI-generated practice problems” (in which students are given the sorts of fact patterns we already give them, but written by AI instead of us).
Many law schools have upper-year writing requirements: to graduate, every student must submit an extended piece of written work - typically, a lengthy research paper. The University of Chicago Law School is no different in this respect, requiring upper-year students to submit a “Substantial Research Paper”. Obviously, AI poses an especially grave threat in this context. Yet, the Statement correctly notes, we cannot abandon such writing requirements. As I said in an earlier post:
Without belaboring the point, a law school that cannot, practically speaking, require its students to write - and therefore learn to write better, more persuasively, more elegantly - is not a law school as many of us understand it. …. We can stop assigning take-home written assignments - research papers, memos, (sigh) reflective journals - and replace those modes of assessment with hand-written, in-class examinations invigilated by a platoon of heavies. That, however, is not victory. It is an admission of failure; an admission that we can no longer get future lawyers to learn some of the basic skills of lawyering.
What to do then? The answer, in essence, is to build an oral component into the paper requirement.
We will be adding one additional requirement, which is that all students will be required to engage in an oral discussion of their SRP with their supervising professor, in an in-person setting. This discussion will occur after a complete draft (or final version) of the paper has been submitted to the professor. The discussion could take place one-on-one, or as a class presentation in the style of an academic workshop. Either way, the oral exchange will involve the student answering questions that probe the reasoning of the paper and the implications of its arguments.
This is an approach that has been widely suggested as a means of making paper requirements ‘AI-proof’. I have alerted my students in next year’s seminar (Advanced Constitutional Law) that there may be just such an expectation. Hell, it could be fun. Imagine going for a coffee with your student, shooting the shit about the paper he or she just submitted, digging into this or that aspect of the paper. (“What did you mean when you said that thing you said? I found that interesting, but I wasn’t sure quite what you had in mind…” Long sip of coffee.) It doesn’t have to be an inquisition.
Moreover, there is (again) an argument that this is the kind of thing that we should be doing anyway. Consider these remarks in the Statement, with which I agree completely:
We are training students for a profession where they will be called upon to explain and defend their ideas in person and in real time, whether in the courtroom, during negotiations, when counseling clients, or when working in collaboration with other lawyers. Indeed, one of the hallmarks of intellectual life at the University of Chicago Law School is our robust workshop culture, in which faculty regularly present their own research paper drafts and address questions, suggestions, and criticisms from colleagues. An oral discussion component to the SRP is a way to formalize a piece of that workshop culture for all students.
Terrific stuff. Yes, if we want to produce intellectually engaged lawyers who are capable of explaining and justifying themselves to others, this is a good way to do it. Again, I wonder how easily this could be done outside the University of Chicago or other elite law schools. Many of my students are not accustomed to defending their ideas in one-to-one exchanges with their professors, or with anyone at all. I may wear my biggest, friendliest smile, but students will know what the score is - they’ll know, during our caffeinated walk across campus, that their answers are being judged. Some will find that extremely stressful.
And how is that judging to be done exactly? Strictly speaking, the paper is the assignment, not the ‘discussion’. If the student’s oral answers to my questions are unsatisfying as justifications for how the paper was constructed or argued, but persuade me that the student was nonetheless the paper’s author - perhaps because the paper itself wasn’t really all that good - has the student ‘passed’ the ‘oral discussion’? Or does the fact that the student can’t really defend his or her argument a matter of freestanding significance that ‘taints’ the written product? Should the paper and the oral discussion be assessed separately?
I don’t want to sound too persnickety. I raise these questions in part because they are questions that were, even before the Chicago Statement, already being raised here (and I presume elsewhere). Law schools across North America will, sooner or later, all have paper requirements with built-in oral components - that is just the future - and so we will all need to figure out how to make them work.
Finally, if it isn’t already obvious, I would stress that the Chicago Statement is quite receptive to AI in certain contexts. There will be positive investments in AI technology in the Clinical setting. And there are ‘AI and the Law’-type course offerings.
What This Could Mean for Law Students
The Chicago Statement is suitably nuanced on the use of AI in legal education. I am, as I said before, pretty bullish on most of it in principle. I believe that most law schools, going forward, are going to adopt as much of this policy as they think they can.
From a law student’s perspective, what would it mean to live under the Chicago Statement?
For starters, a student must be prepared to talk - and not just spout opinions, but submit to pointed questioning by the professor and one’s peers in a public setting. One must be prepared to demonstrate one’s understanding of cases, arguments, doctrines, statutes, and regulations, and actively defend and justify one’s interpretation. Insofar as students lack understanding of the material - and 1L students just will - they must be ready to have their ignorance exposed in open class. But even in the context of ‘oral discussions’ with professors about their written word, students should be prepared for the possibility that they will be exposed as people who do not fully grasp even their own arguments.
Many contemporary law students are not ready for this. They should steel themselves. (For the record, I am quite sympathetic. I hate engaging in oral debates.) More importantly, they need to get used to the idea that they have to actually prepare to be confronted, and to have justifications demanded of them by the professor. That means they should get used to discussing the materials with their peers in advance of class - engaging them in discussion and debate. Iron sharpens iron.
Now, if one jettisons the Socratic Method from one’s AI strategy - as I have suggested many law schools will - many of those concerns drop away. There will still be oral discussions and presentations, but no cold-calling in public settings. A student can continue to be mute during lectures. That immediately takes away much of the stress for the student, but it may also undermine much of the effectiveness of the overall AI strategy - because of course a student can just listen to the lecture (or not), and then outsource their thinking to the chatbot.
Indeed, it is worth asking an even more fundamental question: if the point of any sensible AI strategy is, in part, to force students to engage actively with the material - to struggle with it - what are we to do with other widespread practices in law school courses? For example, should law faculties rethink the common practice of distributing Powerpoint slides to students, or even the professor’s lecture notes? What about the ubiquity of ‘canned notes’ (CANs) distributed by local Law Student Associations? Surely, these also provide easy answers for law students, facilitating cognitive offloading and never-skilling. If using AI is a problem, why not these? We know full well, after all, that many students lean heavily on these resources rather than their own notes.
In a world where the Socratic Method was consistently used, none of these crutches would save the hapless student from being exposed. But in a world where the Socratic Method is actually an endangered species, law schools should take this opportunity to revisit a whole swath of teaching practices - not just permitting laptops in classrooms.
Speaking of laptops, many students will find it difficult to take notes without them - at least at first. Doing so requires a level of attention and comprehension that many have simply never needed to possess in the classroom - the ability to distinguish the core ideas and themes of the lecture from those that are peripheral. Making functional notes also requires a certain amount of discipline after class: organizing one’s jottings into a set of coherent propositions that one can understand later - say, during the final exam. There will be a learning curve as students learn how to, well, learn.
Back to the Existential Question
Before wrapping up, I want to tie the Chicago Statement into my earlier argument about the existential challenges facing law schools. I want to begin with this passage:
AI may transform the legal profession, but many aspects of legal practice are likely to remain the domain of humans, not merely because humans are good at them, but because clients, employers, judges, and society will want humans to perform them. Among others, these aspects of legal practice are likely to include oral advocacy, strategic judgment, critical thinking, and developing and maintaining relationships with clients and stakeholders. To be sure, there are ways in which AI can assist humans with these tasks. But legal education should renew its focus on training students for the aspects of legal practice for which humans are likely to remain essential.
I quite agree that legal practice will always need human lawyers. But it would be tough to convince stakeholders that law schools should exist in order to confer “people skills” upon graduates - as if they were something akin to finishing schools. After all, the best place to learn oral advocacy skills and people skills is surely in the courtroom and the office, not in classrooms. As I have said, law schools can and do offer mooting and experiential learning opportunities, but these are, ultimately, simulacra of what practicing lawyers can provide apprentices and articling students in the field, ‘organically’ as it were. It is, needless to say, question-begging to claim that law schools should exist because they provide something like (but not quite) apprenticeships.
The case for law schools, if they are to exist in anything like their present form, must surely rest on the idea that there is value - intrinsic and instrumental - in learning the law slowly, under controlled conditions; in being forced to struggle with legal ideas and doctrines and arguments; reading them and, when you find that they make no sense, reading them again; synthesizing bodies of law; gradually building arguments. The case for law schools must be premised on the value of imposing slowness and struggle on students. Otherwise, there is simply no way to explain why students should not acquire a bespoke, individualized legal ‘education’ by plugging prompts into Claude or ChatGPT.
And that, I think, is the central message of the University of Chicago Statement: If law students are to be forced to grow and stretch, they must be deprived of ready access to easy answers. That might seem startling to those who imagine that what law schools (and universities more generally) sell is information. If what we sell are facts and data points, then who cares how the consumer gets them? But the point of legal education, and a university education, has never been - was never supposed to have been - to buy information. The point is to acquire a kind of wisdom; or, if that’s too high-falutin’, expertise. A person who simply collects disconnected facts and data points is helpless when he or she confronts a question he or she has never thought to ask before. An expert, on the other hand, has sufficient depth of understanding in his or her field that he or she can look at a brand-new problem and intuitively grasp how it should be resolved given the internal logic of the discipline.
But true expertise - hell, even bare competence - is hard-won. You cannot get it except through struggle; through steady engagement with material over an extended period of time, and by paying close attention to human beings who are already experts in the field. And, importantly, one must have something at least approaching that sort of deep, intuitive understanding in order to properly determine not just whether a legal argument can be run but should be run - strategically and ethically. I wrote a little bit about the ethical aspects here. At the time, I was focusing on the use of AI by judges (specifically, in constitutional cases), but the “duty to think” (to quote Frank Pasquale) is at least as important for lawyers and law students.2 Hence the suggestion in the above quoted passage that law students must learn “strategic judgment” and “critical thinking” in order to effectively be accountable for the arguments and analyses produced by AI; in order to “ethically” use AI in their legal research and writing, and in their professional work.
The Chicago Statement, without descending into Ludditism, tacitly recognizes that the grueling nature of legal education is not a bug, but a feature. I want to gently suggest that law schools have often lost sight of that fact. Now is a good time to recover it.
Have a great weekend, everyone.
Yes, the Statement recognizes that there may be a need to make accommodations for some students.
In my earlier post, I expressed some skepticism that law schools are as well-placed as other university departments to provide a certain kind of ethical-philosophical-historical education. But maybe it’s enough that law students, as they acquire legal expertise, become accustomed to engaging in interpretation in ways that are suitably sensitive to moral and historical considerations. As many have observed, one of the great dangers of over-reliance on AI is that human beings will suffer from a kind of hermeneutical de- or never-skilling. See, in particular, Andrei Marmor’s “AI and the Loss of Hermeneutics”, but also the work of Kevin Lee.
> The case for law schools must be premised on the value of imposing slowness and struggle on students.
I love this SO much! A terrific argument and an endlessly entertaining piece of writing.