This is a solution and one lots of people are going to. It works. It’s also pedagogically terrible. Fundamentally ableist in structure. It assesses skills that are not the relevant skills for any meaningful intellectual activity. It represses skills that are meaningful. I don’t have an answer.
I have a simple solution to this. Grade students entirely by means of three hour hand-written exams in a big hall with no computers. If they all fail because they never learned to write without AI, sucks to be them
May 7, 2025 12:25That’s not true. The solution is: students who care about learning, small classes, deep, humanistic connection, no grades or grades solely on process rather than outcome, changing the whole notion of assessment to remove the things that rewards cheating.
But of course, that takes resources that most institutions don’t have, as well as a fundamental rethinking of what is a grade, what is value? So … in class exams are the solution to LLM cheating but at least want everyone to recognize the losses that come with that solution.
The answer cannot be that we can never assign independent research again.
I know!!! It can’t be! I want to believe it’s possible to teach students to care enough not to use LLM’s. I’m sure smarter people than I have process-based assessment that at least mitigates it.
It's a great example of managing to regress into the worst kinds of 18th and 19th century education? Somehow technology is sending us hurtling backwards? But I didn't need that illustrated.
This is how my finals were in the late 80s/early 90s. Blue books.
There were often accommodations for people who didn't test well this way. Oral exams, usually.
as someone who sucked at that but was very good at research that would have been miserable for me
I think anyone who is serious admits they don't have an answer. I've got 152 HS students this year. The system that we've had for the past 70+ years isn't equipped for what most would consider authentic assessment.
My solution so far has been to focus on authentic assignments as much as possible. Lower stakes, more opportunity for corrections/retries, etc. Legally I have to give assessments, but I don't have to have them weigh more than the work.
Some things I have done in my class to bring learning in w/out AI, if anyone wants ideas:
- In class “conference” with poster presentations about their final projects - I bring markers, paper; posters made in class (quality of poster is *not* judged), they present to groups of peers in rounds;
We have weekly readings with discussion posts - a lot of this gets slopped by AI, so we reserve one discussion question to kick off each session. Each week a couple of students (at random) are singled out to have to kick off the discussion in conversation with each other (everyone gets 2 “passes”)
Education is another one of those areas where there are plenty of countries that do an objectively better job than us and I don't know why we can't just copy them.
The military uses pass/fail oral boards for many qualifications. Open book and notes, but you have to be able to explain your reasoning to the people who are going to trust you to drive the ship or whatever.
Of course that also requires three instructors and an hour or more per student.
right. a resource/labor issue rather than a question of what's best.
Some of my colleagues want PhD comps to be in a room, no Internet, closed book & oppressively long. They are so focused on stopping AI use, they are creating an exam that is not at all based on reality. In no modern setting is this what we ask of scholars.
If students are using AI to cheat & graders can't tell (e.g. catch them), the problem isn't the students, it's the exam or the training in general
Law schools have long done exams on computers that lock other software. People can still type. The answer is to force them to write.
Forcing people, who never have to write any more to suddenly revert back to hand exams is terrible.
A challenge is that the level of individual attention that becomes necessary for every student to find an AI-proof evaluation method becomes impossible in the modern mega-sized class setting. But if you don’t have huge class sizes, how can you afford to pay the football coach $25 million per year?
This year in my own class I’m trying to find ways to adapt to the existence of AI. I’ve decided to allow it, subject to the university policy on its use, and have asked the students to tell me if they’ve used it and explain why they liked the AI response. I’m hoping asking for that meta will help.
I'm not sure it's ableist, at least at schools (like mine) that have places with clean computers where students with disabilities can take exams.
I grew up in a time when blue book essay tests were common. But reverting back to that now with a student population that hasn’t had to hand write anything and doesn’t know or use cursive just tells you who has writing stamina, not who mastered the material.
Yup. It says something that I didn’t realize I had a motor disorder until college, when I was shut in a big room at a cramped desk with a pencil and required to fill up bluebooks for hours on end.
And once diagnosed with dystonia, the hoops I had to jump through to get an accommodation to take my exams on a computer (this was mid-‘90s) and the skepticism I received from several professors about my disability almost made me want to go back to bluebooks.
I'm wrestling with this right now in a comp class, and I think I'm having at least a little success with lots of in-class writing and in-class *revision.* Literally hanging over their shoulders asking "What do you mean here?" and letting them squirm. Only feasible with <20 students though.
I cannot fathom a reason these have to be done by hand
In my department we’ve talked about adding an oral exam component to classes to just talk to students about what they’ve learned and ask about their assignments. This of course couldn’t work on huge classes but in smaller advanced ones it’s a possible way to weigh what a student learned
Also my students handwriting is some of the worst I have ever seen. Can’t imagine reading eight hand written essays! I am older and my eyesight isnt what it was.
A lot of people are hung up on what we need to do to ensure a clean product. This misses the fact that the product is, generally, the least important part in terms of learning. It’s the process. This solution requires stripping away so many vital parts of that process.
Lost me at “three-hour exams”.
I think this solution works best in heavily quantitative fields. Either a student can apply Newton's laws to a situation or they can't. Being able to apply the laws of physics to a novel situation (even incompletely or incorrectly) *is* a good assessment of whether they actually learned anything.
It's not "pedagogically terrible" for engineering exams where you're posed a problem to solve, and you need to draw diagrams, formulate & solve equations, show your work. Same as your homework problem sets.
IRL computers are used AFTER you understand the problem & formulate the steps to program it.
Ok
I really agree. For one thing, typing isn't so terrible. Plus, I'm doing a Masters: some assignments are practice at doing the task: eg, to plan some research, which involves researching, thinking, refining. Not relevant to see what somebody can do in 3hrs.
This user blocks the author of the focused post.
Being able to articulate & structure an argument--as in an essay--is among the most relevant skills humans can have. It's essential not only to making one's own viewpoint understood, but to analyzing others' arguments: a key skill in democracy.
Many accommodations are available for in-class essays.
I agree, though sadly without an answer either. Funny thing is it made me think about 30(ish) years ago sitting down to do some revision for my 1st year degree exams and realising I hadn't written an essay from start to finish by hand since my A level exams!
Still feels wrong and inaccurate to base a grade for years of study on 3 hours on an arbitrary day
[This post was deleted]
I hate this “solution.” If it were that easy, wouldn’t we already be doing it?
I would also question how this strategy would accommodate a research essay or project. That would require work to be done outside of class and is essential for a lot of degrees.
Why isn't this pretty much the same skill as writing a paper? Reading something, understanding it, and being able to write about in such a way that you can explain it in writing is not a relevant skill for any meaningful intellectual activity? This is not a college-level skill?
What about half hour in person verbal exams, relying in part on TAs for lectures? 3 hour handwritten exams is 6 half hour verbal examinations. Also gives the professor and TA flexibility to gauge knowledge and comprehension.
True. But isn’t any kind of in-class timed exam ableist to some extent? Whether you hand write it or use a computer?
This also doesn't work for anyone attending college in a non-traditional path where they aren't on campus in a lecture hall full time. Taking evening and weekend classes because you also work, or online classes because you have a family? Guess you're SOL.
Nobody writes by hand, nobody needs to. There used to be computer labs with just software, students had to rely on themselves. Now those labs are too expensive to maintain, and they use their own laptops. Only time limits their access during exams (ableist too, no?). We pretend this is something.
📌
but aren't all tools used to measure knowledge flawed and biased in some way? is this really such an awful solution? at the very least, as you said, it works.
and I'm sorry to disagree but producing factual content under pressure is definitely a relevant skill for many intellectual activities
Yes. Thank you. This is fundamentally ableist.
Hoping you will be able to share these views at the CLA symposium on AI next Tuesday.
Nope. No one listens to me. Why bother.
get Amstrad to make a computer like the one I bought in 1983: It typed my letters, including allowing me to make spelling and grammer corrections, and do basic arithmetic. But it did not have spell-check, let alone an internet connection.
Maybe have them do outlines in class or sketch the arguments they want to make.
The ultimate answer is for educators to rethink their educational system, but educators tend to not be innovators but rather cogs in a system.
Systems theory says parts of the system will always work to retain system integrity - and that's exactly what educators are doing and why they're failing.
Look at your suggestion: let's just return to a previous horrific state of our educational system.
What's most likely is that good solutions will come from innovators outside the current educational system - those who don't have a vested interest in maintaining the current educational system.
I still think of ChatGPT like a calculator. When I was young, we learned math without calculators, but eventually being good at math meant doing advanced calculations with a calculator. This person's idea is like gathering college students in a room and having them multiply 4-digit numbers by hand.
My guess is that today's AP Stats and AP Calc students are doing math far beyond what was thought possible in high school a century ago, partly due to calculators. Humanities have to make the same uncomfortable leap by raising the bar of what students learn and produce with AI.
if students could do the exams on unnetworked word processors, maybe? but you’re still going to run up against timing issues for various disabilities, and the professors who are most concerned about cheating are often the least accommodating of learning disabilities.
You sound like the type of student that prefers multiple choice to essay questions.
You sound like the type of poster who responds to the first post without reading a two post thread, or paying attention to who you’re talking to.
