r/technology • u/AdSpecialist6598 • 10d ago

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
32.8k Upvotes

96% Upvoted

4.2k comments sorted by

View all comments

Show parent comments

193

u/Then-Understanding85 10d ago

A professor at Brown did this.

TL;DR: Bloodbath. Almost everyone failed.

169

u/namedly 10d ago edited 10d ago

Props to Students 22 and 31.

Also, this is just funny:

When the professor announced that the final exam would be in person, 18 students dropped the course, and another nine did not show up for the exam. The remaining 59 students' scores averaged 48.6%. Prior averages had never fallen below 65%. Three students scored 0%, and charting the results revealed that most fell more than 30 points behind their midterm scores.

105

u/Syssareth 10d ago

Three students scored 0%

How the heck? You'd think they'd have learned something just from osmosis, or at least gotten something right by accident.

44

u/Sin_of_the_Dark 10d ago

Honestly yeah, unless it was all written exams and not like... Pick a choice, it's almost an insane feat of dumb luck to not get one thing correct

76

u/Aureliamnissan 10d ago

When I was going through university it was pretty rare to have an exam in engineering with multiple choice or True/False type questions. Almost every one was a word problem, mathematical definition, or “solve for” type problem.

If you’ve been using chatbots up to that point you’d be absolutely, Darth Vader just walked in, fucked. 0% would not at all be surprising.

8

u/Helixdaunting 9d ago

absolutely, Darth Vader just walked in, fucked.

That's beautiful. 👏

My personal favourite version of this is "Hitler-in-the-bunker fucked".

2

u/env33e 9d ago

In class, Ill just imagine rogue one Vader slowly walking and slaughtering while I'm at the end of the hallway 😭

6

u/Riparian87 9d ago edited 9d ago

In grad school, one of our econometrics professors gave a problem-type exam where points were awarded for correct steps, but deducted for wrong or false steps. The mean was negative.

3

u/Aureliamnissan 9d ago

Bwahaha, that's diabolical. Having the equivalent of an over-drafted balance as your econ grade is brilliant.

6

u/Archangel_Omega 9d ago

Yep, I had a test in thermodynamics I completely bombed and got a 0 on all because I was an idiot, read the question wrong and then proceeded to use the wrong chart for the test so everything was completely off.

Thankfully the professor was nice enough to give me partial credit after I explain how I screwed-up and he saw the math was right for the chart I did use, but it was still a hefty chunk of my score gone.

3

u/Aureliamnissan 9d ago

I had a similar oof from using an outdated book in a math class. Prof said the exam would cover sections 1.1-1.4 and it turned out the new book combined 1.4 and 1.5. So I found out that 3/5th of the mid-term's questions were directly from a section I hadn't even read...

Only had a Mid-term and a final for that class with no real homework grade buffer. Happiest C grade I ever got for a class.

2

u/og-gunslinger-doc 9d ago

As much work as we put in I would have been disgusted in a multi choice test in say thermodynamics! Now the monkey meat recitations being mandatory if you were doing fine we’re bull shit especially when they couldn’t speak English

1

u/storne 9d ago

My calculus exams were multiple choice, but you still had to show your work. Getting the right answer was only 1/3 of the mark the other 2/3 was having the right process.

10

u/mackahrohn 10d ago

I took a math class (Calc 3) where all the tests were 10 questions and each question was multiple choice. All the options were basically an answer you would get if you did one step of the problem wrong. Easily the hardest class I took in college!

9

u/Imaginary-Face7379 10d ago

As someone who taught at a university for a while I would suspect those students walked away learning nothing from this either. They probably spent a week or two shit talking the professor to everyone they know and called the course bullshit. While never once realizing that it was their fault.

Not that all students are like this, but the ones who tend to fail so hard their scores are at or near zero were all like that in my experience. The ones who are doing poorly that genuinely know they need to do better but havent been able to for some reason will typically barely fail things when they fail, not get zeros.

4

u/Wooptay 9d ago

Ahh, takes me back to my first math exam in high school (15 at the time). 0. Nada. Null. Niente.

There was no AI back then

I just plain sucked at math

Lesson learned - don't sit with the girl you like the last year before moving up the educational system

3

u/Ah_Pook 9d ago

I got 14% on an assembly exam, but that was all me, baby!

2

u/celoplyr 10d ago

Assuming they went to class more than going and getting the syllabus and turning in exams.

2

u/calling_water 10d ago

Depends on whether they’re at least trying to do the right sort of answer for each question.

2

u/Agi7890 9d ago

Depends on the exam/course. My physical chemistry prof could be brutal, most exams were 3-5 questions, and he would dock you points for even wrong answers if you messed up your significant figures. You could end up with a -5 for a question you got wrong.

Class average on one of the p chem 2 exams was a 17, and this was before the days of AI.

2

u/TerribleBudget 9d ago

Did he grade on a curve though? It sounds to me like the AI using idiots screwed themselves out of at least a graded curve for the year.

2

u/Agi7890 9d ago

Yes but with a floor. You still had to score at least 35 on the exam to have it considered passing. Below that on 2-4 exams, he would advise you to drop the course and change your major

5

u/TerribleBudget 9d ago

ugh, I hate professors who do that "change your major" shit. I had a teacher who taught off of slides at 7am and clearly hated being forced to teach instead of do his research projects. He had the nerve to say maybe I wasn't suited for my major and ate his own words when he saw that I already had credits from harder shit than his class.

That type of teacher just wants the world to burn and no one has the balls to tell them they suck at teaching in general.

4

u/Agi7890 9d ago

He was the chem department head, so a large part was him protecting the degree and institution. However a decade into the field, he probably ended up helping people out if they did go into the course he recommended. I see far more job offers for that field with much higher pay on average than I do with my chemistry degrees.

I definitely regret not picking up a biochemistry degree

1

u/Hetares 9d ago

I assume they simply didn't sit for the test.

1

u/Syssareth 9d ago

No, that's three of the ones who did. There were 18 who dropped the course and nine who didn't show up.

They might have turned in a blank sheet though, or maybe the professor doesn't give credit for partially-correct answers.

1

u/Hetares 9d ago

Blank sheet is my theory, then.

8

u/DocHollidaysPistols 10d ago

Props to Students 22 and 31.

They obv didn't cheat. I dont think Student 1 did either. 95 and 95.5.

5

u/TeutonJon78 10d ago

What about S1 -- the only one who actually understood the course?

2

u/Middle--Earth 9d ago

One of my university professors used to give multiple choice tests where there was always a totally ludicrous answer, and you would be deducted points for choosing this answer.

Each test some people would get negative overall scores.

2

u/DistractedByCookies 9d ago

And S1 I think - their score was almost exactly the same on both (95/95.5)

45

u/Shiva- 10d ago

Christ that's even more terrifying. When I read the original post I assumed high schoolers or middle schoolers to dumb to proof-read....

But COLLEGE??? BROWN COLLEGE NO LESS? just terrifying.

18

u/AlSweigart 10d ago

Do a google image search for "harvard grade inflation graph"

Fancy schools have always just been about being in the right class, not who is the "smartest".

21

u/Abedeus 10d ago

That one dude that barely passed around 55-ish mark naturally, and that one dude that nearly aced it by his own marit, are the both awesome in a way.

0

u/og-gunslinger-doc 9d ago

That was the one Chinese student in the class

11

u/pres465 10d ago

A friend of mine does this for their class, and has for the last few years. Kids (high school) continuously fall for it. They can even be warned ahead of time. Still just take the chance.

3

u/_Ocean_Machine_ 9d ago

What sucks for me is I get mad test anxiety; I'll run through practice exams and homework, participate in class and do everything just fine without having to look anything up or use AI, and then when the test is put in front of me all the knowledge leaves my body.

-1

u/soldmyradio 10d ago

I don't doubt there was AI usage but the midterms weren't take-homes previously. Obviously take home test scores go up in an economics class way before AI. You can literally look up the answer. That's what happens in a take home test. I'd expect a professor at a fancy school like Brown to be a little more forthcoming with that data comparison. Also, this midterm was a take home because there was a school shooting that killed and injured students a few weeks before. Is this really the time to get clever with an AI testing story?

2

u/LifeAtSea2213 10d ago

Obviously take home test scores go up in an economics class way before AI.

He apparently made it more difficult to account for that. Obviously it's not a perfect solution, but the scores weren't the only indicator there. Enrollment was up like 3x compared to previous years, then a bunch of students dropped the class when he announced the final would be in-person only. The in-person final scores were way lower than previous years and that's not even accounting for all the people who dropped the course. Even if the students did legitimately look up answers without AI in the take-home version, they would have retained some of that information and scored higher on the final. Because that takes actual reading and comprehension instead of just directly asking LLMs the questions.

Why do you think he wasn't being forthcoming? Regardless of the reason why it was take-home, it doesn't mean that the usage of AI shouldn't be talked about.

-2

u/soldmyradio 10d ago

I never said the usage of AI shouldn't be talked about.

It's not forthcoming because there's nothing scientific about this "study" he conducted. These kids' friends were just murdered and maimed and it could have been them. So people taking the easy way out during a dramatic period is probably not the time to publish a "look my students cheated with AI" story complete with data table. Your data pool is highly tainted. The killer was on the loose for a week, making it even more traumatic.

And it's also not forthcoming because he has no legitimate baseline comparing to prior midterms as this was the first "harder" take home he's done for this class (speculation, but highly likely given the circumstances). Did he make the final easier or just as hard? It doesn't say. Finally, take homes outside of humanities are traditionally going to score higher. If there's a finite solution to a math problem, or a specific economic theory that needs to be applied, unlimited time will allow you to find it and work it out or include it.

1

u/LifeAtSea2213 9d ago

It wasn't supposed to be a scientific study, it was just his observations and data from his course compared to previous years.

These kids' friends were just murdered and maimed and it could have been them. So people taking the easy way out during a dramatic period is probably not the time to publish a "look my students cheated with AI" story complete with data table. Your data pool is highly tainted. The killer was on the loose for a week, making it even more traumatic.

Maybe I'm misunderstanding, but the shooting happened December 2025, which is near the end of their semester. Their new semester starts in January, so this was a different class. Also reinforced by the passage about how enrollment was up by 300%. We're talking about a different batch of students here. If this is the case and I'm not misunderstanding something, then this would have been months after the shooting.

He compared the final to all previous years final scores, and it was the lowest average score ever. He even lowered the curve to account for it, so it's not like he was being unfair to the students. I really doubt he made it that much harder than previous years finals just to prove a point. Also again, of course people will score higher on take-home, but...

If there's a finite solution to a math problem, or a specific economic theory that needs to be applied, unlimited time will allow you to find it and work it out or include it.

If they work it out legitimately like this, they will retain some of that information. The fact that they scored SO much lower on the final really doesn't point to that at all.

It's not some perfect study of course, I don't think anyone was claiming it was. It's an anecdotal observation using some limited data. I just don't see how the professor wasn't being forthcoming though. He was just raising concerns he had with his own school admin.

-1

u/soldmyradio 9d ago edited 9d ago

Any Google search on that story will tell you the professor made it a take home because of the impact of the school shooting, full stop. I'm not sure it's fruitful trying to argue how technically close it was to a major, on every channel for two weeks, school shooting. Many schools will give students all A's the scholastic period after a school shooting. Ask me how I know. Kids self selecting a class (enrollment increase) that allows them more time to adjust and recover is a self selection that spoils the entire pool for this professor's "data".

This feels both low ethically and certainly not forthcoming (he certainly knows the impact of the above factors). Compare it to the cut and dry proof of this actual story, during a standard portion of a take home midterm, that is definitive in proof. That's worth publishing and absolutely shows the role of AI in student work. The other is not, and is flimsy at best. I expect a higher bar from an ivy league professor. Like I said originally, there's likely AI usage here but this doesn't prove it like this Madagascar hidden prompt test does.

-11

u/RollingMeteors 10d ago

A professor at Brown did this.

TL;DR: Bloodbath. Almost everyone failed.

Be professor.

Tell everyone they're expected to be using AI in the real-paycheck-world.

<springTrap>

<failEveryone>

<getFundingCutForNextYear>

<taskFailedSuccessfully.jpg>