While everyone gets mad about college students using AI...I am extremely curious about this very vague survey cite in the NY Mag story
May 7, 2025 14:29The closest match I can find is this from January 2023, which found that only 46% of students were familiar at all with ChatGPT and just 30% said they'd used it
www.intelligent.com/nearly-1-in-...Nearly 1 in 3 College Students Have Used ChatGPT on Written Assignments - Intelligent
Of the 46% who said they were familiar with ChatGPT, 64% (30% of the total sample) say they have used it to help them complete a written assignment.
Pew, which has data I actually trust on this, finds that about one quarter of teens say they're using it
www.pewresearch.org/short-reads/...anyway, I would need a lot more evidence to believe that 90% of college students in early 2023 were using ChatGPT for homework, and if we are going to berate the Youth for intellectual laziness, let's at least make sure we apply some amount of rigor to the data we're using to do it, please
Looks like this might be it, in which case the methodology statement reads in full: "Study.com surveyed over 100 educators and 1,000 students in January 2023, over the age of 18, on the use of ChatGPT in schools."
Which is not encouraging!
bsky.app/profile/sand...
Online Courses for College Credit, Exam Prep & K-12 | Study.com
Take online courses on Study.com that are fun and engaging. Pass exams to earn real college credit. Research schools and degrees to further your education.
For illustrative purposes, the study dot com "methodology" (first image) vs. Pew's methodology statement (images 2-4)
(This is veering off topic, but low quality nonprobability samples seem to significantly inflate use of AI and I'm curious why...could make an argument either for sample composition or for the incentive for respondents to say yes to screen themselves into a survey)
I also feel like folks who don't spend a lot of time with survey data don't necessarily have an intuitive understanding of how unlikely it is that *9 in 10* students would all be early adopters of new technology. Society is wonderfully diverse! It is hard to find 90% doing anything!
I also expect the number of students who have used it in a crunch to be reasonably high and the ones who do literally none of their own homework to be comparatively small.
also "Had used for help" is very low bar. esp. since prof. are being told to have students use it and the revise - terrible idea in my opinion but it's likely many of them were actually required to use it in some way.
Why do we take NY Magazine seriously? For as long as I have known of it, they have had the occasional alarmist article that typically turns out to be exaggerated beyond recognition. I suspect the survey relied upon lacks rigor or was done to confirm the author's predetermined thesis.
This doesn’t surprise me. I saw solid evidence of rapid adoption at my institution.
I teach. I believe it.
Same with copy/paste and early internet days :)
"used" is a word with a broad definition. It's a terribly worded question.
I was 12 when the first cheap consumer calculators came out. Most HS & college math classes banned their use for a decade after, and then they stopped. Probably a similar story later with graphic design courses & design software like Adobe Illustrator. Just sayin.
Dunno bout early '23, but am really glad I'm no longer teaching college comp in mid '25, as that must be awful now. There are decent AI prose detectors but you obviously can't detect research & organizing, which is fully half of composition skill. OTOH, can we even maintain that 2nd assertion now?
Whilst I assume my students (16+) are generally ahead of me on things like AI. Not one of them was using it then. It turns up daily now.
I don't (can't) ban it for research. Now I just try to teach the pitfalls. It is useless for regulations/legislation and easily proved incorrect so students soon learn not to bother anyway.Lucky for me it's a mainly practical subject, can't cheat that.
I expect a gap this big is due to deliberate omissions. The most likely one is that it's a survey of students in a compsci class about AI.
Productive Teaching Tool or Innovative Cheating? | Study.com
ChatGPT is a new AI tool that has swept the education world. With its development, it has also welcomed a new host of problems that is leading to numerous questions. Students, professors, and educators are all wondering what the impact of this tool can be now and in the future.
Thank you!
Why do we take NY Magazine seriously? For as long as I have known of it, they have had the occasional alarmist article that typically turns out to be exaggerated beyond recognition. I suspect the survey relied upon lacks rigor or was done to confirm the author's predetermined thesis.
