The Hidden Cost of AI-Generated Feedback
What new research tells us about the potential risks of "dehumanised" feedback
Hey folks 👋
This week I read a new study published by MIT Media Lab on AI-generated feedback which raises some interesting questions about the potential risks and benefits of replacing human feedback with AI.
When it comes to AI-generated feedback, we often ask: can AI write feedback as well as a human educators?
Much more rarely do we ask what I think is the more critical question: can AI make learners feel seen as individuals and make them feel a sense of belonging, which is so critical to their engagement, motivation, and—ultimately—achievement?
This is exactly the question asked by Morris and Maes (2026) and the answer is fascinating.
In this post I share what the new research by Morris and Maes (2026), and other work like it, tells us about some complex and perhaps uncomfortable realities of the impact of swapping human-generated feedback for AI-generated feedback.
Let’s dive in!
What Do Learners Actually Need in Order to Learn?
We talk a lot about what AI can do for learning — generate feedback, explain concepts, scaffold tasks, personalise pathways. But we talk far less about what learners actually need in order to learn. Importantly, these are not the same thing and in practice they’re often in conflict.
Here’s a quick summary of what decades of online and blended learning research tell us.
1. Learners need to feel an instructor is really there
The Community of Inquiry framework — one of the most extensively researched models in online education — tells us that perceived teaching presence (the sense that an instructor is actively designing, facilitating, and directing learning) is a strong predictor of both perceived learning and satisfaction. A meta-analysis of CoI studies confirmed this across online and blended settings (Richardson et al., 2017). In online discussions, students’ perceived presence of the instructor correlates more strongly with satisfaction than perceived presence of peers (Hostetter & Busch, 2013).
This isn’t about content quality. It’s about learners sensing that an instructor is actively involved. When that signal disappears, motivation drops — even if the materials are excellent.
2. Learners need social connection, not just access
A 2024 systematic review of 28 studies found that social presence consistently predicts higher learning satisfaction and stronger persistence in online learning (Salihu & Alemu, 2024). The CoI meta-analysis showed that social presence interacts with teaching presence to support cognitive presence — the deep thinking that is the whole point (Richardson et al., 2017).
Even structural details matter: small, stable discussion groups significantly increase perceived social presence compared to whole-class forums (Akcaoglu & Lee, 2016). Feeling “with” others supports sticking with hard tasks.
3. Learners need to feel seen as individuals
Studies using the CoI framework show that teaching, social, and cognitive presence together predict engagement and learning outcomes. Learners explicitly value personalised guidance and interaction as integral to their experience (Putra & Santosa, 2025). Research on social presence cues — personal profiles, photos, identity markers — finds these help learners feel connected and reduce the anonymity that depresses participation (Phirangee & Hewitt, 2016).
This is not about personalised content. It’s about personalised attention — the feeling that someone knows who you are and is responding to you.
4. Learners need belonging
Conceptual and empirical work on relatedness, belonging, and connectedness argues that a sense of belonging — being accepted and valued by others — is a core psychological need underpinning motivation (Peacock & Cowan, 2023). A 2024 study on social capital in online learning found that both close ties and broader networks are positively associated with course satisfaction (Bates et al., 2025). This is Self-Determination Theory territory: relatedness isn’t a nice-to-have. It’s foundational.
5. Learners need human interaction to self-regulate
Research on “learning presence” in the CoI model shows that successful online students use cycles of goal-setting, monitoring, and adjustment that are embedded in social interaction and instructor guidance (Shea et al., 2013). A systematic review on persistence highlights social support from instructors and peers as a key mechanism behind persistence (Greenland et al., 2024).
Learners don’t just need content and feedback. They need human contact to help them regulate their own learning.
TLDR
Learners are more motivated, persist longer, think more deeply, and learn more when they experience visible instructor presence, social connection, being seen as individuals, belonging, and human support for self-regulation. These aren’t soft outcomes or student preferences. They’re robust predictors of learning across hundreds of studies and multiple theoretical frameworks.
AI-Generated Learner Feedback: Silver Bullet or Empty Promise?
So what happens when AI enters this picture?
Right now, the question most people are asking is: “Can AI write feedback as well as a human instructor?” But given everything we know about what learners need, that’s not the most important question. The more important question is: does it matter to learners that the feedback isn’t from a human?
Does “always on”, hyper-personalised AI feedback make up for the loss of the human signals we just spent five sections establishing are critical to engagement, motivation, and learning?
A study published this year by Morris and Maes (2026) provides one of the clearest experimental probes so far of this pattern, albeit via a small, experimental pilot.
In a creative-coding course, all student feedback was generated by the same large language model (Claude). The feedback was tailored to each student, well-structured, and high-quality. But half the learners were told the feedback came from an AI system. The other half were told it came from their human TA, “Cass.
Same words, but with different source framing and interaction cues. Here’s what happened in this short, single‑session course:
The AI feedback was good — and learners recognised that. Both groups rated the feedback as equally helpful. By any quality metric, the AI-generated feedback passed.
But what learners did with it was completely different. The learners who believed a real person had read their work spent significantly more time on later modules, ran their code more often, and wrote more code. These weren’t subtle shifts in survey responses — they were large, measurable differences in behaviour over the rest of the (albeit short) course (Morris & Maes, 2026).
In other words: educators used AI to create and deliver optimal-quality, optimal-quantity feedback for every student. The learners said the feedback was good. But when they knew it was AI, they didn’t try as hard.
The key finding here is this: technical success — high-quality, accurate, personalised feedback at scale — was not pedagogical success — behaviour change, persistence, deeper revision.
This isn’t an isolated finding:
Morris and Maes confirms a pattern emerging across the research. A 2026 systematic review of human vs ChatGPT feedback across eight studies and 461 students found that while AI feedback was more detailed and immediate, students valued human feedback for personalisation, emotional nuance, and context-specific guidance (Alkhalaf, Alkhayat & Alzahrani, 2026) — the same qualities the CoI research identifies as teaching presence.
When students in a large survey said AI feedback felt opaque and less contestable (Chan & Hu, 2023), they were describing a breakdown in the social interaction that supports self-regulation. When a review of GenAI feedback in higher education found that AI-generated comments improved writing quality but didn’t automatically push learners into deeper revision (Owan et al., 2023), that’s the active processing problem the learning presence research predicts.
When Morris and Maes (2026) found that the “TA Cass” group worked harder, they were documenting social accountability — the same force that the belonging and relatedness literature identifies as a driver of motivation. Wisniewski, Zierer, and Hattie (2020) found in their meta-analysis that feedback’s effect on learning is moderated by how it’s processed, not just how it’s delivered — and that beyond a certain point, more or more detailed feedback can reduce impact if students can’t prioritise it. AI tools make that overload problem trivially easy to create.
And when studies outside education find that people perceive AI as competent but low in warmth (Bogert, Schecter & Watson, 2023), that they reciprocate kindness from humans more than from algorithms (Luzsa & Mayr, 2023), and that they’re more willing to exploit benevolent AI (Capraro & Capraro, 2021) — they’re describing the absence of the social reciprocity that makes feedback feel consequential.
Eye-tracking research on how students read written feedback confirms this: the behaviours linked to learning gains are active selection, processing, and revision planning — not the mere receipt of comments (Mao & Benesch, 2023). When feedback comes from a person, learners feel more compelled to do that work.
The “feeling gap” isn’t a bug in learners’ reasoning. It’s what happens when you remove the human signals that decades of research tell us drive engagement, persistence and deep processing.
When AI-Generated Feedback Is Probably Fine — And When It Isn’t
Before I go further, I want to be clear that I’m not arguing that AI feedback doesn’t “work”.
The weight of current evidence suggests it can be effective — particularly on structured tasks and when used as a supplement to existing teaching. A meta-analysis of human–machine feedback in smart learning environments (35 studies, 2,200+ learners) found significant positive effects on learning processes (d ≈ 0.59) and outcomes (d ≈ 0.41) when automated feedback is added to instruction (Zheng et al., 2023).
A meta-review of automated writing evaluation for ESL learners reported moderate gains in writing performance (g ≈ 0.60), with immediacy and frequency as strong moderators (Karimova & Csapó, 2024). When AI prompts are aligned to rubrics, the output can be consistent and criterion-linked in ways human markers struggle to maintain (Owan et al., 2023) — and in domains like physics and clinical anatomy, around two-thirds of LLM-generated feedback required only minor editing, while the rest needed substantial correction (Zhu et al., 2024; Kääriäinen et al., 2025).
AI feedback works, but it “works” on specific measures. Many of these studies focus primarily on performance outcomes and satisfaction ratings, and most are short-term. Far fewer look at what Morris and Maes (2026) measured: what learners do with the feedback over time — how hard they try, how much they persist — and how those behaviours might relate to achievement across later stages of a course. This is where the complexity and nuance sits, and where more research is required to understand the impact of AI on human learning more fully.
The work of Morris and Maes, and others, doesn’t mean that every piece of feedback needs a human behind it. The evidence suggests a boundary, and students themselves are clear about where it falls.
AI-only feedback and support seems most effective when tasks are low-stakes, tightly structured, and not identity-relevant: grammar correction, coding syntax, multiple-choice practice, drill exercises. In these contexts, AI is functional, welcome, and effective. The Karimova and Csapó (2024) meta-review found its strongest gains in exactly these structured, skill-based domains.
Students consistently draw the line themselves. In Alkhalaf, Alkhayat and Alzahrani (2026) and across domain studies, they say: AI for drills and drafts, humans for capstones, professional work, and emotionally difficult tasks. The boundary hinges on identity relevance and emotional stakes. When the work is “just practice,” AI is a useful tool.
When the work is “this is me showing what I can do” or “I’m not sure I belong here,” learners need to feel a person is on the other end. This aligns with the systematic review’s conclusion that ChatGPT can complement but not replace human feedback, with a clear preference for hybrid models (Alkhalaf, Alkhayat & Alzahrani, 2026).
This is useful for us as designers: not all feedback or support needs the same level of human presence. The question is whether you’ve correctly identified which moments do and which don’t — and whether you’re protecting human presence where it matters most.
Implications for Learning Designers
So what do we do with this? The Morris and Maes (2026) findings, combined with the broader evidence base, don’t demand that we stop using AI. They demand that we think about learners’ needs as carefully as we think about our own productivity.
Here’s what that looks like in practice:
1. Use AI to increase your feedback capacity, not to decrease your presence
Let AI handle the volume — rapid, structured comments on low-stakes work so learners get frequent signals. But reinvest the time you save into more high-value human interaction, not less total interaction. A review of educators’ reflections on AI-automated feedback found exactly this: the freed time should go toward the interactions only humans can do well (Slade et al., 2025).
Stop treating AI as a way to reduce your workload. Start treating it as a way to redirect your attention to where it matters most.
2. Make your human attention visible
When you use AI to help generate feedback, don’t let it be a black box. Add a short human note, an audio comment, or a debrief that references the AI feedback: “I’ve reviewed the AI’s comments — here’s what I’d focus on first.”
This is what learners consistently say they want: AI detail plus human nuance and direction. Alkhalaf, Alkhayat and Alzahrani (2026) found students preferred a hybrid approach — and Morris and Maes (2026) shows us why: the visible human presence is what triggers the engagement.
Stop sending AI-generated feedback without any human fingerprint. Start wrapping AI feedback in visible signs that a real person has reviewed it and is paying attention.
3. Use AI to make your human feedback better
Here’s the possibility that gets lost in the “AI replaces humans” framing: AI can help you give better human feedback. Use it to generate alternative phrasings, to check your comments against frameworks like Hattie and Timperley’s (2007) “Where am I going? How am I going? Where to next?”, to draft first-pass comments that you then refine and personalise.
This uses AI as a design tool in service of human presence — not as a replacement for it.
Stop thinking of AI feedback and human feedback as an either/or. Start using AI as a thinking partner to improve the feedback you give personally.
4. Triage by stakes and identity relevance
Not all learning moments need the same level of human presence. Be deliberate about where you draw the line.
Stop applying the same feedback approach to every task. Start mapping your feedback strategy to a simple framework: AI-only for low-stakes, structured, non-identity-relevant tasks. Human presence (or human-endorsed AI) for high-stakes, emotionally significant, identity-relevant work. Capstones, professional placements, moments where learners are struggling or doubting themselves — these are where your presence matters most and is least replaceable.
5. Design for active processing, not just delivery
Wisniewski, Zierer, and Hattie (2020) tell us that feedback’s impact depends on how learners process it. Eye-tracking research confirms that learning gains come from active selection and revision planning, not passive receipt (Mao & Benesch, 2023). AI tools make it trivially easy to deliver more feedback — but more isn’t automatically better, and Morris and Maes (2026) suggests learners are more likely to skim AI feedback and pick off easy fixes.
Stop assuming that delivering feedback is the same as learners learning from it. Start building tasks where learners must select the most important AI comments, write a revision plan, or identify where they disagree. Turn AI feedback into a prompt for thinking — not a to-do list.
6. Audit and constrain
Cap the number of feedback points. Ask for prioritisation (”top three improvements”). Structure comments around “where to next?” rather than an exhaustive list of errors. And check AI feedback regularly for accuracy, tone, and bias. Studies in physics and clinical anatomy found that around two-thirds of LLM feedback required only minor editing — but the remaining third needed substantial correction, and that’s in well-structured domains (Zhu et al., 2024; Kääriäinen et al., 2025).
Stop letting AI generate unconstrained feedback that overwhelms learners. Start designing prompt templates that enforce pedagogical structure and limit volume.
Concluding Thoughts
The question that dominates the AI-in-education conversation right now is: can students tell the difference between AI and human feedback? The answer is often no, and that feels like a win.
What strikes me is that the work of Morris and Maes reminds us that we are actually asking the wrong question. Their learners weren’t asked to detect anything. They were told the source, and knowing that their feedback was AI-generated changed not what they thought about the feedback, but what they felt about it — and what they felt changed what they did.
This isn’t a surprising finding if you know the online learning literature. Twenty years of Community of Inquiry research has told us that visible instructor presence, social connection, being seen as an individual, belonging, and human support for self-regulation are what drive engagement, persistence, and deep learning (Richardson et al., 2017; Salihu & Alemu, 2024; Shea et al., 2013). AI feedback — however well-written — can’t trigger those mechanisms if learners know there’s no human behind it.
And here’s the uncomfortable part: this isn’t a new trap. We built exactly the same one with asynchronous e-learning — scalable, efficient, text-heavy environments that depressed engagement even when the content was excellent. The Community of Inquiry framework exists because we already learned this lesson once. AI feedback reproduces the same conditions: disembodied, one-way, functional rather than relational. The technology is new. The mistake — assuming that good content delivered efficiently is enough — is not.
The temptation right now is enormous: AI is so good at generating feedback that it’s natural to ask “why wouldn’t we automate all of this?” Morris and Maes (2026) gives us the answer: we can build technically excellent AI learning support — and learners may still not engage with it as deeply as they would with a human who’s paying attention.
Technical success is not the same as pedagogical success.
The disembodiment trap isn’t inevitable. But avoiding it means we have to stop optimising only for what we need — efficiency, scale, productivity — and pay equal attention to what learners need: to feel seen, to feel accountable to someone real, to feel that their effort matters to a person, and to feel held when the work gets hard.
Those aren’t soft preferences about “nice” learning experiences. They’re the conditions under which feedback — AI-generated or not — actually changes what learners do.
Happy [selective] innovating,
Phil 👋
PS: Want to learn how to minimise the risk and optimise the value of AI with me? Check out my AI & Learning Design Bootcamp.
