AI is making you faster AND slower, better AND worse at your job
What new data tells us about the complex impact of AI on L&D (mid-2026)
Hey folks 👋
If you work in L&D, you probably already know that strange feeling: your team is using AI and producing more than ever — faster drafts, faster modules, faster everything — and yet overall it feels like AI hasn’t had much impact. More gets made, but about the same amount of work actually lands.
Welcome to the messy reality of AI’s impact on our work.
The AI debate is usually black and white — it’s either saving us or ruining us. The reality is less convenient: AI’s impact on our work and our roles isn’t good or bad, it’s both, at the same time. A new MIT and Wharton study of more than 100,000 software developers (Demirer, Musolff & Yang, 2026) — the most rigorous look yet at AI’s effect across the whole chain of how work gets made — lets us put numbers on that contradiction for the first time.
This is coding data, not L&D data, so some of what follows I’m reading across by analogy (and backing with the smaller, emerging L&D research where it exists) — but the pattern is striking, and parallel data shows the same pattern emerging in L&D.
Here’s the TLDR:
AI makes us dramatically faster at producing stuff — think up to a +740% jump in production task velocity.
Extra volume creates a new bottleneck — a backlog of human reviewing, integrating and validating.
Overall, AI makes us a little more productive — but far less than headline speed gains suggests, because downstream bottlenecks absorb most of the gain before it converts into anything real.
Also, critically, AI tends to drag quality down — because generic AI is brilliant at producing the tidy, structured average content which is weak on depth, context, and judgement.
At the same time — used well, AI can make the work genuinely better, not just faster. Aimed at the right part of the job, it lifts quality rather than dragging it down. But that’s a deliberate choice, and it’s the opposite of what most of us are doing with it today.
Five effects, pulling in different directions, all at once. We’ve been loudly celebrating (and selling) the first, quietly ignoring (but feeling) the middle three — and almost entirely missing the fifth, the one that could actually redeem the whole thing.
In this week’s post, I’ll walk through what the research actually shows, what it means for L&D specifically, and where the evidence is solid versus where I’m reading across from adjacent fields.
I’ll also share my take on what we should actually do about all of this. Spoiler: the answer isn’t “use less AI.” It’s to stop aiming AI at the one thing we can already measure, and start using it on the parts we’re currently missing.
Let’s go!
The Wharton MIT Study
Here’s how that study works, because the detail matters here. Demirer, Musolff and Yang (2026) tracked 100,000-plus GitHub developers as they adopted three generations of AI coding tools — autocomplete, then “sync” agents that work alongside you, then “async” agents that run on their own — and measured what happened at every stage of how software gets made, from the first line of code to the finished product people use.
What they found is more interesting than the usual “AI makes you faster” headline. AI’s impact wasn’t one thing — it was several things at once, pulling in different directions.
There are four flavours of impact in the data, plus a fifth we get to choose. The coding research has now measured the first four — and the same pattern is emerging underneath L&D right now. Here are the four in one table, before I take you through each (the fifth, “better,” I’ll come to after):
Let me take you through each, with what the wider evidence shows, and what it means for those of us in L&D.
Impact Type #1: AI Makes “Creation” Work Faster
This is the one everybody talks about, and it’s real.
In the coding study, turning on AI drove enormous gains at the task of producing code: autocomplete raised lines of code by around 228%; the newer agents pushed it past 740% (Demirer, Musolff & Yang, 2026). This isn’t a rounding error. It’s the biggest measured task-level productivity effect of any workplace technology in recent memory, and it shows up across every tool they studied.
The wider picture agrees. A P&G field experiment with NBER (Dell’Acqua et al., 2026) found that one person working with AI performed about as well as a two-person team without it — AI didn’t just speed people up, it collapsed a whole second role into one.
In L&D, the same acceleration is everywhere. LinkedIn’s workforce data has 70% of skills changing by 2030 (LinkedIn, 2025), and L&D teams are reaching for AI just to keep pace. In industry surveys, L&D professionals now routinely report that generative AI can automate or accelerate the bulk — often “up to 80%” — of content-creation tasks, from drafting modules to building quizzes and copy.
TLDR: we are, unquestionably, faster. But that’s only a small part of the story…
Impact Type #2: We “Ship” Only 20% More
Here’s where the data gets interesting: all of that extra production velocity barely moves the needle on the thing that actually matters: finished output that reaches people.
The coding study followed the gains all the way up the chain — lines of code, to files, to commits, to pull requests, to projects, to shipped releases — and watched them drain away at every step. The 740% surge in code-produced became a 65% rise in review requests and — overall — just a ~20% rise in actual releases (Demirer, Musolff & Yang, 2026).
The productivity gain doesn’t vanish — but it gets absorbed, layer by layer, by every human-dependent stage between “produced something” and “shipped something people use.” Reviewing it, integrating it, deciding it’s good enough.
The wider picture is starker still. MIT’s GenAI Divide survey (MIT Project NANDA, 2025) found that around 95% of organisations are seeing no meaningful measurable return from generative AI — only about 5% are generating real business impact. RAND (2025) puts the historical AI project failure rate above 80%, twice the rate of comparable non-AI IT projects. In nearly all of these, the technology worked fine. The failure was downstream — organisations couldn’t convert what the tools produced into anything that shipped and counted.
In L&D, this is the gap between “we made a lot of content” and “anyone’s behaviour changed.” Most corporate and online programmes still hover around 20–30% completion (industry e-learning benchmarks) — and completion itself is closer to “app downloaded” than “app used a month later.” We are producing learning faster than ever and shipping real capability at roughly the rate we always did. The needle that matters barely moves.
Impact Type #3: AI Increases Complexity
This is the impact that almost nobody names, and it’s the mechanism that explains the relatively low increase in productivity versus creation metrics. The reason the gains drain away isn’t bad luck — it’s that fast production creates more downstream mess, and a new sort of bottleneck.
The coding research is explicit about this. A study by He et al. (2026), for example, is titled, “Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity.” AI generates more, and more verbose, code; the volume has to be reviewed, integrated, and untangled by humans whose capacity hasn’t changed. The upstream got cheaper; the downstream got heavier. That’s the trade, and it’s why “more output” and “more shipped” aren’t the same number.
The wider picture has a name for where this complexity bites: the foundation. Gartner (2025) expects organisations to abandon around 60% of AI projects that aren’t supported by AI-ready data. The model is almost never the problem — the problem is that what AI produces fast has nowhere solid to land.
In L&D, the complexity is concrete. A tool can draft a hundred modules, but if your content isn’t modular, tagged, and governed, there’s nothing for anything agentic to stand on — you’ve multiplied the assets and multiplied the mess, and the cleanup lands on exactly the same human bottlenecks (stakeholders, reviewers, line managers) that already decide whether behaviour ever changes.
Impact Type #4: AI Lowers Quality
This is the flavour of AI’s impact that we least want to look at, but feel perhaps most. When AI accelerates the cheap, upstream part of the work, the average quality of what ships tends to fall — not rise or even hold.
The coding study can only infer this; it measures counts and downloads, and says so plainly (Demirer, Musolff & Yang, 2026). But the surrounding research is pointed. Reimers and Waldfogel (2026) found that LLMs roughly tripled new book releases on Amazon between 2022 and 2025 — while average quality declined.
More content produced faster, but of worse quality. The coding study’s own marketplace data echoes the same pattern: new apps surged across four major stores, total usage stayed flat, and the share of new apps that fail to reach even a modest audience went up (Demirer, Musolff & Yang, 2026). The marginal output isn’t just unshipped — increasingly, it’s invisible.
In L&D, we do not yet have a single, clean, field‑wide causal study that quantifies how AI changes the quality of shipped learning experiences – that is an honest gap. What we do have, though, is a growing body of work that treats quality as a live risk rather than an automatic dividend. SWOT and case‑study analyses of ChatGPT‑supported course design consistently find that while generative AI is highly efficient at outlines, course maps and draft content, its unedited output is generic, error‑prone and context‑insensitive, and still depends on substantial domain expertise and instructional‑design skill to reach an acceptable standard (Various, 2024; AACE Review, 2026).
Taken together with the evidence from books, science and software, the sober prior is clear: when AI makes the easy part of creative and knowledge work radically cheaper, volume explodes, while average quality drifts down unless we deliberately build new filters, incentives and forms of human editorial effort around it
Impact Type #5: AI Can Make Our Work Better
The title of this piece promises “better” as well as “worse,” and I owe you that half, because it’s real — and it’s the most hopeful thing in this post.
In controlled settings, with a scaffolded, domain‑specific assistant, AI can genuinely improve instructional‑design quality and designer capability, especially when it is embedded in structured workflows, aligned with explicit pedagogical criteria, and used as a co‑creative partner rather than a generic idea pump.
For example, multi‑agent systems such as EduPlanner use specialised evaluator and optimiser agents plus domain‑specific rubrics (e.g. clarity, integrity, depth, practicality, pertinence) to iteratively generate and refine mathematics lesson plans, producing higher‑quality designs than single‑model baselines on these dimensions (Zhang et al., 2025).
Similarly, human‑centred frameworks for AI‑assisted instructional design, like ARCHED (2025) show that when educators remain primary decision‑makers and AI support is tightly coupled to pedagogical goals and transparency, overall design quality improves without undermining professional judgement. Knowledge‑enhanced approaches that guide GPT‑4 through stepwise lesson‑plan generation and self‑critique against human‑defined criteria also report gains in lesson‑plan coherence and pedagogical soundness relative to unguided use (Zheng et al, 2025).
In higher‑education and teacher‑education settings, AI‑supported, co‑creative ID approaches have been found to help novice designers and preservice teachers produce clearer, more engaging materials, while also building their understanding of design principles when the AI is integrated into structured design tasks and analytics‑driven feedback cycles (Egloffstein, 2023; Moore, 2025).
Beyond plan quality, generative‑AI‑powered instructional models in advanced mathematics have demonstrated substantial gains in learning outcomes—such as reduced time to concept mastery and larger improvements in conceptual understanding—when AI‑generated content and visualisations are systematically embedded in an instructional model rather than used ad hoc (Xie et al, 2026).
Together, these studies support the claim that when AI is carefully scaffolded within domain‑specific, pedagogy‑aligned workflows, it can both lift the quality of instructional products and extend what designers are able to do
But here’s the catch: “better” and “worse” don’t come from using different tools. They’re the same tool, used two different ways. Point AI at the cheap upstream task — crank out more modules — and you get flavour 4: tidy, generic, common-not-best practice. Point the same AI at diagnosis, feedback, reinforcement, and judgement — and quality goes up. The technology doesn’t decide which one you get. You do.
The Real Story: a Complex Web of Impact
AI is making production a lot faster. The result is that those of us using AI for production tasks end up a little more productive than before — but nowhere near what the speed-up promised.
But that positive impact on productivity is being eroded by increasing complexity. What gets more complex isn’t the production — that’s the easy bit now — it’s everything downstream of it. When AI multiplies the volume of what you make, it multiplies the amount that has to be reviewed, checked for accuracy, tagged, integrated into existing systems, and signed off.
When AI becomes part of the picture, the bottleneck shifts: it’s no longer “can we make the content?” but “can our people keep up with reviewing, integrating, and governing everything the machine just produced?” Capacity, not content, becomes the primary constraint — and that capacity is human, and it hasn’t grown.
Perhaps most importantly, the net effect of AI is to drag average quality down. This happens because generic AI is brilliant at producing the structured, plausible average — it confidently reproduces the most common version of whatever you ask for — and weak on the depth, context, and judgement that separate good work from merely competent work. So you don’t get more bad content exactly; you get a flood of technically-fine, faintly hollow content that reproduces common practice rather than best practice. And because there’s so much more of it to review, the human checks that might once have caught the shallowness get overwhelmed — the very complexity from the previous point makes the quality problem harder to catch.
The good news is that none of this is fixed. Used well, AI can make the work genuinely better — not just faster. The same research that warns about generic tools shows that when AI is scaffolded, pointed at a specific problem, and aimed high up the chain — at diagnosis, feedback, coaching, reinforcement — it lifts quality rather than dragging it down. The catch is that “better” and “worse” aren’t different tools; they’re the same tool used two different ways. Which one you get is a choice, and it’s the opposite of the choice most of us are making by default.
All of these are true at the same time, and the first four are causally linked: we ship only a little more because what we produce fast arrives more complex and lower in quality. Those two effects — complexity and quality drag — load exactly onto the human review, integration, and adoption stages that were already the weak link. The fifth, “better,” is the one we choose: it only shows up when we aim AI at the right part of the work.
The trouble is that the public conversation — in software and in L&D — is almost entirely about flavour 1. We measure speed because speed is easy to see. The other three are where the value actually leaks, and they’re the three we don’t instrument.
It would be easy to read all this as “AI doesn’t work for L&D.” That’s the wrong lesson, and the coding study points to the right one. As the tools got more capable, the bottleneck moved up the chain — autocomplete only touched the very bottom, but the newer agents reach higher. And the paper’s most useful finding for us is this: a modest tool that enters high in the production chain beats a powerful tool that enters low, because its gains have fewer human-dependent stages left to drain through (Demirer, Musolff & Yang, 2026).
For L&D that reframes the whole question. The move is not “generate more content, faster” — that’s pouring water into the widest part of a funnel that narrows to nothing. The move is to point AI at the higher layers, the ones nearer to behaviour change. Which, concretely, looks like this.
This is, more or less, the Three Ds written up as a production function. Data and Doing — generating and assembling — is the cheap, substitutable bottom of the stack where AI delivers its 700%. Deciding is the human layer that governs whether any of it converts into something worth shipping.
A Practical Response
If the research and our own experience on the ground tells us anything, it’s that in order to feel the benefits of AI we must build the downstream capacity that turns fast production into something that actually ships and sticks.
Here are five strategies to set you on this path, roughly in order of leverage:
Stop optimising for speed. Every vendor pitch and every instinct pushes you to make content generation faster. That’s the one lever the evidence says doesn’t convert. Treat “we can make modules quicker” as solved, and move your attention — and your budget — everywhere else.
Point AI up the chain, not down it. The single most useful finding here is that a modest tool entering high in the chain beats a powerful one entering low. So the investments that matter aren’t better generators; they’re AI that works at diagnosis (is this even a training problem?), at feedback and coaching in the flow of work, and at reinforcement after the event. A mediocre coach that nudges application beats a brilliant tool that drafts more modules nobody finishes. This is also the only one of these moves that doesn’t just stop the downside — it’s how you get the fifth effect, genuinely better work, rather than just less waste.
Fix the foundation before you scale the generation. Fast output has nowhere to land without modular, tagged, governed content. The unglamorous work — content architecture, metadata, a real quality rubric, named ownership — is the precondition for anything agentic. Generating faster on top of a messy base just multiplies the mess.
Protect the human review layer — and stop anthropomorphising it away. The quietest finding in the research is that “AI buddy / coach / mentor” framing makes reviewers measurably sloppier (Kropp et al., 2026). So drop the anthropomorphic language internally, frame AI as a tool that requires human accountability, and name explicitly who owns the quality check on AI-generated learning. Build review in; don’t assume polished output is finished output.
Build the instrumentation you don’t have. You can’t manage attenuation you can’t see, and right now most of us can’t see it. That means measuring past completion — application, behaviour change, some business signal — even crudely. Until L&D builds the equivalent of the developer’s dashboard, most of what matters stays invisible, and you’re guessing about whether AI is helping or quietly hurting.
The TLDR is this: our goal when working with AI shouldn’t be to produce more, faster. Instead, it must be to build the downstream capacity — foundation, review, measurement, workflow — so that we can convert what AI produces into something that actually changes behaviour. That’s where the value leaks, and where it also gets recovered.
Closing Thoughts
For a couple of years now, the L&D conversation about AI has been stuck on a single question: how much faster can it make us? This research is an invitation to ask a better one.
Because speed was never our problem. We were never short of content — we were short of content that lands, that’s good enough to matter, that actually changes what someone does on a Tuesday morning. AI has made the thing we were already good at — producing — spectacularly cheaper, and left the things we’ve always struggled with — judgement, review, reinforcement, proof — exactly where they were. That’s why the strange feeling I opened with is so common: the dashboard says we’re flying, and the destination hasn’t moved.
The uncomfortable truth in all five effects is that AI doesn’t have an impact on its own. It amplifies wherever you point it. Point it at the content factory and it floods you with fast, generic, average work that buries your reviewers and quietly lowers the bar. Point it at diagnosis, feedback, and judgement — the deciding layer that was always the real job — and the same tool makes the work genuinely better. The technology is identical. The outcome is a choice, and right now most of us are making it by default rather than on purpose.
So if you take one thing from this, let it be this: the teams who win with AI over the next few years won’t be the ones who produce the most. They’ll be the ones who got honest about where the value actually leaks, built the boring downstream capacity to catch it, and aimed their best tools at the part of the work that only humans used to be able to do.
That’s harder than generating another hundred modules, but it’s also why we are here.
Happy shipping (not just building!),
Phil 👋
PS: If you want to explore how to get the most from AI in your day to day work, check out my AI Bootcamp for L&D.
This is such an important distinction: AI doesn’t automatically create value. It amplifies the part of the system you point it at.
If we point it at content production but that doesn't mean learning improves.
The real opportunity for L&D is not to use AI to produce more learning assets. It is to use AI to strengthen the parts of learning that have always been hardest to scale: diagnosis, feedback, practice, coaching, application, and measurement.