Registro Nornic · Documento vivo
“AI makes experienced developers 19% slower.”
A real result from sixteen developers, still circulating after its own authors withdrew the larger follow-up as unreliable.
La fuente
- Estudio
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
- Editor
- METR
- Tipo de publicación
- Randomised controlled trial, published research
- Muestra
- 16 developers, 246 tasks
- Trabajo de campo
- February – June 2025
- Financiación y conflictos
- METR — a non-profit AI evaluation organisation
- Verificado el
- 2026-07-26
La aritmética
The study is careful and its authors are unusually candid about its limits. The problem is not the study; it is the citation.
n = 16. That supports a directional observation about a specific population — experienced open-source maintainers working in repositories they already know well. It does not support a general claim about developers.
In February 2026 METR abandoned its own 57-developer follow-up, stating it was unreliable due to selection bias. The team noted the raw data pointed toward a speedup and wrote that the true effect "could be much higher".
Almost nobody citing the 19% figure cites that correction. The original number now travels without the authors' own revision attached to it.
What it does support: that experienced developers can be slower than they expect on familiar code, and that self-reported speedup is unreliable. That is a genuinely useful finding, and a smaller one.
Qué cambiaría este veredicto
A replication at meaningful scale, with the selection-bias problem addressed, reporting a negative effect. METR's own withdrawn follow-up suggests the opposite is more likely.
Fuentes