The unreasonable effectiveness of AI
The unreasonable effectiveness of AI
摘要: The unreasonable effectiveness of AI
The 6 big milestones of the last decade that made the impossible, possible.
Ilias Antonopoulos
Aug 24, 2026
正文
The unreasonable effectiveness of AI
The 6 big milestones of the last decade that made the impossible, possible.
Ilias Antonopoulos
Aug 24, 2026
It recently dawned on me, as I was typing away at one of my Claude Code sessions:
I haven’t written roughly 99% of the code I’ve produced this year.
The little code I did write manually this year was mostly for my own understanding of specific implementation details - like analysing certain aspects of some dataset features. Maybe some SQL queries here and there as well.
How did we reach this point?
Back in 2018, when I started working in the field, it would usually take us weeks to properly run a serious end-to-end machine learning modelling experiment. A few years later, when we were experimenting with neural network architectures, we would strain our eyes for hours to properly compile a model by reading the framework’s docs and searching online.
With these thoughts in mind, I wanted to write this article as a sort-of-homage to the last decade.
From 2017 until today - summer of 2026 - one can easily highlight six pivotal moments in AI research and development
.
It all starts with
attention
.
Thanks for reading The Artisan ML Substack! Subscribe for free to receive new posts and support my work.
Subscribe
Milestone #1: Attention Is All You Need
By the mid-2010s, the best language models “understood” sentences:
one word at a time.
left to right.
with everything held in memory.
It was a pretty reasonable model architecture, especially for solving the most challenging problem of the time:
translation
. The model would read the sentence word by word, squeeze the whole thing into a fixed amount of memory - a few thousand numbers - and then discard the sentence and produce the translation from that summary alone.
But this model architecture had a very significant drawback: although it worked fine for translating short sentences, it doesn’t work well for longer sentences. The problem is that t