8.17.2026

Four Convergences: A History of Modern AI

Written by: Claude Sonnet 5.0 AI.

Curator/Editor: Học Trò.

In the summer of 2012, a Canadian professor and two of his graduate students uploaded a neural network to a computer-vision competition nobody expected them to win, and blew the result apart by a margin that stunned people who had spent their careers in the field. That single afternoon is as good a place as any to start this story — except that it isn't really a beginning. It's a collision. Three separate bets, made independently by three people who mostly didn't know each other, had each been running for years — one of them for decades — before they crashed into one another in a Toronto lab's entry to a leaderboard. Everything that follows in this essay, from ChatGPT's quiet Tuesday-night launch to Jensen Huang selling graphics cards to gamers who had no idea they were funding the future, is the story of what happened after that collision, and of the handful of people who kept showing up at the intersection of it.

The Thesis, Stated Plainly

Modern AI did not arrive as one invention. It is the story of four scarce things — an idea, a dataset, a kind of computer chip, and eventually a training recipe for language itself — each stranded without the others for years or decades, until they converged in the hands of a surprisingly small, densely interconnected group of people who kept crossing paths at the same handful of institutions: the University of Toronto, Stanford, Google, and a nonprofit research lab that would eventually split into two of the most valuable companies in the world.

The idea was neural networks and backpropagation, kept alive against the field's own prevailing skepticism by Geoffrey Hinton and a small circle of believers for the better part of thirty years. The dataset was ImageNet, Fei-Fei Li's unglamorous, initially unfashionable bet that computer vision's real bottleneck was the absence of data at the scale the visual world actually requires, not a missing clever algorithm. The chip was the graphics processing unit, and specifically Jensen Huang's decade-early wager that NVIDIA's gaming hardware could be turned into a general-purpose scientific computer years before anyone needed it to be one. And the recipe, arriving five years after the first three converged, was a single 2017 paper out of Google's research group proposing that attention — the mechanism by which one word in a sentence decides which other words matter to it — was, on its own, sufficient to build the most powerful language models anyone had made. Four convergences. This essay walks through all four, and through the people who built each one and then kept building on top of what the others had built.

I. Cold Winter, True Believers

Long before there was a dataset large enough or a chip fast enough, there had to be an idea worth waiting for, and for most of the second half of the twentieth century, that idea was out of fashion. Artificial neural networks — loose, simplified models of how neurons in a brain might pass signals to one another — had a brief vogue in the 1950s and then fell into a period the field itself came to call an "AI winter," when funding dried up and most serious computer scientists moved on to other approaches. Geoffrey Hinton did not.

Hinton's own path into the subject was unusual even by academic standards. Born in 1947 in Wimbledon, into a family stacked with mathematicians and unconventional thinkers — his great-granduncle was George Everest, for whom the mountain is named; an earlier relative coined the word "tesseract" — he studied at Cambridge, alternating between physics, philosophy, and physiology before settling on experimental psychology, then spent a year training as a carpenter before returning to academia for a PhD in artificial intelligence at Edinburgh in 1978, with a dissertation on neural networks written at a moment the field considered them close to a dead end. He has been at the University of Toronto since 1987, and it was there, across the 1980s, 1990s, and 2000s, that he did the work that eventually made him one of the three researchers the press would later dub the "godfathers of deep learning" — a label attached to Hinton alongside two contemporaries, Yann LeCun and Yoshua Bengio, who arrived at related ideas independently rather than as his students, a distinction worth keeping straight since Hinton's actual students matter enormously to the rest of this story in a much more direct way.

Two pieces of Hinton's own research turn out to be load-bearing for everything that follows. The first is backpropagation — technically not invented from nothing by Hinton, since related mathematical ideas had circulated earlier in control theory, but brought to wide scientific attention by his 1986 Nature paper with David Rumelhart and Ronald Williams, "Learning representations by back-propagating errors." Backpropagation is the algorithm that lets a multi-layer neural network figure out, after making a wrong guess, exactly how much each of its internal connections contributed to the error, and adjust each one accordingly. It is, in a real sense, still the algorithm nearly every neural network trains on today, including every large language model discussed later in this essay. The second is the Boltzmann machine, which Hinton built with Terrence Sejnowski in the early 1980s, borrowing tools from statistical physics to construct a network that could learn to recognize patterns in data by settling into low-energy states — work so foundational that it would, decades later, win Hinton a Nobel Prize in a discipline he had never formally practiced.

But for most of the two decades after 1986, none of this mattered very much to anyone outside a small circle, because neural networks kept losing to simpler, more established methods on the problems anyone cared about, and the field's collective attention drifted elsewhere. Hinton kept working anyway. In 2006, with Simon Osindero and Yee-Whye Teh, he published "A fast learning algorithm for deep belief nets," which showed how to pretrain a deep network — one with many layers — one layer at a time, using stacked Boltzmann machines, giving the whole system a far better starting point before final training. It is one of the key technical moves that quietly revived interest in deep learning years before the public ever noticed, and 2006 is sometimes cited, half in jest, as the year the revival actually began — coincidentally, and not coincidentally, the same year a chip company most people knew only for gaming hardware made its own quiet, unrelated bet on the future.

II. The Data Nobody Wanted to Label

If Hinton's problem was that his idea had no fashionable home, Fei-Fei Li's problem was the opposite: she had identified exactly what the field needed, and almost nobody thought it was worth the years it would take to build.

Li's own path to that insight ran through circumstances about as far from an elite research lab as it is possible to get. Born in Beijing in 1976, she immigrated to the United States at fifteen, arriving in Parsippany, New Jersey with her parents, little English, and little money — her father had gone ahead three years earlier to find work while the family waited. She worked in a Chinese restaurant for two dollars an hour, and while an undergraduate physics major at Princeton on a near-full scholarship, she borrowed money from friends and even from her old high-school math teacher to buy her parents a dry-cleaning shop, then personally ran it — answering the phones, handling billing, dealing with inspections, all in English her parents couldn't manage themselves — for seven years, from age eighteen through the middle of her later graduate studies. She has said in interviews and in her own memoir that this period is not a detour from her later "human-centered AI" advocacy but its direct source: an immigrant family's survival was never an abstraction to her, and she has argued throughout her career that AI development owes the same accountability to ordinary lives that her own family's shop once depended on.

The technical insight came later, on the Princeton faculty in 2007. By the mid-2000s, most computer-vision research was focused on refining algorithms against small, curated image datasets — the conventional wisdom held that the field's bottleneck was a missing clever technique. Li's contrarian bet, unfashionable enough at the time that colleagues reportedly questioned why she was spending years on "just" a dataset, was that the real bottleneck was data at a scale nobody had attempted: not thousands of labeled images, but millions. Working with her team, she organized images against the WordNet lexical hierarchy and used Amazon Mechanical Turk — itself a novel choice for computer-vision research — to crowdsource human verification of labels at a scale that eventually reached more than fourteen million images across more than twenty thousand categories. The dataset, ImageNet, was first presented at a computer vision conference in Miami in 2009. The following year, Li and her team launched the ImageNet Large Scale Visual Recognition Challenge, an annual competition explicitly designed, in her own framing, to "democratize" the use of large labeled datasets and give the field a shared, rigorous benchmark to compete against.

For its first two years, the competition validated nobody's suspicion that anything unusual was about to happen. No deep-learning models were entered in 2010 or 2011; the best top-5 error rate — the percentage of images a system failed to classify correctly even when given five guesses — sat around 25.8%, achieved by conventional, non-neural computer-vision techniques. The dataset was doing exactly what it was built to do: providing a fair, consistent arena. It just hadn't yet met the algorithm that could actually use it.

Li's own account of why she cared about the arena as much as the dataset carries forward into everything she did afterward, and it is worth following that thread here rather than returning to it later, since it never really breaks from her ImageNet work — it simply widens. In 2015 she co-founded AI4ALL with fellow researcher Olga Russakovsky, a nonprofit aimed at building pathways into AI for people the field's talent pipeline routinely overlooked, growing out of a Stanford summer program for high-school girls. In 2017 she took a sabbatical from Stanford to become Vice President and Chief Scientist of AI/ML at Google Cloud, and in June 2018 — by then back at Stanford, where she had co-founded the Human-Centered AI Institute the following year — she testified before the U.S. House of Representatives at a hearing titled "Artificial Intelligence — With Great Power Comes Great Responsibility," pressing the case that AI development needed to be accountable to ordinary people's lives, not judged on capability alone. She has said directly, in her 2023 memoir The Worlds I See, that this is not a separate commitment sitting alongside her technical career but the same instinct that ran the dry-cleaning shop's phones: "I believe our civilization stands on the cusp of a technological revolution with the power to reshape life as we know it. To ignore the millennia of human struggle that serves as our society's foundation, however — to merely 'disrupt'... would be an intolerable mistake." The essay's final section returns to her once more, in a considerably different register, when the same instinct for identifying an under-built foundation resurfaces a second time, nearly two decades after ImageNet.

III. The Bet Nobody Understood Yet

While Hinton's idea waited in Toronto and Li's dataset waited in Princeton, a third and entirely unrelated bet was quietly compounding in Santa Clara, California, made by a man who had never worked in AI research and, for most of the 2000s, had no obvious reason to care about it.

Jensen Huang was born in 1963 in Taipei and emigrated to the United States at nine — he and his older brother were sent ahead of their parents and, through what turned out to be an address mix-up, were enrolled for a period at a boarding school for troubled teens in rural Kentucky rather than the school their parents intended. He became a competitive table-tennis player as a teenager, earned engineering degrees from Oregon State and Stanford, and worked as a chip designer before co-founding NVIDIA in 1993 with Chris Malachowsky and Curtis Priem, reportedly at a Denny's in East San Jose, with the straightforward goal of bringing 3D graphics to gaming and multimedia. The company's first chip, the NV1, was a commercial failure in 1995; its second real attempt, the RIVA 128, was built almost entirely in simulation after the first failure and saved the company. In 1999, NVIDIA launched the GeForce 256, which it marketed — not entirely modestly — as the world's first "GPU," a graphics processing unit distinct from the general-purpose CPU at a computer's core.

The decision that matters most for this essay came in 2006, and at the time looked considerably less inevitable than it does in hindsight. NVIDIA introduced CUDA, a platform for general-purpose computing on its graphics chips — the bet that a GPU's particular strength, which is doing the same simple calculation across thousands of pixels simultaneously, could be redirected toward any computational problem with that same "do-the-same-thing-many-times-in-parallel" shape, not just rendering video game graphics. There was, in 2006, no market for this. Huang has since described CUDA and the earlier RIVA 128 recovery using the same phrase — "bet the company" — while also insisting, in a 2023 interview for NVIDIA's thirtieth anniversary, that the bets were de-risked through simulation and emulation as much as possible beforehand rather than made blind. Either way, it took roughly a decade of sustained investment, widely doubted the whole time, before anyone found a use for CUDA that justified it. As it happened, the group that would eventually justify it was working two thousand miles away, entirely unaware that a graphics-card company had spent years quietly building the exact tool their idea needed.

IV. AlexNet: Where the Three Lines Cross

By 2012, the ImageNet Challenge had run for two years without a deep-learning entrant, CUDA had been available for six years without a widely recognized AI application, and Hinton's backpropagation-and-deep-belief-nets research program had spent a quarter century mostly outside the mainstream of computer vision. Then Alex Krizhevsky and Ilya Sutskever, two of Hinton's graduate students at the University of Toronto, entered the competition with a deep convolutional neural network — trained, critically, on NVIDIA GPUs, because a GPU's capacity for massively parallel arithmetic turned out to be exactly what training a large neural network on millions of images required, and because CUDA had made programming that hardware for a non-graphics task tractable for the first time.

The system, trained on a pair of NVIDIA GTX 580 consumer graphics cards — gaming hardware, repurposed through CUDA for a task NVIDIA had never designed it to perform — and soon known as AlexNet, posted a top-5 error rate of 15.3%, a full eleven points below the second-place entry's 26.2% — an improvement so large, over a benchmark that had moved only incrementally in prior years, that it is routinely described as having genuinely shocked the computer-vision field. Hinton himself has said in later interviews that the scale of the win surprised even him. Within roughly a year, deep learning went from a minority interest inside computer vision to the dominant approach nearly everyone in the field was racing to adopt.

It is worth being precise about what actually happened in that single event, because it is the hinge the rest of this essay's first half turns on. Fei-Fei Li's ImageNet supplied the scale of labeled data a deep network needed to generalize rather than merely memorize. Geoffrey Hinton's decades of unfashionable research supplied the algorithm — backpropagation, refined by his own students' architectural choices — capable of learning from that data. Jensen Huang's decade-early, widely doubted CUDA bet supplied the only hardware fast enough to make training that network computationally feasible in a reasonable amount of time. None of these three people built AlexNet expecting to meet the other two on a leaderboard. Each had simply kept building the one piece they controlled, for reasons that had almost nothing to do with each other, until the pieces fit.

Google noticed immediately. Within months, Hinton, Krizhevsky, and Sutskever incorporated a shell company, DNNresearch, around their result; Google acquired it for roughly $44 million in early 2013, and Hinton became a part-time Google employee, dividing his time between Toronto and Mountain View for the next decade.

V. The Talent Land-Grab

The years immediately after AlexNet were a scramble, as every major technology company raced to acquire the small number of researchers who understood how to build and train large neural networks — and as a second, entirely separate lineage of AI research, with almost no overlap with the Hinton/Fei-Fei Li line, was simultaneously proving itself in a very different arena.

That second lineage belonged to Demis Hassabis, whose path into AI looked nothing like either of the researchers already covered in this essay. Born in London in 1976, Hassabis was a competitive chess player from age four and a chess master by thirteen; at seventeen he designed and programmed Theme Park, a multimillion-selling commercial video game, before earning a Double First in computer science at Cambridge and founding his own games studio. He then pursued a PhD in cognitive neuroscience at University College London, explicitly to understand how the human brain manages imagination and memory, followed by postdoctoral work at Harvard and MIT — a games-and-neuroscience combination that led him, in 2010, to co-found DeepMind on the premise that studying biological intelligence directly would produce better artificial intelligence, a founding rationale genuinely distinct from the statistics-and-scale arguments driving the Hinton/Fei-Fei Li line of work.

Google acquired DeepMind in January 2014 for roughly $500 million, at the time its largest European acquisition, and the payoff arrived spectacularly in March 2016, when DeepMind's AlphaGo faced the world Go champion Lee Sedol in a five-game match broadcast live to an estimated 60 million viewers. AlphaGo won 4–1. The match is remembered above all for a single play — "Move 37" in Game 2 — a placement human commentators initially dismissed as a mistake before recognizing it as a piece of genuine creative insight no human professional would likely have found; AlphaGo had not simply memorized human games, it had improved by playing millions of games against itself and discovering strategies outside existing human experience. For a public that had not yet heard of GPT or ChatGPT, AlphaGo was the moment "AI arrives" — six and a half years before a chatbot would do the same job for a much wider audience, and by a very different route.

AlphaGo was not, as it turned out, DeepMind's last collision with a problem the rest of science had spent decades failing to crack. In 2020, Hassabis and DeepMind colleague John Jumper unveiled AlphaFold2, which effectively solved the decades-old "protein folding" problem — predicting a protein's three-dimensional structure directly from its amino-acid sequence. By the time it won the two of them a share of the 2024 Nobel Prize in Chemistry (split three ways with David Baker, honored separately for computational protein design), AlphaFold2 had been used by more than two million researchers across 190 countries and had predicted structures for virtually all two hundred million proteins known to science — a piece of scientific infrastructure Hassabis has since extended directly into drug discovery through Isomorphic Labs, a DeepMind spinout he also leads. The essay's earlier section on Geoffrey Hinton is worth recalling for a moment here, because 2024 handed out an unusually pointed coincidence: the same year Hassabis won a Nobel Prize in Chemistry for teaching a machine to solve one of biology's oldest structural puzzles, Hinton won a Nobel Prize in Physics for the statistical-mechanics tools — the Boltzmann machine chief among them — that helped make machine learning itself possible in the first place. Two Nobel Prizes, in two different sciences, both awarded in the same year to work whose common thread was a neural network learning to find structure in data no human had been able to fully see.

Google's own AI research, meanwhile, was still split across two separate organizations during this period — DeepMind in London, operating with real independence after the acquisition, and Google Brain in Mountain View, the original deep-learning group that had also recruited Hinton part-time and that, as this essay's next section covers, was about to produce the single most consequential research paper of the decade. The two groups would not be formally merged until 2023, and understanding them as separate through this period matters for keeping the essay's cast of characters straight: DeepMind is Hassabis's story; Google Brain, for the next several years, belongs to a different set of researchers entirely.

VI. OpenAI's Founding Bet

In December 2015, a group of researchers and investors — including Elon Musk as co-chair, Sam Altman, and Ilya Sutskever (one of AlexNet's two graduate-student authors, now moving from Toronto's academic lineage into the industry's next chapter) — announced the founding of OpenAI, explicitly framed as a nonprofit counterweight to a small number of large companies, implicitly Google, controlling the frontier of advanced AI research. Sutskever became chief scientist. Among the founding research scientists was Andrej Karpathy, whose own path to that seat had run through the other academic lineage this essay has been tracking: a PhD at Stanford under Fei-Fei Li, completed in 2015 with a dissertation on connecting images and natural language, and — while still a student — the creation and teaching of CS231n, Stanford's first deep-learning course, which grew from 150 to 750 students during his time on it.

Karpathy is, in a real sense, the connective tissue of this entire story more than any single "inventor" is. He trained under Fei-Fei Li on the data side of the field's history; he then worked alongside Sutskever, Hinton's own former student, at the company that would eventually build GPT; years later, as this essay's later sections will show, he would end up at Anthropic, the company Sutskever's OpenAI colleague Dario Amodei left OpenAI to found. Almost everyone named in this essay is one or two degrees from either Hinton or Fei-Fei Li. Karpathy sits at the crossing point of both lines at once.

Sam Altman's own path to OpenAI's leadership had run through a very different track. Born in 1985 in Chicago and raised in St. Louis, he dropped out of Stanford's computer science program after two years — he has said he learned more playing poker with classmates than in lectures — and founded Loopt, a location-sharing app and one of the first companies funded by the newly launched Y Combinator accelerator. Loopt struggled to attract users and was acquired for $43 million in 2012; Altman succeeded Paul Graham as Y Combinator's president in 2014, before co-founding OpenAI the following year. He became CEO in 2019, after Musk's departure over disagreements about the company's strategic direction — a split that, in retrospect, was the first of two major internal ruptures at OpenAI over exactly how fast and how openly the technology should be developed, the second of which, involving Dario Amodei, would produce an entirely new company five years later.

VII. Attention Is All You Need

If AlexNet was the moment three separately-running bets collided to solve vision, the essay's second climax is the moment a single research paper did something structurally similar for language — and it happened inside the other half of Google's split AI operation, the Mountain View-based Google Brain group, in 2017.

In December 2017, eight researchers — Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin — presented a paper at the Conference on Neural Information Processing Systems in Long Beach titled, with a confidence the field would spend the next several years proving justified, "Attention Is All You Need." The paper proposed a new architecture, the Transformer, built entirely around a mechanism called self-attention, and dispensed with two techniques — recurrence and convolution — that nearly every prior state-of-the-art language model had relied on.

The plain-English version of what changed is this. Earlier language models processed a sentence roughly the way a person reads it: word by word, in order, carrying forward a running summary of everything seen so far. This works, but it is slow to train — each step depends on the one before it, so the calculations cannot be spread across many processors at once — and it makes it hard for a model to weigh a word against another word far away in the same sentence, because the "memory" of that earlier word has to survive many intervening steps. Self-attention does something different: it lets every word in a sentence look directly at every other word at the same time, and learn, for each pairing, how much attention it should pay. A model reading the sentence "the trophy didn't fit in the suitcase because it was too big" can, in principle, directly connect "it" to "trophy" rather than "suitcase," not because it stepped through the intervening words one at a time, but because attention lets it check every candidate at once and learn which one actually matters. Because this mechanism does not require processing words in strict sequence, it is dramatically more parallelizable — which mattered enormously for a very practical reason: the same GPUs Jensen Huang had spent a decade turning into general-purpose computers were, by 2017, sitting in data centers by the tens of thousands, and a training method that could actually use all of them at once, rather than waiting on one word at a time, was the difference between a model that took months to train and one that took days.

It is difficult, from inside 2026, to fully recover how quiet the paper's actual reception was. It was one contribution among many at a large annual conference, competing for attention against hundreds of other machine-translation and sequence-modeling papers that December — nothing about its NeurIPS presentation announced that it would go on to become one of the most cited papers in the history of computer science, cited, at last informal count, well over ninety thousand times. Nobody in that room, including its own authors, was in a position to know that the same eight-person team's architecture would, within five years, be running underneath a chatbot capable of holding a fluent conversation with a hundred million people in its first two months on the internet.

The Transformer did not, on its own, produce ChatGPT. What it did was become the substrate underneath essentially every frontier language model that followed — GPT, Claude, Gemini, and the rest — the same way AlexNet's convolutional approach became the substrate under image recognition after 2012. OpenAI's own research direction over the following years was, in large part, the story of what happens when you take the Transformer's decoder half and simply make it bigger: more parameters, more data, more compute, applied with an almost stubborn literalness. That story runs directly into the next section of this essay, and into the career of the man who ran it.

VIII. The Scaling Era and the Amodei Split

Dario Amodei's own route into this story ran through physics and neuroscience before it ever touched AI research. He began his undergraduate studies at Caltech before transferring to Stanford for a physics degree, then earned a PhD in biophysics at Princeton in 2011, studying how groups of neurons fire together in coordinated patterns — a loose structural echo of Demis Hassabis's own neuroscience training, though the two took entirely different routes afterward. After a postdoctoral stint at Stanford's medical school, Amodei joined Baidu in November 2014, working on large-scale speech recognition under Andrew Ng, then moved to Google Brain — the same institutional home, as it happens, that would produce the Transformer paper the following year, though Amodei himself moved on to OpenAI before that paper appeared. At OpenAI he rose quickly to Vice President of Research, setting the company's scientific direction and directly leading the teams that built GPT-2 and GPT-3; he is also credited as a co-inventor of reinforcement learning from human feedback, or RLHF, the technique that later made ChatGPT's responses feel conversational and controllable rather than merely fluent.

GPT-2 and GPT-3 were, in the plainest possible sense, the Transformer architecture scaled up — more layers, more parameters, more training text, more compute — guided in part by research on "scaling laws," the empirical observation that a language model's performance improves in remarkably predictable ways as you increase its size and the amount of data and compute used to train it. This turned model-building, for a period, into something closer to an engineering discipline than a series of individual breakthroughs: if the scaling laws held, you could largely predict how much better a model would get before you had even finished training it, which made enormous, expensive training runs into calculated bets rather than long shots.

It also, within OpenAI, produced a genuine and consequential disagreement about pace. In December 2020, Amodei left OpenAI after nearly five years; the following month, he co-founded Anthropic with his sister Daniela Amodei — who had led safety and policy operations at OpenAI and became Anthropic's president — and six other former OpenAI colleagues, eight people leaving together. Every account of the departure that survives in public reporting describes the same underlying concern: that OpenAI's commercialization was outrunning the safety research needed to keep increasingly capable models controllable, and that a new organization built around safety as a founding principle, rather than a downstream concern, was necessary. Anthropic was structured from the start as a Public Benefit Corporation — a for-profit company with a public-benefit mission written directly into its legal charter — a deliberate structural choice distinct from OpenAI's original 2015 nonprofit-plus-capped-profit-subsidiary model, and one OpenAI would itself largely converge toward, in a different form, a decade later.

Anthropic's own flagship technical contribution followed in 2022: Constitutional AI, a training method in which a model critiques and revises its own outputs against a written, explicit set of principles — a "constitution" — rather than relying purely on case-by-case human judgments the way standard RLHF does. By an accident of timing that is easy to miss, Anthropic actually finished training its first model, Claude 1, in 2022, before ChatGPT's public release that November — meaning the company already had a working large language model in-house at the exact moment its rival was about to make the entire category a household topic. It chose a slower, more limited rollout instead, consistent with the safety-first rationale its founders had left OpenAI to build.

IX. ChatGPT's Big Bang

If AlexNet was deep learning's proof of concept and AlphaGo was the field's first true public spectacle, ChatGPT was the event that made "AI" a subject of conversation at family dinner tables that had never discussed a research paper in their lives.

It launched on November 30, 2022, with no advertising campaign and no press event — Sam Altman's own announcement was a single tweet: "today we launched ChatGPT. Try talking with it here: http://chat.openai.com." In the same thread he added, with a caution the product's reception would soon outrun: "Language interfaces are going to be a big deal, I think. Talk to the computer (voice or text) and get what you want, for increasingly complex definitions of 'want'!... This is an early demo of what's possible (still a lot of limitations — it's very much a research release)." The product itself began less as a grand strategic launch than as an internal demo — reportedly informally called "Chat with GPT-3.5" inside OpenAI — built after the team noticed how much developers enjoyed simply talking to the model through its existing API playground.

The growth that followed made the modesty of the launch almost comic in retrospect: roughly one million users within five days, and by most published estimates around one hundred million monthly active users by January 2023 — a pace that made ChatGPT, according to UBS analysts, the fastest-growing consumer application in history to that point. Instagram had taken about two and a half years to reach a comparable milestone; TikTok took about nine months; ChatGPT did it in roughly two. GPT-3, the underlying model family, had actually been publicly available for over two years by that point — the essay's earlier section on scaling laws covers the research behind it. What made November 2022 different was not a new capability so much as a new interface: a chat window instead of an API console, and a name ordinary people could actually say out loud.

The event reshaped the industry's incentives almost overnight. Microsoft, which had already invested $1 billion in OpenAI's newly created for-profit subsidiary in 2019, added another $10 billion in 2023, bringing its total to roughly $13 billion. The pressure to match or counter ChatGPT's momentum is a large part of why, four months after the launch, Google merged its two separate AI research organizations — DeepMind and Google Brain — into a single unit, Google DeepMind, under Demis Hassabis, explicitly reported at the time as a response to the intensifying race. And within OpenAI itself, the tension between commercial momentum and governance oversight that had already driven the Amodei group's 2021 departure resurfaced almost exactly a year after ChatGPT's launch, in a very public form.

On November 17, 2023, OpenAI's board fired Altman, stating it no longer had confidence in his ability to lead the company and that he had "not been consistently candid" with the board — a deliberately vague public explanation that triggered days of speculation. More than 700 of OpenAI's roughly 770 employees signed a letter threatening to resign and follow Altman to Microsoft, which had immediately offered him a role leading a new internal AI team; under that combined internal and external pressure, an agreement in principle was reached within days, and Altman was formally reinstated as CEO on November 29, under a substantially reconstituted board. It is worth placing this crisis directly alongside the Amodei departure two years earlier — both events trace back to the same fundamental tension between how fast a frontier-AI company can move and how much independent oversight its own governance structure can actually exercise once the company's products are worth tens of billions of dollars.

X. Anthropic, Claude Code, and Agentic Engineering

By 2024, the industry's center of gravity had shifted again — from training the biggest possible model to figuring out what to actually do with the models that already existed, and nowhere was that shift more concretely visible than inside Anthropic, in the work of two people whose careers this essay has been tracking separately until now.

Andrej Karpathy left OpenAI for the second time in February 2024 — his first stint had run 2015 to 2017, followed by a tour as Tesla's Director of AI and Autopilot Vision from 2017 to 2022, then a return to OpenAI in early 2023 — and in July 2024 founded Eureka Labs, an "AI-native school" he has described as the convergence of "roughly two decades" of parallel interest in both AI and education, a lineage running directly back to CS231n and his years teaching under Fei-Fei Li. Between those two OpenAI stints, his years as Tesla's Director of AI and Autopilot Vision had already made him a public teacher as much as an engineer: at the company's first AI Day in 2021 he walked an audience through Autopilot's multi-camera neural architecture in the fullest technical detail he ever gave of what he built there, and in a May 2023 Microsoft Build keynote, "State of GPT," delivered as a then-OpenAI founding member, he walked a room of ordinary developers through the entire GPT training pipeline — pretraining, supervised fine-tuning, reward modeling, reinforcement learning from human feedback — the same pipeline Dario Amodei's teams had built the underlying models for. Across a series of talks and a widely read 2017 essay, "Software 2.0," Karpathy had by this point developed a framework he returned to constantly: Software 1.0 is code a human writes explicitly; Software 2.0 is a neural network's weights, "written" by gradient descent over data rather than typed by a programmer; and Software 3.0, the framework he articulated most fully in a June 2025 talk, is the prompt itself — an LLM is "a new kind of computer," and you program it in English, with natural language functioning as source code. In a companion analogy he first floated on social media in 2023 and later folded into the same talk, he described the "LLM OS": the model as the kernel or CPU of a new kind of operating system, its context window as RAM, retrieval systems as a file system, and tool calls as system calls.

Karpathy is also responsible for one of the single most consequential tweets in the field's recent history. In February 2025, he wrote that he had taken to "vibe coding" — describing what he wanted to an AI coding assistant and, in his words, "fully giving in to the vibes... I don't read the diffs anymore." The tweet drew more than 4.5 million views and turned "vibe coding" into a term of art almost overnight — and then, by his own account, immediately started getting flattened in public use into something closer to "prompt and pray with no discipline at all" than what he had actually meant. He spent much of the following year publicly correcting the drift, introducing a companion term, agentic engineering, at a 2026 Sequoia Capital talk: the disciplined practice — writing specs, reviewing diffs, building evaluation loops, treating an AI agent as a fallible collaborator to be orchestrated rather than a black box to be trusted — that has to grow on top of vibe coding for anything that actually matters. His compact summary of the whole distinction: "traditional software automates what you can specify; AI automates what you can verify."

That distinction, arrived at through talks and essays, was being built into an actual shipped product at almost exactly the same time by Boris Cherny, a few floors away at Anthropic. Cherny had spent five or six years at Meta, rising to Principal Engineer on Instagram and authoring the book Programming TypeScript, before joining Anthropic in September 2024 as a founding engineer. In his first week, largely as a personal side project rather than a chartered initiative, he hooked Claude up to his own terminal via a small AppleScript prototype — the beginning of what became Claude Code. It built on institutional groundwork Anthropic had already laid: work on reinforcement-learning infrastructure for coding agents that had begun as early as 2022, associated with research engineer Dawn Drain, years before Cherny himself arrived. The internal prototype, initially called "Claude CLI," was renamed Claude Code on a suggestion from product marketing ahead of its public research preview in February 2025; it reached general availability that May, by which point more than 80% of Anthropic's own engineers were already using it daily; and by November 2025 — roughly six months after general availability — it had passed a billion dollars in annualized revenue, one of the fastest revenue ramps reported in the history of software.

Karpathy's own account of his career treats teaching as far more than a side interest running alongside the research — in his telling it is close to the whole point. He has written under three successive blogs across three career chapters: an early GitHub Pages site from his Stanford and first-OpenAI years, home to "The Unreasonable Effectiveness of Recurrent Neural Networks" and a widely quoted, deliberately unglamorous checklist called "A Recipe for Training Neural Networks" (its central piece of advice: overfit a single batch first, because "neural net training fails silently"); a Medium blog during his Tesla years, home to the 2017 "Software 2.0" essay that first named the framework everything since has built on; and a current blog, started in 2025, where he has developed a second coinage that recurs constantly in his more recent talks — "jagged intelligence," describing frontier models' oddly uneven capability profile, where a system that can solve olympiad-level math can simultaneously fail to say whether 9.11 or 9.9 is the larger number. His explanation is structural rather than mysterious: AI labs pour reinforcement-learning effort into domains where an answer can be automatically verified, like math and code, and comparatively little into domains that resist verification, like everyday judgment — so the resulting skill surface ends up jagged rather than smooth, a direct precursor to the "verifiability" argument behind his and Cherny's shared framing of what agentic tools can and cannot yet be trusted to do alone.

Anthropic's own scale by this point had grown well past what its 2021 "safety-first counterweight to OpenAI" founding rationale might have suggested was possible. Its new public "constitution" for Claude, released under a Creative Commons license in January 2026, extended the Constitutional AI research program from a 2022 paper into what the company describes as the most comprehensive public framework yet for governing an advanced AI system's behavior. And in May 2026 — the same month Karpathy joined its pretraining team — Anthropic closed a $65 billion funding round at a $965 billion post-money valuation, briefly the highest of any AI company's, on revenue reported to have grown from roughly $10 million in 2022 to something like $47 billion annualized. The company that eight people left OpenAI to build, arguing commercialization was already outrunning safety research, had itself become one of the two or three largest players in exactly the race it was founded to slow down — a tension this essay does not think it is its place to resolve, only to name plainly.

Cherny's own description of what he has built and what comes next tracks Karpathy's framework almost exactly, arrived at independently through building rather than theorizing. In a lengthy 2026 interview, he stated flatly that "coding is largely solved" — not that every software problem is solved, but that the mechanical act of producing working code is no longer the field's binding constraint — and predicted the job title "software engineer" would increasingly give way to a broader one, "builder," as more people gain the ability to build software on demand. He also described the next frontier as Claude moving from executing specified tasks toward proactively surfacing its own ideas — reading through user feedback, bug reports, and telemetry to identify what to build or fix next, functioning less like a tool that waits to be prompted and more like a coworker with initiative. In May 2026, Andrej Karpathy himself joined Anthropic's pretraining team — closing, in a single move, the loop this essay has been tracing since Section VI: the Fei-Fei Li lineage and the Hinton-adjacent OpenAI-and-Anthropic lineage, meeting at last inside the same company.

XI. The Hardware Race, Part Two — and Intel's Missed Turn

Every model discussed in this essay, from AlexNet to Claude Code, was trained and run on somebody's chips, and for most of the last decade, that somebody has overwhelmingly been NVIDIA. The 2012 AlexNet result retroactively justified Jensen Huang's decade-old CUDA bet, but the payoff compounded for years afterward: as deep learning moved from a research curiosity to the industry's central product, NVIDIA's GPUs — and the software ecosystem CUDA had spent a decade building up around them — became something close to infrastructure. NVIDIA briefly became the world's most valuable company in 2024, and Huang's own public framing of the moment has only escalated since: at CES 2026, describing the shift from Software 1.0-style hand-written programs to trained models as a permanent change in what computing means, he put it as bluntly as the thesis of this whole essay could be compressed into two sentences: "You no longer write the software; you train the software. You no longer run CPUs, you run GPUs." Asked repeatedly whether the current scale of AI infrastructure spending constitutes a bubble, Huang has consistently declined to say demand is infinite and argued instead that the real limiting factor is physical — power, land, fabrication capacity — framing the current buildout as "the largest infrastructure build-out in human history" rather than a speculative overreach.

Huang himself has told the CUDA story, and NVIDIA's two other near-death moments — the RIVA 128 recovery after the NV1's commercial failure, and a costly 2010s retreat from the mobile-phone chip market — as a single recurring parable across commencement speeches, podcast interviews, and NVIDIA's own thirtieth-anniversary retrospective. At a Stanford Graduate School of Business talk in 2024 that produced his most-quoted line outside a keynote, he told the audience he wished them "ample doses of pain and suffering," arguing that "greatness is not intelligence. Greatness comes from character. And character isn't formed out of smart people, it's formed out of people who suffered" — a philosophy he has said he invokes inside NVIDIA "with great glee" as a way of treating institutional setbacks as the mechanism by which a company's character actually gets built, not merely survived. A year earlier, delivering a commencement address in Taipei, the city he'd left as a child, he put the same instinct more bluntly: "Either you're running for food, or you are running from becoming food. Oftentimes, you can't tell which. Either way, run. ... Run. Don't walk." It is not an accident that the same man who tells this story about his own company's survival is also the one whose decade-early, widely doubted CUDA bet is the reason this essay's central 2012 collision was even possible — persistence through years of doubt is, in a real sense, the one trait every figure in this essay's first four sections actually shares.

Intel, the company that had defined the chip industry for the previous thirty years, missed almost the entire transition this essay has been describing, and the story of how is worth telling in some detail because it is the clearest counter-example to everything Huang got right.

The clearest single moment came years before AI was even the question. In the mid-2000s, Apple approached Intel about building the processor for a new device — the iPhone — and Intel, led by then-CEO Paul Otellini, passed, because the price Apple wanted to pay was below Intel's forecasted manufacturing cost. Nobody, including Apple, fully knew what the iPhone would become; in hindsight, both Intel's cost forecast and its volume estimate turned out to be wrong by roughly a hundredfold. Otellini himself, in a candid 2013 interview given during his final month as CEO, called it his biggest regret: "We ended up not winning it or passing on it, depending on how you want to view it. And the world would have been a lot different if we'd done it." He drew a lesson from the decision that reads, in retrospect, like a direct rebuke of the caution that would keep costing Intel similar opportunities for the next decade: "The lesson I took away from that was, while we like to speak with data around here, so many times in my career I've ended up making decisions with my gut, and I should have followed my gut. My gut told me to say yes." Losing the iPhone meant losing the entire mobile-chip market to competitors building on the ARM architecture instead — a market that, within a few years, dwarfed the PC processors Intel still dominated, and that arguably absorbed leadership attention and capital Intel would otherwise have had available for an early GPU bet of its own.

Intel's later, direct attempts to compete in AI-relevant computing arrived late and mostly failed. Xeon Phi, its answer to GPU-style parallel computing for high-performance workloads through much of the 2010s, never became a serious platform for deep learning the way CUDA did — Intel bet on extending a CPU-like architecture rather than building a genuinely GPU-native one, and the software ecosystem never caught up to NVIDIA's decade-long head start. More recently, Intel's Gaudi line of AI accelerators, its most direct attempt yet to compete with NVIDIA for AI training and inference workloads, has struggled commercially badly enough to produce inventory-related charges of $922 million in 2024 and a further $375 million in 2025. Underneath all of this sat a manufacturing problem: Intel's own chip fabrication technology, once the industry's clear leader, fell behind Taiwan's TSMC through a series of delayed transitions to smaller, more efficient manufacturing processes, at almost exactly the years AI chip demand was exploding and precision manufacturing capacity became the industry's tightest bottleneck. There are recent signs of a partial foundry recovery — including a large 2028 manufacturing order from Google for its own AI chips, reportedly the largest external win in Intel Foundry's history — but as of this writing Intel remains a manufacturing and general-purpose-chip company trying to catch up to a race NVIDIA started running fifteen years earlier, on a bet almost nobody else thought was worth making at the time.

XII. What Comes Next

Every figure in this essay's cast is, at this moment, making a public bet about what happens next, and the bets do not agree with each other — which is itself the most honest thing this essay can report about the field's actual state, rather than smoothing the disagreement into false consensus.

Jensen Huang's bet is that the frontier is moving from generative and agentic software toward physical AI — robotics, self-driving decision systems, humanoid and tabletop robots — the same continuous logic, in his own telling, that has carried NVIDIA's chips from graphics to AI training to agents to now the physical world: each phase simply "the next thing GPUs turn out to be needed for," rather than a sequence of separate pivots. Fei-Fei Li's bet runs on a parallel but distinct track: her company World Labs, founded in 2024 and valued near $5 billion by early 2026 after a $1 billion funding round that included NVIDIA and AMD among its investors, argues that the field's next frontier is spatial intelligence — AI that understands and can act within three-dimensional physical space, not just language or flat images — built on what she calls "world models," which she has described as requiring "something even more ambitious than LLMs: new types of generative models whose capabilities of understanding, reasoning, generation and interaction with the semantically, physically, geometrically and dynamically complex worlds — virtual or real — are far beyond the reach of today's LLMs." World Labs' first commercial product, Marble, generates navigable three-dimensional environments from an image or a text prompt, and Li's own compact framing of the whole bet is the same kind of plain declarative sentence she once used to defend a dataset nobody thought was worth building: "Spatial intelligence is the next frontier of artificial intelligence." It is, in a real sense, the same ImageNet-era insight applied a second time: identify what the field is under-investing in beneath the current layer of hype, and build that directly, years ahead of demand for it.

On the question of timing — how close artificial general intelligence actually is — the cast disagrees sharply enough that the disagreement itself deserves to be the essay's central open question rather than resolved in either direction. Demis Hassabis has repeated, across several interviews in 2025 and 2026, that AGI could arrive within roughly five years, comparing its likely impact to "something like ten times the impact of the Industrial Revolution, but happening at ten times the speed," while framing the remaining distance as requiring perhaps "one or two more breakthroughs" on the scale of AlphaGo's core innovations — not simply more scale on current techniques. Sam Altman, in a June 2025 essay titled "The Gentle Singularity," wrote that "the takeoff has started," describing the current moment as the early stage of an already-underway transition to superintelligence, while insisting the transition can be managed smoothly rather than as a destabilizing break, and committing to "ensuring access to superintelligence is widespread, not concentrated in the hands of a few." Andrej Karpathy, in an October 2025 interview, staked out the most explicit skepticism among frontier-lab-adjacent voices, arguing that today's AI agents are, in his own phrase, "impressive autocomplete" with genuine cognitive deficits — weak long-horizon memory, brittle judgment — and that closing the gap will be a slow "march of nines," incremental reliability improvements rather than a single breakthrough; he expects AI's economic effect to "blend into" the roughly 2%-per-year GDP growth trend the developed world has run at for two and a half centuries, rather than produce a sudden discontinuity, and states his own position plainly: "AGI is still a decade away."

And on the question of what AI is actually doing to the people whose jobs it touches, the field's own architects disagree with themselves as much as with each other. Geoffrey Hinton, the researcher whose lifetime of unfashionable work made nearly everything else in this essay possible, left Google in 2023 specifically to warn, without an employer's interests constraining what he could say, that AI's risks deserved to be taken far more seriously than he once believed — telling a CBC interviewer it was "conceivable that this kind of advanced intelligence could just take over from us," and, in more recent interviews, warning that AI would have the capability to replace many white-collar jobs within the current year. Dario Amodei, in May 2025, told Axios that AI could eliminate up to half of entry-level white-collar jobs within five years, potentially pushing U.S. unemployment into the double digits, framing the warning as an obligation: "We, as the producers of this technology, have a duty and an obligation to be honest about what is coming." By 2026, reporting suggested both Amodei and Altman were softening the starkest versions of their earlier warnings — Amodei shifting toward an argument grounded in the economic concept of Jevons paradox, that efficiency gains can expand the total amount of a kind of work available rather than simply eliminating it — a walk-back reporters have connected, with appropriate caution about how confirmed that connection actually is, to both companies' approach to prospective public offerings. Whether that softening reflects genuinely updated evidence or simply more careful public messaging as IPO season approaches is not a question this essay can settle; it is worth naming as an open tension rather than resolving in either direction.

Underneath all of these disagreements about timing and jobs sits a stranger, more basic question Hinton himself raised in an October 2023 Toronto talk, and it is worth closing on because it is the one piece of this story that is genuinely about what these systems are, rather than what they might do next. Hinton has argued that digital computation has a property biological brains structurally cannot match: because a digital model's exact weights can be copied onto any compatible hardware, many copies of the same model can run in parallel, each learning from different data, then average their learned changes together — a form of knowledge-sharing no human brain can perform, since one person's specific neural wiring cannot simply be copied into another person's skull. He calls digital computation, in this sense, "immortal": the model survives the hardware it happened to run on. Biological computation, by contrast, is "mortal" — a brain's learning is inseparable from its own particular physical substrate, and dies with it, which is also, in Hinton's account, precisely what makes biological learning so much more energy-efficient than training a large neural network. He does not offer this as a settled argument that digital intelligence will surpass biological intelligence. He offers it as a reason not to assume, the way earlier decades of AI skepticism generally did, that biological intelligence has some permanent, structural edge — the same instinct for taking his own field's confident assumptions apart that made him spend thirty unfashionable years defending an idea nearly everyone else had given up on.

What can be said with more confidence is the shape of the pattern this whole essay has been tracing. Nothing in this story arrived as a single breakthrough. An idea waited decades for a dataset; a dataset waited years for a chip; a chip waited a decade for a use case; and once those three finally collided in 2012, it took five more years and an entirely different research group, working on an entirely different problem, to produce the architecture that would let the same convergence happen again for language instead of vision. The people in this story mostly did not set out to build what they eventually built together — Huang was selling graphics cards to video game players; Fei-Fei Li was trying to fix a narrow methodological problem in computer vision; Hinton spent a quarter century defending an idea most of his field considered a dead end. What connects them is not a shared plan but a shared willingness to keep building the one piece each of them controlled, for years, without knowing when or whether it would meet the others. The next phase of this story — physical AI, spatial intelligence, agentic systems that propose their own next steps rather than waiting to be asked — is very likely, on the evidence of everything that came before it, going to look the same way the last four did: not one clean invention, but several separately-running bets, made by people who mostly don't yet know they're building toward the same collision, suddenly lining up.

References

Every named fact, date, and quotation in this essay traces to one of the sources below. Sourcing rests on eight person-level research compilations built for this project (linked in full at the top of each section) plus targeted searches for material — the Transformer paper, Intel's specific missteps, and a handful of numbers below — that didn't belong to any single named person. Before publishing, three of the essay's more surprising or load-bearing claims were independently re-checked against primary sources rather than left resting on the original research pass: AlexNet's training hardware, NVIDIA's 2024 market-cap milestone, and Anthropic's May 2026 funding round. All three held up; the Anthropic citation below was upgraded to the company's own announcement after the check turned up a better source than the one first used. Nothing in the essay rests on a single, unverifiable, or anonymous source — every link below is live and publicly checkable.

I. Cold Winter, True Believers — Geoffrey Hinton

II. The Data Nobody Wanted to Label — Fei-Fei Li

III. The Bet Nobody Understood Yet — Jensen Huang and NVIDIA

IV. AlexNet: Where the Three Lines Cross

V. The Talent Land-Grab — Demis Hassabis and DeepMind

VI. OpenAI's Founding Bet — Sam Altman and Andrej Karpathy

VII. Attention Is All You Need

VIII. The Scaling Era and the Amodei Split — Dario Amodei and Anthropic

IX. ChatGPT's Big Bang

X. Anthropic, Claude Code, and Agentic Engineering — Karpathy and Boris Cherny

XI. The Hardware Race, Part Two — NVIDIA and Intel

XII. What Comes Next

Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)