Author: Claude AI, under the supervision, prompting and editing by HocTro
Opening
When you ask AI a question and get an answer within a second, it feels like magic happening somewhere in an invisible, weightless cloud. But there's nothing invisible about it. Behind every answer is a real building — sometimes several buildings — packed with tens of thousands of specialized computers, running around the clock without pause, consuming as much electricity as a small city.
That's the data center — where AI actually lives.
What Does a Data Center Look Like?
Picture a large industrial building — usually with no windows, a plain exterior that looks like a warehouse. No grand signage, no impressive lobby. From the outside, you'd never guess this is one of the most expensive things humans have ever built.
Inside is a different world entirely. The first thing you notice is the sound — the constant roar of cooling fans, like standing in a wind tunnel. Temperature is controlled to within a degree. Air is continuously filtered. Each building may have multiple floors, and each floor is a room stretching dozens of meters end to end, filled with server racks — metal cabinets rising almost to the ceiling, lined up in long rows.
Inside each rack are servers — thin, flat computing units like drawer inserts, each one a separate computer. A single rack holds 20–40 servers. A large data center may have thousands of racks. The total number of machines inside is counted in the tens of thousands to hundreds of thousands.
All of them are interconnected by a dense web of fiber optic cables — hundreds of thousands of strands running through channels in the ceiling and under the floor, carrying data at the speed of light between machines.
GPU vs CPU — Why Does AI Need Special Chips?
Not all computers are alike. Ordinary computers use a CPU (Central Processing Unit). AI computing runs primarily on GPUs (Graphics Processing Units). This distinction matters so much that it shapes the entire hardware architecture of modern AI.
Think of a CPU as a small team of highly skilled specialists. Each one can tackle complex problems, handle many different types of tasks, and process things sequentially with intelligence. A modern CPU has 8–64 cores, each one very powerful and flexible.
A GPU is completely different. It's more like an army of thousands of workers doing simple tasks, all in parallel simultaneously. A modern GPU has tens of thousands of small cores — the NVIDIA H100 has 16,896 CUDA cores. Each core isn't especially intelligent on its own, but when tens of thousands operate together, the total processing throughput is extraordinary.
Why does AI need that? Because the entire computational process of a neural network — both training and token generation — is fundamentally matrix multiplication: multiplying enormous arrays of numbers together in a massively parallel way. This is precisely what GPUs were designed to do. A CPU is powerful but slow at this specific task compared to a GPU.
The practical result: training a large model on CPUs would take hundreds of years. On GPUs, it takes months. That difference is why NVIDIA — the company that makes GPUs — became one of the most valuable companies in the world over the past few years.
A single NVIDIA H100 chip costs roughly $30,000–40,000. A GPU cluster for training a large model might contain tens of thousands of these chips. The hardware alone is a billion-dollar investment.
Power Consumption — How Much Does AI Use?
This is the number that surprises most people.
A mid-sized data center consumes roughly 20–50 megawatts (MW) of electricity. A large AI-focused data center can consume 100–500 MW — equivalent to the power supply for a city of 100,000–500,000 people.
Microsoft, Google, Amazon, and Anthropic are all building or leasing data centers at this scale — and the numbers are growing rapidly. According to IEA (International Energy Agency) estimates, by 2026, AI data centers globally could consume a combined 1,000 terawatt-hours per year — roughly equivalent to Japan's entire annual electricity consumption.
Where does the power come from? It depends on location. Some data centers in regions with abundant hydroelectric power (like Oregon in the US, or Norway in Europe) can run largely on clean electricity. Others rely on natural gas or even coal. This is one reason the AI industry faces criticism about its environmental footprint.
Cooling — The Problem Nobody Thinks About
Electricity flowing through chips generates heat. Tens of thousands of GPU chips running at full capacity generate enormous amounts of heat. Without cooling, chips fail within minutes.
Data centers use several cooling approaches:
Air cooling: Massive fans blow cold air through the racks. This is the traditional approach — common but less efficient at the high power densities modern AI chips demand.
Water cooling: Cold water runs through pipes placed directly against the chips, drawing heat away far more efficiently than air. The warmed water is then carried outside, cooled in a cooling tower, and circulated back — a continuous loop.
Liquid immersion: Some next-generation data centers submerge entire servers in a special non-conductive liquid — extremely efficient cooling, but costly to build and maintain.
Cooling accounts for roughly 30–50% of a data center's total electricity consumption — meaning for every 1 kWh used to run AI computation, another 0.3–0.5 kWh is needed just to keep the chips cool.
Water is an environmental concern that rarely gets mentioned. A large data center can consume millions of liters of water per day for its cooling systems. In regions where water scarcity is a genuine issue, this is a real problem.
Where in the World Are Data Centers?
Location isn't random. Several factors determine where a data center gets built:
- Cheap, stable electricity: Regions with abundant hydroelectric power, like Oregon in the US
- Cool climate: Reduces cooling costs — Northern Europe (Ireland, Norway, Sweden) is highly sought after
- Strong internet connectivity: Proximity to major undersea fiber optic cable landing points
- Favorable policy: Data privacy laws, tax incentives, and environmental regulations
The world's largest data center clusters: Northern Virginia (US), Oregon (US), Singapore, Dublin (Ireland), Amsterdam (Netherlands), and Tokyo (Japan). Northern Virginia — the suburbs just outside Washington D.C. — has the highest concentration of data centers on Earth. People in the industry call it "the capital of the internet."
Microsoft once experimented with placing a data center on the ocean floor off the coast of Scotland — Project Natick — using the cold seawater for cooling. After two years, they found that chips underwater actually experienced fewer failures than chips on land, because there was no oxygen (oxygen corrodes chips) and humidity stayed constant.
When You Ask AI — How Many Machines Are Involved?
Every time you send a message and receive an answer, a chain of servers is quietly coordinating:
- A load balancer distributes your request to whichever server is least busy at that moment
- An authentication server checks your API key and validates your account
- The GPU cluster actually runs the model and generates the tokens
- A caching server speeds up responses for common or repeated queries
- A monitoring system tracks performance and catches errors in real time
And all of that is replicated across multiple physical locations around the world — so if one data center goes down, others continue serving users seamlessly. This is called redundancy — the most important design principle in cloud infrastructure.
Environmental Impact — The Uncomfortable Question
AI consumes energy and water at scale. That's a reality that can't be avoided.
The major AI companies all have renewable energy commitments — Google, Microsoft, and Amazon have all declared carbon neutral or carbon negative targets. But "carbon neutral" is sometimes achieved by purchasing carbon credits rather than actually running on clean electricity. And while long-term commitments are being built out, short-term power demand still has to be met with whatever sources are currently available.
A more balanced view: AI is also being used to optimize power grids, forecast renewable energy output, and design better batteries. The question is whether those benefits outweigh the electricity and water it consumes — and that debate is still very much ongoing.
Summing Up
AI doesn't live in a weightless cloud. It lives in real buildings, with real costs, consuming real resources. Every answer you receive is the result of thousands of GPU chips working in coordination, cooled by complex systems, powered by enough electricity to light up a neighborhood.
Knowing this doesn't mean you should feel guilty about using AI. But it helps you understand that AI isn't "free" in any resource sense — and the infrastructure decisions made by AI companies have real consequences in the real world.
The final article: now that you understand AI from the inside out — tokens, servers, neural networks, data centers — it's time to put your hands on it. Article 7 is about Claude Code: how to install it, how to get started, and how to use AI as a genuinely powerful tool right inside your terminal.
Quick Reference Table
| Concept | Vietnamese Term | Short Definition |
|---|---|---|
| Data center | Trung tâm dữ liệu | Building housing thousands of servers for AI and cloud |
| Server | Máy chủ | A single computer inside a data center, running continuously |
| Server rack | Giá đỡ máy chủ | Metal cabinet holding 20–40 stacked servers |
| GPU | Bộ xử lý đồ họa | Massively parallel chip — ideal for AI computation |
| CPU | Bộ xử lý trung tâm | General-purpose chip — powerful but not parallel like GPU |
| Matrix multiplication | Nhân ma trận | The core calculation of neural networks; GPUs excel at this |
| Load balancer | Cân bằng tải | Distributes requests to whichever server is least busy |
| Redundancy | Dự phòng | Replicating systems across locations to prevent outages |
| Carbon credit | Tín chỉ carbon | A mechanism for offsetting emissions by funding green projects |
Key Things to Remember
- AI lives in real buildings. A data center is concrete, physical infrastructure — not an invisible cloud.
- GPUs are the heart of AI. Tens of thousands of parallel cores — ideal for the matrix multiplication that neural networks need.
- One H100 GPU costs ~$30,000–40,000. A training cluster may contain tens of thousands of them.
- Power consumption equals a small city. A large AI data center uses 100–500 MW of electricity.
- Cooling adds 30–50% more power consumption — plus millions of liters of water per day.
- Final article: Install Claude Code and start using AI as a genuinely powerful tool in your terminal.