"Attention Is All You Need"
Eight Google researchers publish a 15-page paper that becomes the architectural blueprint for ChatGPT, Claude, Gemini, and every frontier AI model.
In June 2017, Ashish Vaswani and seven co-authors at Google Brain published "Attention Is All You Need," introducing the Transformer architecture. It replaced the sequential processing of recurrent neural networks (RNNs) with parallel self-attention mechanisms, allowing models to process entire sequences simultaneously. This single paper is the foundation of GPT, BERT, Claude, Gemini, and virtually every modern AI system.
Open in interactive timeline →Key Numbers
- Paper Length
- 15 pages
- Citations (by 2026)
- 150,000+
- Architectures Spawned
- GPT, BERT, Claude, Gemini, LLaMA
Verified Facts
- The paper was initially about machine translation, but the Transformer architecture turned out to be universal—it works for text, images, audio, video, protein folding, and code.
- Key innovation: "self-attention" allows the model to weigh the relevance of every word to every other word in a sequence, capturing long-range dependencies that RNNs could not.
- Of the 8 authors, several went on to found AI companies: Noam Shazeer co-founded Character.AI, Aidan Gomez co-founded Cohere, and Niki Parmar co-founded Adept AI.
- The Transformer enabled scaling laws: performance improves predictably with more data and compute. This insight drives the multi-billion-dollar AI training runs of 2025-2026.
Frequently Asked Questions
What was "Attention Is All You Need"?
In June 2017, Ashish Vaswani and seven co-authors at Google Brain published "Attention Is All You Need," introducing the Transformer architecture. It replaced the sequential processing of recurrent neural networks (RNNs) with parallel self-attention mechanisms, allowing models to process entire sequences simultaneously. This single paper is the foundation of GPT, BERT, Claude, Gemini, and virtually every modern AI system.
When did "Attention Is All You Need" happen?
"Attention Is All You Need": 2017 CE.
Why does "Attention Is All You Need" matter?
Eight Google researchers publish a 15-page paper that becomes the architectural blueprint for ChatGPT, Claude, Gemini, and every frontier AI model.
Sources & Further Reading
Cite This Page
AskHistoryAI. “"Attention Is All You Need".” AskHistoryAI — Interactive Timeline of Everything. Updated 2026-09-11. https://askhistoryai.com/event/tech-transformer/
Every fact on this page is checked against the published fact ledger and methodology; sources are listed above.