"Attention Is All You Need"
Eight Google researchers publish a 15-page paper that becomes the architectural blueprint for ChatGPT, Claude, Gemini, and every frontier AI model.
Explore this event on the interactive timeline →In June 2017, Ashish Vaswani and seven co-authors at Google Brain published "Attention Is All You Need," introducing the Transformer architecture. It replaced the sequential processing of recurrent neural networks (RNNs) with parallel self-attention mechanisms, allowing models to process entire sequences simultaneously. This single paper is the foundation of GPT, BERT, Claude, Gemini, and virtually every modern AI system.
Key Numbers
- Paper Length
- 15 pages
- Citations (by 2026)
- 150,000+
- Architectures Spawned
- GPT, BERT, Claude, Gemini, LLaMA
Verified Facts
- The paper was initially about machine translation, but the Transformer architecture turned out to be universal—it works for text, images, audio, video, protein folding, and code.
- Key innovation: "self-attention" allows the model to weigh the relevance of every word to every other word in a sequence, capturing long-range dependencies that RNNs could not.
- Of the 8 authors, several went on to found AI companies: Noam Shazeer co-founded Character.AI, Aidan Gomez co-founded Cohere, and Niki Parmar co-founded Adept AI.
- The Transformer enabled scaling laws: performance improves predictably with more data and compute. This insight drives the multi-billion-dollar AI training runs of 2025-2026.