The Best SI
EN中文

Essential papers · A reading path

Essential
reading.

From Transformers and scaling laws to reasoning, tool use, and autonomous learning. A path through the papers that explain how AI got here, and where its next advances may come from.

16 essential papersUpdated 2026-10-12

A place to begin

  1. Stage 01

    Architecture & scale

    How attention, data, and compute became the foundations of modern language models.

  2. Stage 02

    Alignment & reasoning

    How feedback and additional reasoning change what a pretrained model can do.

  3. Stage 03

    Knowledge & action

    From retrieving evidence to using tools and running an experimental workflow.

  4. Stage 04

    World models & safety

    Learning through interaction, predicting consequences, and choosing objectives that hold up.

The selection

Papers worth reading closely.

  1. 01Research paperTransformer architecture

    Attention Is All You Need (opens in a new window)

    Ashish Vaswani et al.Intermediate

    Core contribution

    Introduces the Transformer, an encoder–decoder architecture that uses attention instead of recurrent or convolutional sequence processing. Multi-head attention, positional information, and parallel training produce strong machine-translation results with substantially less training time than the compared systems. The central design makes relationships between distant tokens directly accessible within a layer.

    Why read it

    Start here to understand the architectural vocabulary behind later language models. Focus on the attention equations and architecture diagram, then connect each component to the computational bottleneck it addresses.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  2. 02Technical reportScaling laws

    Scaling Laws for Neural Language Models (opens in a new window)

    Jared Kaplan et al.Intermediate

    Core contribution

    Measures how language-model cross-entropy loss changes with parameter count, dataset size, and training compute. Across the experimental range, the authors find regular power-law relationships and use them to estimate how a fixed compute budget should be allocated. This turns model scaling into a quantitative question about interacting resource constraints.

    Why read it

    Read it before Chinchilla to see both the appeal of predictable progress and the importance of experimental assumptions. Pay attention to what is measured: prediction loss, rather than a universal score for intelligence.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  3. 03Research paperIn-context learning

    Language Models are Few-Shot Learners (opens in a new window)

    Tom B. Brown et al.Accessible

    Core contribution

    Presents GPT-3 and a broad evaluation of language tasks performed through text prompts, without task-specific gradient updates at evaluation time. Increasing model scale improves many zero-shot and few-shot results, while examples supplied in the context let the same pretrained model adapt its responses to different tasks.

    Why read it

    This is the key reading for understanding why prompting became a useful interface. Compare the zero-shot, one-shot, and few-shot settings, and examine the uneven results across tasks to see what in-context adaptation actually changes.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  4. 04Research paperCompute-optimal training

    Training Compute-Optimal Large Language Models (opens in a new window)

    Jordan Hoffmann et al.Intermediate

    Core contribution

    Revisits the balance between model size and training data under a fixed compute budget. Experiments with more than 400 models suggest scaling parameters and training tokens in roughly equal proportions. The resulting 70-billion-parameter Chinchilla outperforms the much larger Gopher across the reported evaluations while using the same training compute.

    Why read it

    Read this alongside the earlier scaling-laws paper to understand why more parameters alone can be an inefficient investment. The important lesson is how to design controlled scaling experiments and distinguish training efficiency from deployment costs.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  5. 05Research paperRLHF and instruction following

    Training language models to follow instructions with human feedback (opens in a new window)

    Long Ouyang et al.Accessible

    Core contribution

    Shows a practical pipeline for turning a pretrained language model into a more useful instruction-following system: supervised demonstrations, a reward model trained on human rankings, and reinforcement learning. On the authors’ evaluation prompt distribution, human raters prefer a 1.3-billion-parameter InstructGPT model to the much larger 175-billion-parameter GPT-3 baseline.

    Why read it

    Read it to separate language-model capability from useful behavior. The data-collection choices and evaluator population are especially important: they explain how human judgments enter the training objective and shape what the system learns to optimize.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  6. 06Research paperPreference optimization

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model (opens in a new window)

    Rafael Rafailov et al.Technical

    Core contribution

    Derives a direct training objective for learning from preferred and rejected responses. By relating an optimal policy to its reward function under a regularized objective, DPO removes the need to train a separate reward model and run the usual reinforcement-learning optimization loop. The paper evaluates the method on sentiment, summarization, and dialogue tasks.

    Why read it

    Read it after InstructGPT to understand a simpler route from feedback to behavior. Following the derivation reveals which assumptions make that simplification possible and why preference data quality still matters.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  7. 07Research paperReasoning prompts

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (opens in a new window)

    Jason Wei et al.Accessible

    Core contribution

    Demonstrates that prompting sufficiently large language models with worked examples containing intermediate steps can improve arithmetic, commonsense, and symbolic reasoning results. The method changes the examples and generated computation at inference time, without requiring a new task-specific training run. Its experiments make the relationship between model scale and prompting strategy particularly visible.

    Why read it

    Read it as the bridge from answer generation to explicit multi-step problem solving. The ablations help identify what contributes to improved answers, and the error examples are useful for judging the quality of generated reasoning.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  8. 08Technical reportReasoning reinforcement learning

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (opens in a new window)

    DeepSeek-AITechnical

    Core contribution

    Studies reinforcement learning for improving reasoning in pretrained language models. R1-Zero applies reinforcement learning without an initial supervised fine-tuning stage; R1 adds cold-start examples and a multi-stage pipeline to improve usability and performance. The report also studies transferring reasoning behavior to smaller models through distillation using generated examples.

    Why read it

    Read it after chain-of-thought prompting to understand the shift from eliciting reasoning to training for it. Compare the reward design, R1-Zero and R1 pipelines, and distillation results to see which ingredients drive the reported gains.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  9. 09Research paperRetrieval and memory

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (opens in a new window)

    Patrick Lewis et al.Intermediate

    Core contribution

    Combines a pretrained sequence-to-sequence generator with a dense retriever over Wikipedia. The model draws on both knowledge stored in its parameters and retrieved passages, with variants that select evidence for an entire output or for individual tokens. Joint fine-tuning improves several knowledge-intensive question-answering and generation tasks in the reported experiments.

    Why read it

    Read it to understand why an AI system’s knowledge need not live entirely inside model weights. The distinction between retrieval, generation, and updating the document index is fundamental to designing systems that can consult changing evidence.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  10. 10Research paperAgent interaction loops

    ReAct: Synergizing Reasoning and Acting in Language Models (opens in a new window)

    Shunyu Yao et al.Accessible

    Core contribution

    Interleaves written reasoning steps with actions such as looking up information or interacting with an environment. New observations feed back into the next step, allowing the model to revise its approach rather than generate a complete plan in isolation. Experiments cover question answering, fact verification, simulated household tasks, and web shopping.

    Why read it

    This is an accessible starting point for understanding language-model agents. Follow a complete trajectory and ask how each observation changes the next action; that feedback loop is more important than any particular prompting template.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  11. 11Research paperLearned tool use

    Toolformer: Language Models Can Teach Themselves to Use Tools (opens in a new window)

    Timo Schick et al.Intermediate

    Core contribution

    Trains a language model to insert useful API calls into text. Starting from a few demonstrations, the system proposes calls, executes them, and retains those whose results improve prediction of subsequent tokens. Fine-tuning on this augmented data teaches when to use tools such as a calculator, search system, or translation service.

    Why read it

    Read it alongside ReAct to compare two ways of building tool-using systems: interaction guided by prompts and tool use learned during training. The data-filtering criterion offers a concrete example of generating supervision from a model’s own trials.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  12. 12Research frameworkAutomated research

    The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery (opens in a new window)

    Chris Lu et al.Accessible

    Core contribution

    Connects idea generation, code editing, experiments, plots, paper writing, and automated review in a single research workflow. Starting from human-provided experiment templates, the system explores small machine-learning research projects across three areas. The paper includes generated outputs and analyzes failures, including experimental mistakes and misleading results that demand careful checking.

    Why read it

    Read it to examine what automating scientific work actually requires beyond fluent writing. Its most useful questions concern experimental validity, reproducibility, and evaluation: a completed research loop needs evidence that the resulting claims survive scrutiny.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  13. 13Research paperSelf-play and search

    Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (opens in a new window)

    David Silver et al.Intermediate

    Core contribution

    Presents AlphaZero, which combines a neural network, Monte Carlo tree search, and reinforcement learning from self-play. Starting without human game records, instances trained for chess, shogi, and Go achieve strong play using the same general approach. Search produces improved action targets, while completed games provide outcomes that train the network.

    Why read it

    Read it to understand a concrete mechanism for improving beyond imitation of demonstrations. The interaction between learned intuition and search is especially important, as are the clear rules and feedback that make this learning environment tractable.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  14. 14Research paperLearned environment models

    World Models (opens in a new window)

    David Ha and Jürgen SchmidhuberAccessible

    Core contribution

    Builds a compact model of a visual environment by combining a learned image representation with a recurrent model of how that representation changes over time. A small controller uses those features to act. In one experiment, a policy trained inside the learned simulator transfers back to the original game environment.

    Why read it

    Read it for an intuitive introduction to learning through imagined experience. The separation between perception, prediction, and control makes the architecture easy to inspect, while the experiments show why a useful simulator need not reproduce every visual detail.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  15. 15Research paperWorld-model reinforcement learning

    Mastering Diverse Domains through World Models (opens in a new window)

    Danijar Hafner et al.Technical

    Core contribution

    Presents DreamerV3, a reinforcement-learning algorithm that learns a world model and trains an actor and critic on imagined trajectories. Normalization and other robustness techniques let the same algorithm configuration work across more than 150 tasks. The experiments include collecting diamonds in Minecraft without human demonstrations or a handcrafted curriculum.

    Why read it

    Read it after World Models to see how an appealing concept becomes a broadly usable learning algorithm. Focus on stable optimization across different reward scales and observations, and on the distinction between reusing an algorithm and transferring a trained policy.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)
  16. 16Research agendaObjectives and reliable learning

    Concrete Problems in AI Safety (opens in a new window)

    Dario Amodei et al.Accessible

    Core contribution

    Organizes accidental harmful behavior into five practical research problems: side effects, reward hacking, scalable supervision, safe exploration, and distributional shift. Rather than proposing one solution, the paper connects these problems to existing machine-learning methods and sketches directions for experiments. Its examples make failures of objectives, feedback, and generalization easy to distinguish.

    Why read it

    Read it as a technical checklist for the more capable systems in this list. It helps turn broad questions about control into specific questions about what gets rewarded, what can be observed, and what changes outside training.

    Read the paper (opens in a new window)Paper PDF (opens in a new window)