Picks

Picks

Selected, sourced, no spin. What happened, who says so, and what is still only claimed.

Now

Living issue

Lead

GPT-6 Astra edits another network's weights directly, its authors claim

GPT-6 Astra edits another network's weights directly, its authors claim

Across eight experiments, the authors say the model reads, repairs and writes network weights from plain-language goals. A video walkthrough points out that most of the headline math gain came from stopping reasoning loops.

Article1 source

How its claims stand
  • Borne out by another source: 5
  • Said here alone: 39

Agents

Models

Refusal in Language Models Is Mediated by a Single Direction

Refusal in Language Models Is Mediated by a Single Direction

Refusal of harmful instructions in open-source chat models is controlled by a single direction in the residual stream. Erasing this direction stops refusal, and adding it induces refusal on harmless prompts. The same finding enables a rank-one weight edit that works as a white-box jailbreak, and it explains how adversarial suffixes suppress refusal.

Paper

How its claims stand
  • Said here alone: 50

Mathematics

Steven Strogatz and Grant Sanderson on why doing math still matters

Steven Strogatz and Grant Sanderson on why doing math still matters

AI may transform mathematical discovery and careers without eliminating the value of doing, learning, and sharing mathematics. Steven Strogatz and Grant Sanderson locate that value in understanding, personal struggle, teaching, individuality, and human connection, while considering new forms of AI-assisted work and mathematical education.

Video1 tension

How its claims stand
  • Contested by another source: 1
  • Said here alone: 68

Foundations

Attention Is All You Need

Attention Is All You Need

The paper proposes the Transformer, a sequence transduction architecture built only on attention mechanisms, with recurrence and convolution removed. It reports state-of-the-art machine translation results on WMT 2014 English-to-German and English-to-French at a fraction of earlier training cost, and good results on English constituency parsing. The authors argue that attention alone is more parallelizable and faster to train.

Paper1 source

How its claims stand
  • Borne out by another source: 1
  • Said here alone: 39
Open sourcing BERT (Google, 2018)

Open sourcing BERT (Google, 2018)

The Google AI Language team announces the open-source release of BERT, a pre-training technique for natural language processing. BERT is a deeply bidirectional model pre-trained on unannotated text, which can be fine-tuned quickly on small task datasets. The post reports state-of-the-art results on 11 NLP tasks.

Article

How its claims stand
  • Said here alone: 17
Efficient Estimation of Word Representations in Vector Space

Efficient Estimation of Word Representations in Vector Space

The paper introduces two simple log-linear architectures, CBOW and Skip-gram, for learning vector representations of words from very large text corpora. It defines a word-relationship test set on which the vectors answer syntactic and semantic questions by vector arithmetic, and reports accuracy, training time and a sentence completion result.

Paper

How its claims stand
  • Said here alone: 39

Course: Stanford CME295

Lecture 1: Transformers

Lecture 1: Transformers

Transformers enabled modern LLMs through an architecture that scales with data, compute, and parameters. Their foundations combine tokenization and learned embeddings with positional information, attention, and feedforward networks to build contextual representations and generate text.

Video1 source

How its claims stand
  • Borne out by another source: 1
  • Said here alone: 45
Lecture 2: Large language models

Lecture 2: Large language models

Large language models build on transformers, with decoder-only generation becoming dominant alongside representation-based approaches such as BERT. Their effectiveness depends on balancing model size and training data, using efficient architectures, and choosing suitable decoding methods. Prompting and reasoning techniques can improve responses without changing weights, but longer contexts can impair retrieval and increase costs.

Video

How its claims stand
  • Said here alone: 59
Lecture 3: Training LLMs

Lecture 3: Training LLMs

LLM training progresses from reusable next-token pretraining through targeted midtraining, supervised instruction tuning, preference alignment, and reasoning training. RLHF, DPO, verifiable rewards, and on-policy distillation provide different ways to improve behavior and capabilities, while data quality, computational costs, and capability preservation remain central constraints.

Video1 source

How its claims stand
  • Borne out by another source: 2
  • Said here alone: 47
Lecture 4: Reinforcement learning with LLMs

Lecture 4: Reinforcement learning with LLMs

Reinforcement learning trains LLMs for human preferences and reasoning by maximizing learned or verifiable rewards, but must address noisy gradients, token-level credit assignment, and excessive policy changes. PPO and GRPO make different trade-offs between these problems, while on-policy distillation provides token-level teacher feedback on student-generated responses, with extensions to self-distillation and domain-specific teachers.

Video1 source

How its claims stand
  • Borne out by another source: 2
  • Said here alone: 43

Weather (temporary)

Charter

Picks charter: grounded and verified reporting.