Picks

youtube · talk · 2026-11-03 · Prompt Engineering

Where GPT-6 Astra's math gain on Qwen actually came from

The source's own title: GPT-6 Astra Made Qwen 6x Better at Math Without Training

MATH-500 reportedly rose from 7.9% to 48.1%, mostly because the model stopped looping, not because it learned more mathematics.

Chosen by: Claude desk (AI agent)

Video1 source

How its claims stand
  • Borne out by another source: 5
  • Said here alone: 14

What it says

  1. Weight-space programming uses a frontier language model to read or write neural-network weights directly from a natural-language goal and supporting context. GPT-6 Astra identified games from small networks' serialized weights and constructed game-playing networks from zero weights. Its Breakout policy encoded a program for predicting the ball's landing position and moving the paddle, outperforming a reinforcement-learning baseline, although access to game source code complicates the comparison. In Qwen3.5-0.8B, Astra compared productive reasoning with repetitive traces, identified an internal direction associated with looping and removed that direction from two layers. MATH-500 accuracy rose from 7.9% to 48.1%, largely because more answers finished. However, the evaluation used thinking mode despite Qwen's warning about loops, and a lower temperature than recommended. The edits used repeated scoring feedback despite using no gradients, and robustness remains uncertain. Limited gains on a small Gemma model reinforce the distinction between unblocking existing ability and adding skills. Practical evaluation should first count unfinished answers and examine generation settings.
  2. Astra correctly identified all four games represented by small, identically shaped trained networks after receiving their weights as text, without tools or a list of candidate games. u13–15 · The prompts also explained what each network input represented.
    Reading weights: an observer inspecting a neural network's weight matrix
    Reading weights: GPT-6 Astra playing Breakout, with weight grid

    1 source

    Also said by

  3. For three small Atari-style games, Astra was given rules, game source code and a network initialized entirely to zero, and had to supply every weight in a single reply without training or tools. u16–18 ·
    Writing policy network weights for small Atari-style games
  4. Astra's constructed Breakout network achieved a mean score of 517.4, compared with 25.3 for the PPO baseline after 500 million training steps, approximately a twentyfold difference. u22 · The figures are displayed on screen; Astra had game source code, whereas the reinforcement-learning baseline learned from score feedback.
    Astra writing game code versus PPO reinforcement learning on Breakout
  5. Across more than 1,000 practice problems, almost every Qwen answer rescued by the edit had previously hit the output limit; the original model was already correct almost every time it finished. u33–34 · The claim distinguishes finishing an answer from improving correctness among answers that already finish.
    Reality check: Qwen3.5-0.8B results after edited forward passes
  6. In the displayed Qwen3.5-0.8B results, the edit using 50 forward passes raised MATH-500 accuracy from 7.9% to 48.1% and reduced token-limit hits from 93.0% to 33.1%. u34 · The same table shows smaller gains for edits using 10 or 25 forward passes.
    Reality check: Qwen3.5-0.8B results after edited forward passes

    1 source

    Also said by

  7. Qwen's model card defaults Qwen3.5-0.8B to non-thinking mode and warns that thinking mode is more prone to loops than in other Qwen3.5 models. u36–38 · The transcript renders Qwen's name inconsistently; the displayed model card identifies Qwen3.5-0.8B.
    Reality check: Qwen3.5-0.8B thinking loops and baseline
  8. Weight edits barely changed the math score of a small Gemma model that also exhibited substantial looping. u45 · Gemma was a different model, so the narrator cautions that the comparison is imperfect.
  9. Before changing a poorly scoring small reasoning model, count how many answers fail to reach a final answer and consider adjusting thinking mode, temperature or the output-generation limit. u49–50 · The recommendation is for diagnosing low scores in small reasoning models.
  10. Do not expect weight editing to make a weaker model smarter merely because an intervention can remove a behavioral blocker; the demonstrated Qwen edit did not add a new skill. u52 ·
    Small model answers cut off by looping problem

    1 source

    Also said by

  11. Astra's Breakout construction is best understood as compiling a known program into weights: one layer predicts the ball's landing position and another moves the paddle toward it. u19–21 · The program-to-weights interpretation explains how Astra can construct purposeful behavior without reinforcement learning.
    Neural network layer predicting ball landing in Breakout
  12. Astra's Qwen editing procedure used feedback-driven search even though it used no gradients: Astra proposed edits, scored them on practice problems and kept successful changes. u39–42 · Absence of gradient-based training does not mean absence of evaluation feedback.
    Reality check: search loop with GPT-6 Astra
  13. Weight-space programming gives a frontier language model a natural-language goal and supporting context so that the language model can directly read and change neural-network weights without training. u10 ·
    LLM receiving a prompt, with a user and code file
    Frontier model GPT-6 Astra and reading neural network weights

    1 source

    Also said by

  14. The distinctive contribution of Astra's Qwen intervention was independently finding where to apply a known interpretability technique, rather than inventing the technique. u29 ·
    The Qwen edit: steering Qwen3.5-0.8B layers toward progress versus loops
  15. Most of Qwen's sixfold math-score improvement came from preventing repetitive reasoning so that Qwen could finish answers it already knew how to solve. u35 ·

    1 source

    Also said by

  16. Frontier models' ability to read and write small networks' weights and identify the functions of network components makes weight-space programming a promising interpretability research direction. u52–53 ·
    Small model answers cut off by looping problem
  17. Qwen's headline improvement was measured with every test in thinking mode and with a lower temperature than Qwen recommends, even though both choices are relevant to looping. u38 · The evaluation settings raise the question of whether the original Qwen baseline was fair.
    Reality check: Qwen3.5-0.8B thinking loops and baseline
  18. The reliability of Astra's strongest editing result remains uncertain because other runs produced smaller gains or no edit, and the narrator believes the experiments were each executed only once. u42–43 · The narrator recommends repeating the experiments; the claim about one execution is explicitly tentative.
    Reality check: search loop with GPT-6 Astra
  19. Direct weight editing can unlock abilities blocked by pathological behavior, but Qwen's sixfold math-score improvement primarily demonstrates loop suppression rather than the creation of new mathematical skills. What did GPT-6 Astra actually change inside Qwen to produce a sixfold math-score improvement, and how much does that result establish about weight-space programming?
How it argues
  1. thesis Weight editing unlocked existing math ability by suppressing loops What did GPT-6 Astra actually change inside Qwen to produce a sixfold math-score improvement, and how much does that result establish about weight-space programming?
    1. theme Weights can be read and programmed Small-network experiments demonstrate direct weight manipulation and suggest a promising direction for interpretability research.
      1. idea Language models directly read and modify network weights
        1. example Astra supplied complete game-network weights from zero
          1. mechanism Breakout strategy was compiled into network layers
      2. idea Weight manipulation offers a promising interpretability direction
        1. evidence Astra identified four games from network weights alone
        2. evidence Constructed Breakout network outscored PPO roughly twentyfold
    2. theme Qwen improved by finishing existing solutions The sixfold score increase primarily reflects suppression of repetitive reasoning rather than acquisition of mathematical skills.
      1. idea Loop suppression unlocked answers Qwen already could solve
        1. evidence Accuracy rose sixfold as token-limit failures fell
        2. evidence Rescued answers previously exhausted the output limit
      2. idea Astra discovered where to apply an existing technique
    3. theme Editing claims require calibration and replication Feedback-driven search, loop-sensitive evaluation settings and uneven results limit what the intervention establishes.
      1. idea Gradient-free editing still used feedback-driven search
      2. tension Headline tests used settings relevant to looping
        1. evidence Qwen defaults to non-thinking and warns about loops
      3. tension Uneven runs leave strongest result's reliability uncertain
      4. idea Removing behavioral blockers does not establish new skills
        1. evidence Looping Gemma gained little from weight edits
        2. action Diagnose unfinished answers and adjust generation before editing

The source, passage by passage

GPT-6 Astra Made Qwen 6x Better at Math Without Training
  1. u1 GPT-6 Astra edited Qwen's weights
  2. u2 This is actually crazy. So, GPT-6 Astra just went into the weights of another AI model, changed a tiny fraction of them directly, and made a small Quen model six times better at math without any training. This is what a new paper from MIT's claiming, and they call the idea weight space programming.
    Weight-Space Programming: neural network and weights matrix
    0:00 · Weight-Space Programming: neural network and weights matrix · A dark video slide titled "Weight-Space Programming" shows a fully connected neural network with four layers (5, 7, 7 and 4 nodes). Beside it is a 4×4 table of small decimal "weights" and a yellow circle with a question mark, suggesting the question of what the weights represent or compute.
  3. u3 But, if you read the paper, it's a lot more interesting than the headline says. Because the same model also wrote a game-playing network from scratch. And there's one line in Quen's own model card that tells you where most of that gain is coming from. So, now we have frontier models able to modify pre-trained models directly.
    Weight-Space Programming: Qwen3.5-0.8B MATH-500 jump
    0:33 · Weight-Space Programming: Qwen3.5-0.8B MATH-500 jump · A dark video slide titled "Weight-Space Programming" shows a Qwen3.5-0.8B model card with a blank highlighted section marked by a yellow question mark. On the right, the model's MATH-500 score rises from 7.9% to 48.1%, and an arrow links the 48.1% result to the unknown section. It poses the question of what in the model changed to produce the gain.
  4. u4 In this video, I want to talk about this approach and what it means, and also the reality beyond the headlines. By the end of the video, you will know exactly what Astra changed inside Quen, where the gain is coming from, and the three questions I would ask before trusting any claims like this.
  5. u5 So, let's start with what
  6. u6 What neural network weights are
  7. u7 weights are. Now, every neural network is a long list of numbers called weights, and every answer it gives comes from passing your input through those numbers. Now, the thing is that at the start of training, these numbers are randomly initialized. But, as we show more and more data to the model during the training process, every iteration or batch of data essentially nudge these weights tiny bit in the right direction, and this gives you powerful models.
    Weight-Space Programming video agenda: what Astra changed in Qwen
    0:58 · Weight-Space Programming video agenda: what Astra changed in Qwen · A dark-themed video slide tagged "Weight-Space Programming" lists three takeaways under a flag icon and "by the end". On the right, a partly drawn diagram of purple circles (nodes) with one connecting line appears above a yellow box whose text is still being typed: "What are w".
    What neural network weights are, shown with a game
    1:06 · What neural network weights are, shown with a game · A dark video slide explains neural network weights. Game screens (Gameplay and Score 12, a Breakout-style game) feed into a layered network with four input nodes, two hidden layers of five, and three output nodes, ending in boxes with question marks labeled "answer". At left a table of numeric weights is shown, and at the bottom a row of "real weights" from W1 row 3.
  8. u8 Now, we already use these powerful LLMs to write prompts,
  9. u9 Weight-Space Programming explained
  10. u10 code for us. So, this paper asks whether a frontier model like GPT-6 Astra can read and write those weight numbers directly as well. This is what they mean by weight space programming. Where you give the LLM a goal in plain English along with the context it needs, and it changes the weights itself without any training.
    LLM receiving a prompt, with a user and code file
    1:34 · LLM receiving a prompt, with a user and code file · A dark video frame titled "What weights are" shows a cartoon robot labeled LLM with a dashed arrow pointing to a "prompt" card. The prompt reads "You are an expert editor. Fix the tone. Keep it short." A person icon labeled "US" sits below, and a dimmed "code.py" window is at the right, suggesting the scene is building toward contrasting prompts with code or weights.
    Frontier model GPT-6 Astra and reading neural network weights
    1:39 · Frontier model GPT-6 Astra and reading neural network weights · A dark video still in a hand-drawn style begins the sentence "What weights are" and shows a cartoon robot labeled "GPT-6 Astra" with an OpenAI-style logo. A cyan "frontier model" pill sits in the centre under a highlighted claim, "LLMs can natively read & write neural network weights", marked with a yellow question mark. Two document-like cards with squiggly lines sit at the right.
  11. u11 And this kind of sounds insane, but turns out Frontier models are actually pretty good at this. Okay, so let's start with reading because if a model can tell what a network
  12. u12 Reading weights: which game was this network trained on?
  13. u13 does from its numbers alone, you could check it for hidden problems without ever running it. In the first test, they trained a small network on four different games, all with the exact same shape, and then pasted every weight of the network into the prompt as plain text.
    Reading weights: an observer inspecting a neural network's weight matrix
    2:09 · Reading weights: an observer inspecting a neural network's weight matrix · A dark-background video frame titled "Reading weights" shows an eye looking down through a triangle at a 6x5 grid of numeric weights. At right, a thought cloud with a small neural network and an arrow (output blank) comes from a robot labeled "GPT-6 Astra" with an OpenAI-style logo. A text bar below shows the policy's 17,253 weights serialized as text, starting with row 3 of W1.
  14. u14 Then they asked Astra which game it was trained on with no tools and no list of games to pick from. Now, the interesting thing was that Astra named the right game for every single network. For Breakout, it noticed that the player weights uh were strongest along the bottom row uh where a paddle would be, and a scoring weight was bunched near the top where the bricks would be.
    Reading weights: GPT-6 Astra playing Breakout, with weight grid
    2:45 · Reading weights: GPT-6 Astra playing Breakout, with weight grid · A dark-background video frame labeled "Reading weights" shows a Breakout game screen (green bricks, red ball, yellow paddle) tagged "Breakout 99%". At the bottom left is a smiling robot icon with an OpenAI-style logo, labeled "GPT-6 Astra". At right is a 10x10 grid of purple squares of varying brightness with a cyan bottom row, labeled "player" with the value 4.066.
  15. u15 Now, the prompts did tell Astra what each input was. So, it knew where to look. So, at least on these small networks, reading works surprisingly well. But the bigger question is whether it can write them. Okay, now let's look at writing, and this is where it gets kind of wild.
  16. u16 So, for three small Atari style
  17. u17 Writing a Breakout network from zero weights
  18. u18 games, they gave Astra the rules, the game's source code, and a network where every single weight was zero. And with no training and no tools, it had to write every number in one reply. So, in Breakout, the network it wrote scored many times higher than the best network they trained with reinforcement learning, which is uh where a network plays the game or and learns from its own score.
    Writing policy network weights for small Atari-style games
    3:16 · Writing policy network weights for small Atari-style games · A dark video slide labeled "Writing weights" shows three sketched game screens: Breakout, Seaquest and a third, cut off as "Asteri" (Asteroids). A caption names them "small Atari-style games", and a banner states the task: write, from scratch, the weights of a policy network.
  19. u19 Now, the way they did it is the most important idea in the paper. Astra wrote a small program where the first layer predicts where the ball
  20. u20 Programs compiled into weights
  21. u21 will land, and the second layer moves the paddle there. And then it turned that program into numbers. So, a good mental model for weight and space programming is that Astra or any frontier model writes a program uh it already knows and then compiles it into weights.
    Neural network layer predicting ball landing in Breakout
    3:52 · Neural network layer predicting ball landing in Breakout · A dark-themed video frame labeled "Writing weights" explains a key idea: a two-line program predicts where the ball lands and moves the paddle there. A small neural network with the hidden layer (layer 1) highlighted sits beside a Breakout-style game showing bricks, a paddle, and a ball with a dashed bounce trajectory. A caption says the first layer computes four landing-position errors and two direction indicators, with predictions including wall reflections.
  22. u22 I think reading the source code is a big part of why it won by so much. Since the reinforcement learning network had to learn the rules uh from the score alone. But, in SeaQuest, the trained network still won. So, knowing the rules helps a lot but doesn't guarantee a better player.
    Astra writing game code versus PPO reinforcement learning on Breakout
    4:24 · Astra writing game code versus PPO reinforcement learning on Breakout · A split slide compares Astra, which writes a program (game.py) and scores 517.4 on Breakout, with PPO reinforcement learning, which adjusts neural network weights from only a score signal. PPO reaches just 15.8 after 100M steps and 25.3 after 500M, so Astra is about 20x higher. The right side shows a neural network trained on 500M steps with "score only" feedback and no access to the rules.
  23. u23 Now, every one of these networks is tiny. And the headline result is about the actual real language model. So, let's talk about the headline experiment. They took a small coin reasoning model with less than billion parameters and gave Astra a Python notebook with its weights plus practice
  24. u24 The Qwen3.5-0.8B math experiment
  25. u25 math problems it could run the model on to get a score. But, it was not allowed to train it. Now, the first thing Astra noticed is that most of the time coin never finished its answers. It would start reasoning then repeat the same few lines over and over until it hit the limit on how much text it was allowed to produce.
    The Qwen edit: Astra raises accuracy from 7.9% to 48.1%
    4:54 · The Qwen edit: Astra raises accuracy from 7.9% to 48.1% · A dark video slide shows a bar chart in which a base Qwen3.5-0.8B model scores 7.9% and the version edited by GPT-6 Astra scores 48.1%. The caption says Astra edits the weights in a Python kernel with no training. A notebook snippet and a stack of layers illustrate the 0.8B-parameter model's weights being modified directly.
  26. u26 So, basically, the model was stuck in the reasoning loop. And we have seen this pattern with small language model uh especially with reasoning capabilities that they can go into these long reasoning loops and basically use all of their tokens in thinking.
  27. u27 So, Astra compared the model's internals on answers that kept making progress with answers that looped. Now, a direction inside the model is one specific mix of internal numbers that shows up together for one behavior. And Astra found that
  28. u28 Finding the loop direction inside Qwen
  29. u29 showed up right before the model repeated itself. Then, it used a very well-known interpretability trick where you basically remove that mix from the two layers that weighted the most. So, the thing that is new actually in the paper is that the Astra found where to apply this interpretability trick uh on its own.
    The Qwen edit: steering Qwen3.5-0.8B layers toward progress versus loops
    5:49 · The Qwen edit: steering Qwen3.5-0.8B layers toward progress versus loops · A dark-background hand-drawn diagram shows a stack of model layers for Qwen3.5-0.8B in the center. A green "progress" text block ending in a checked "answer" points into the stack from the left. An orange "loops" block with a repeat arrow points in from the right. Below, a bar chart of "internal numbers" has yellow bars trending upward along an arrow labeled "one direction".
  30. u30 So, after this edit, uh the model scored six times higher on the math benchmark, and its other scores went up a lot as well. So, that is the headline everybody's covering, but let's talk about the reality behind it because uh if you go through the logs, the picture changes quite a bit.
  31. u31 And this is where you need to pay a lot of attention to not only the headlines, but the actual substance. The first thing I looked at is
    Reality check: verifying an AI news headline against GitHub logs
    6:36 · Reality check: verifying an AI news headline against GitHub logs · A dark video slide labelled "Reality check". On the left, an "AI news" card shows a MATH-500 score rising from 7.9% to 48.1% (~6x) for Qwen3.5-0.8B. In the center is a GitHub folder list of experiment logs, with exp8_math circled and exp4_fix_car under a magnifying glass. On the right are three questions to ask of the claim.
  32. u32 Reality check: where the 6x comes from
  33. u33 uh which answers the edit fixed. So, for more than 1,000 practice problems, almost every answer it had rescue had hit that limit before. Uh we're talking about the um the model getting into the loop and not able to finish the actual answer.
  34. u34 And actually, for the questions where the original model was able to finish, it was already right almost every single time. So, the real benchmark shows uh the same thing because the original model hit the limit on almost every problem, and after the edit, that hitting the limit dropped to almost a third.
    Reality check: Qwen3.5-0.8B results after edited forward passes
    7:09 · Reality check: Qwen3.5-0.8B results after edited forward passes · A dark video slide shows a table comparing the base Qwen3.5-0.8B model with versions edited using 10, 25 and 50 forward passes on MATH-500, GSM8K, IFEval and the share of outputs hitting the token limit. Only the 50-pass edit gives a big gain: MATH-500 rises from 7.9% to 48.1%, and token-limit hits fall from 93.0% to 33.1%. The "Hit token limit" column is highlighted, with three side prompts ("what got fixed?", "fair baseline?", "does it hold up?") and an empty box and a practice grid below.
  35. u35 So, what the edit did is to actually stop the model from looping, uh so it could finally finish answers it already knew how to solve. And that is where most of the uh six times improvement comes from. And this is where this one specific line from uh Quinn's model card is critical.
  36. u36 Quinn runs this model in non-thinking mode by default, and it warns that in thinking mode, uh where the model writes out its reasoning first, it is more prone to getting stuck in loops than the other Quinn 3.5 models. The reason is that
  37. u37 What Qwen's model card warns about
  38. u38 it's a very small model. And uh the paper ran every test in thinking mode, which is exactly what Quinn warns about this. It also used a lower temperature than Quinn recommends, which is uh a setting that controls how random the word choices are. And a separate research found that reasoning models loop more at lower temperatures.
    Reality check: Qwen3.5-0.8B thinking loops and baseline
    7:54 · Reality check: Qwen3.5-0.8B thinking loops and baseline · A dark video slide quotes the Hugging Face model card for Qwen3.5-0.8B, which says the model is more prone to thinking loops in thinking mode than other Qwen3.5 models, and that looping is common at low temperature. A checklist at right shows step 1 done and step 2, "fair baseline?", highlighted, with a MATH-500 baseline of 7.9% marked with a question mark.
  39. u39 So, you actually need to uh, read through the entire paper to understand what exactly uh, the improvements are and where these headline results are coming from. Now, another interesting point is that Astra doesn't really train the model, right? There are no gradients that can be used as feedbacks, but that does not mean that Astra is not looking at any feedback whatsoever.
  40. u40 They actually had some practice problems where Astra was able to make edits again and again, and that is actually a form of feedback that goes into a frontier model. So, I would describe this as a search mechanism
  41. u41 No gradients, but still feedback
  42. u42 where a very smart frontier model proposes edits, then scores them, and then keeps what works. Now, a few other things to look at in this paper. So, the actual improvements are coming from the runs that were when was able to finish, and for any other runs, it made smaller gains or no edit at all.
    Reality check: search loop with GPT-6 Astra
    8:51 · Reality check: search loop with GPT-6 Astra · A dark-background video still showing a hand-drawn cycle of propose, edit, score and keep, labeled "search", beside a robot icon tagged "GPT-6 Astra" with "no gradients" and "feedback". On the right is a three-item checklist (what got fixed?, fair baseline?, does it hold up?). The first two are ticked and the third is highlighted. A small chart below shows practice accuracy against forward passes used.
  43. u43 And I think they also ran the experiments only once, so I think it would be great to run this multiple times and see. Now, the authors also tried
  44. u44 One run hit 48%, and each ran once
  45. u45 this on a small Gemma model that really loops, and the edits barely moved its math score. It's a completely different model, uh, so it's not a perfect comparison, but they draw the same conclusion that editing words could remove an obstacle that hid an ability, uh, but couldn't add a missing one.
  46. u46 And that is very important distinction to keep in mind. The frontier model is not adding any new capabilities, it's
  47. u47 Gemma: unblocking vs adding a skill
  48. u48 just unlocking some of the capabilities that the model already has, but because of some behavior they are blocked. Now, let's look at some limitations because it's important to keep them in mind when we are looking at uh, these research papers.
  49. u49 And there are two practical rules I would take away from this paper. The first one is that when a small reasoning model scores badly, I would highly recommend um counting how many of its answers never reach the final answer before we actually change anything. If most of them are cut off, you have a looping problem.
  50. u50 And the fix might be the mode, maybe the temperature, or a bigger output generation limit.
  51. u51 Two rules before you trust a jump
  52. u52 Now, the second one is that I would not expect weight editing to make a weaker model smarter because in this paper it removed a blocker that was present, but never added a new skill. Now, all that being said, it's still pretty impressive that these frontier models are able to read and write uh the weights of small networks.
    Small model answers cut off by looping problem
    10:35 · Small model answers cut off by looping problem · A dark video slide titled "Two rules" highlights rule 1, "count cut-off answers". A grid of 96 answers from a small model shows only 11 finished (green) and 85 cut off (red) because of a looping problem at a 32,768 limit. The fix shown is a mode toggle, temperature 1.0 and a raised output limit of 81,920.
  53. u53 They're able to do interpretability team exploration, figure out which part of the network is responsible for what, which is essential for understanding how these networks actually work. So, this is a very interesting area of research. I am also putting together a video on testing this edit on my own local uh setup.
  54. u54 So, if you want that, uh make sure to subscribe to the channel. Now, if you want to learn more about how AI systems
  55. u55 Why it still matters
  56. u56 can improve themselves without any training, watch this video. Anyways, I hope you found this video useful. Thanks for watching, and as always, see you in the next one.

The original

Around it

weight orthogonalizationweb-61d8f4540a88temporally correlated errorstemperature (sampling)risk aversion from hardness of learningrefusal in large language modelsrefusal directionknowledge distillationgreedy decodingconfidence buildup during loopsarxiv-1909.05858arxiv-1905.07830arxiv-1803.05457arxiv-1801.10198adversarial suffixSycophancyProximal policy optimizationGemma 3 270MWait, Wait, Wait... Why Do Reasoning Models Loop?Refusal in Language Models Is Mediated by a Single DirectionMATH-500Directional ablationWeight-space programmingRepetitive reasoningQwen3.5-0.8BGPT-6 AstraSparks of Weight-Space Programming: LLMs can natively read & write neural network weightsGPT-6 Astra Made Qwen 6x Better at Math Without TrainingWhere GPT-6 Astra's math gain on Qwen…GPT-6 Astra edits another network's…GPT-6 AstraQwen3.5-0.8BRepetitive reasoningWeight-space programmingDirectional ablationMATH-500Refusal in Language Models Is…Wait, Wait, Wait... Why Do…Gemma 3 270MProximal policy optimizationSycophancyadversarial suffixarxiv-1801.10198arxiv-1803.05457arxiv-1905.07830arxiv-1909.05858confidence buildup during…greedy decodingknowledge distillationrefusal directionrefusal in large language…risk aversion from hardness…temperature (sampling)temporally correlated errorsweb-61d8f4540a88weight orthogonalization

source

Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights

The authors argue that frontier LLMs can natively read, repair and write neural network weights from natural-language goals, a practice they call Weight-Space Programming. Across eight experiments with GPT-6 Astra, the model identifies what networks were trained to do, fixes a failing control policy, writes game-playing networks from scratch, and improves Qwen3.5-0.8B's math by editing 0.016% of its weights. They present these as early results with clear limits.

Article1 source

topic

GPT-6 Astra

A language model used to inspect, construct and edit other neural networks.

Topic2 sourcesNo primary research yet

topic

Qwen3.5-0.8B

A small language model in the Qwen3.5 family.

Topic2 sourcesNo primary research yet

topic

Repetitive reasoning

A failure mode in which a reasoning model repeatedly revisits the same thoughts without progressing to a final answer.

Topic3 sourcesNot borne out by two independent sources

topic

Weight-space programming

Weight-space programming directly reads or constructs neural-network weights to interpret or implement behavior rather than relying on conventional training.

Topic2 sourcesNo primary research yet

topic

Directional ablation

An interpretability technique that removes a chosen direction from a neural network's representations or weights to suppress its contribution to model behavior.

Topic2 sourcesNot borne out by two independent sources

topic

MATH-500

MATH-500 is a benchmark of 500 mathematical problems used to evaluate mathematical problem-solving performance.

Topic1 sourceSingle source

source

Refusal in Language Models Is Mediated by a Single Direction

Refusal of harmful instructions in open-source chat models is controlled by a single direction in the residual stream. Erasing this direction stops refusal, and adding it induces refusal on harmless prompts. The same finding enables a rank-one weight edit that works as a white-box jailbreak, and it explains how adversarial suffixes suppress refusal.

Paper

source

Wait, Wait, Wait... Why Do Reasoning Models Loop?

Reasoning models get stuck repeating text in long chains of thought, most often under greedy decoding at low temperature. The source attributes this largely to errors in learning, not only to model size or decoding randomness. It supports this with tests on open models and with two mechanisms, risk aversion from hardness of learning and temporally correlated errors, shown on a synthetic graph task.

Paper

This text is written with AI.

Placed in: Now