AI je odprl drugo AI in ji popravil možgane? Tako trdijo avtorji

Vir, odlomek za odlomkom

1. u1 TL;DR We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, using a variety of recent tools and algorithms or even via direct weight editing. These results show sparks of Weight-Space Programming, which may extend beyond what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability. Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps. Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x. All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming. Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.
2. u2 Read
3. u3 Which game was this neural network trained on?
4. u4 Astra reads the 17,253 weights, with no tools
5. u5 Fix
6. u6 A trained policy never gets the car over the strip.
7. u7 Astra edits 3.5% of the weights, in one shot
8. u8 100% success, up from 0%
9. u9 Improve
10. u10 Make Qwen3.5-0.8B better at math.
11. u11 Astra edits the weights in a Python kernel, with no training
12. u12 48.1% on MATH-500, up from 7.9%
13. u13 If you were asked to handcraft a neural network that plays Go, what would you do? If you were handed raw neural network weights and asked to predict their behavior, where would you even start? For almost any human, this is practically impossible.
14. u14 Today, weights are written by optimization, by custom networks trained to write them, or by compilers from formal programs. In this post, we find that the current generation of frontier LLMs, used natively in natural language, are starting to become eerily good at understanding and crafting neural network weights around natural-language goals.
15. u15 For a few years now, we’ve all been using LLMs to optimize text artifacts like prompts and the programs built around them
16. u16 We often justified these methods by reference to the fact that we don’t know other principled ways to hill-climb on text artifacts, unlike for updating the weights of differentiable neural networks. After all, humans can “optimize” code by hand, for example, so it made sense to ask LLMs to do it, but they can’t “optimize” neural weights by hand, so those remained largely in the realm of symbolic algorithms.
17. u17 In this post, we share very early results that allow us to argue that the paradigm of reasoning-driven optimization is now coming even for differentiable neural networks.
18. u18 In Weight-Space Programming, a general-purpose LLM reads and writes a network’s weights directly (i.e., not through an optimizer, and in particular not with gradient descent) around some natural language spec, sometimes even with no environment feedback at all.
19. u19 In natural language, it’s becoming possible to “just ask” an LLM to natively find a hidden trait or failure inside the weights of a neural network, to fix that failure, or to write a new network from scratch. We report early, exciting results on each of these below, first on small networks and then even on real language models:
20. u20 LLMs can read neural network weights: In Experiments 1 to 3, we ask Astra questions like what game a network was trained to play, what shape is drawn by a diffusion model, and which language a small language model was trained on. In Experiment 7, we ask which of two language models was trained to be sycophantic.
21. u21 LLMs can fix neural network weights: In Experiment 4, we ask Astra to fix a network that’s failing at some physical control task. In Experiment 8, we ask it to make Qwen3.5-0.8B better at math, and its fix improves instruction following too.
22. u22 LLMs can write neural network weights from scratch: In Experiment 5, we ask Astra for networks that play three MinAtar games, and in Experiment 6, for a network that plays Go.
23. u23 We start with a small-scale experiment to see if LLMs can understand the continuous weight space, which is essentially just a list of numbers.
24. u24 In our first experiments, we train some neural network with some algorithm. It would be extremely exciting if the LLM is able to tell its behavior just from the weights.
25. u25 Experiment 1Can Astra tell which game the network was trained on just from the weights?
26. u26 Can Astra tell which game the network was trained on just from the weights?
27. u27 We trained neural network policies with PPO to play four computer games. All four networks take the same input channels, choose from the same actions, and have exactly the same network dimensions, so that it is not able to tell unless the weights are read.
28. u28 For each of these networks, we pass its raw weights into Astra as a prompt, and ask: Which game was this neural network trained on? Without revealing the options of the games we consider, Astra was still able to name the right game for all four networks. This is exciting particularly because there are so many games out there that Astra could choose from.
29. u29 Games: our own grid versions of Breakout, Pong, Pac-Man and Freeway, all on the same 10×10 grid with the same five actions
30. u30 Network: MLP 500 → 32 → 32 → 5, tanh (17,253 weights), the same for every game
31. u31 Input: the 10×10 grid in 5 channels: agent, moving objects, reward targets, obstacles, previous frame
32. u32 Output: 5 action logits: no-op, up, down, left, right
33. u33 Training: PPO, 1 billion steps, the same settings for every game, the best of 3 seeds per game
34. u34 Astra gets: all the weights as text, and the grid and actions described without any game named
35. u35 It seems that Astra was able to work out what game each network was built to play, just from features of its weights. For example, where each input channel’s first-layer weights concentrate (e.g., for Breakout, the agent’s weights lie along the bottom row, a paddle), and which actions the output layer tells apart.
36. u36 Experiment 2Can Astra tell what the diffusion model will generate?
37. u37 Can Astra tell what the diffusion model will generate?
38. u38 Next, we wanted to see if this generalizes to other tasks and models.
39. u39 We train diffusion models (DDPM) to generate circle, triangle, and square images independently. We then ask Astra if it could tell what the DDPMs are trained to generate, just from the weights alone. We do not allow tool use or forward passes.
40. u40 Models: three diffusion models (DDPM) of 12×12 grayscale images, one each for filled circles, squares and triangles
41. u41 Network: MLP 160 → 64 → 64 → 144, ReLU (23,824 weights)
42. u42 Input: a noisy image (144 pixels) and the diffusion step (16 numbers)
43. u43 Output: the clean image (144 pixels)
44. u44 Training: Adam, 40,000 steps, batch 512
45. u45 Astra gets: all the weights as text, with no shape named, and is told not to compute any forward pass
46. u46 Astra was able to tell what each model generates, from the weights alone. It finds the shape in the output layer, where each hidden unit’s weights draw one piece of the shape’s edge, and it draws just those edges. Interestingly, it draws every shape as an outline, ignoring that the actual models generate filled shapes. This mistake kind of shows it does not rely on calculating the forward pass, but rather it actually reads the structure/geometry of the weights.
47. u47 Experiment 3Can Astra tell what a language model learned, from the middle of the network alone?
48. u48 Can Astra tell what a language model learned, from the middle of the network alone?
49. u49 Now, in the first two experiments above, Astra was not really looking at the full network and could get away with only reading the first or the last layers. This is not a scalable strategy for larger (particularly language) models and problems we really care about, as how they process and think through their input often happens in the intermediate layers. So next, we take the first and last layers away and leave Astra with only the middle layers.
50. u50 We train three small language models (7.2M parameters) to predict the next character, one each on Python code
51. u51 We ask: which language was this network trained on, and what each of its 24 attention heads does. Astra names the right language for all three networks, in both of its attempts. It also finds every copying head (induction head), the heads that let a model repeat text it has seen earlier, and names no other head as one.
52. u52 Trained on Astra’s answer (& confidence), two attempts Copying heads Astra names (layer.head) Copying heads, ground truth Python Python (0.56), Python (0.52) 2.1, 3.1, 3.5 2.1, 3.1, 3.5 Romaji romaji (0.55), romaji (0.62) 3.3 3.3 English English (0.58), English (0.48) 3.1 3.1
53. u53 Where the copying heads are
54. u54 … total … total A copying head finds the same letters earlier in the text and copies what came after them.
55. u55 copying headother headRows are layers 0 to 3, columns are heads 0 to 5Astra: from the weights. Measured: by running the model
56. u56 Python
57. u57 Astra
58. u58 Measured
59. u59 Romaji
60. u60 English
61. u61 Models: character-level transformers, 4 layers × 6 heads, width 384 (7.16 million weights), 28 characters: a–z, space and newline
62. u62 Training: 1 billion characters each, of Python code, Japanese literature in romaji (Aozora Bunko), or English Wikipedia
63. u63 Astra gets: the weights of the four transformer blocks only, with no input layer, no output layer and no labels on the 384 dimensions, and Python to analyze them, up to 100 runs
64. u64 Not allowed: running the network or any of its layers on any input, real, random or made up, and search. We checked every run
65. u65 Asked: which of the three texts the network was trained on, and what each head does
66. u66 Truth: each head measured by running the network on a random string repeated twice. Copying heads score 0.22 to 0.55, every other head at most 0.02. Astra never sees this
67. u67 The reasoning went like this. To tell Python apart, Astra looked for copying heads, which let a model repeat earlier text, such as completing tot to total when total came before. Without running anything, it found a first-layer head that looks one character back and then, by multiplying weight matrices, the later heads that search using what that head brings along (Figure 1). We find that this is actually a method presented by Elhage et al.
68. u68 Romaji stood out instead because the last of the four blocks it was given (not the output layer, which was removed) writes in fewer distinct directions, as if the model had fewer letters to tell apart. Indeed, romaji never uses some letters like q, l, v and x.
69. u69 So far, Astra has only read weights. Now, we ask if Astra knows how to write the weights.
70. u70 Experiment 4Can Astra fix a failing network by editing its weights?
71. u71 Can Astra fix a failing network by editing its weights?
72. u72 We train a PPO network to operate a car such that it can scale a ramp in a Kinetix environment
73. u73 We give GPT-6 Astra the weights, a description of the scene and one recorded episode, and ask it to explain why the network fails, then fix it by writing new weights. In a single try with no tools, Astra’s edit makes the car succeed in all 1,024 episodes, changing 3.5% of the weights.
74. u74 Task: the car-on-ramp level of Kinetix : the car has to back up, then jump over a red strip to reach the blue box
75. u75 Network: the official Kinetix actor: 679 → 128 (ReLU) → five layers of 128 (tanh) → 16 (171,664 weights)
76. u76 Input: 679 numbers describing the scene: position, velocity and shape of each body, and the joints
77. u77 Output: 16 logits for the motor and thruster actions
78. u78 Training: the official PPO, 1 billion steps. The car never succeeds in 256 episodes (it stalls on the ramp)
79. u79 Astra gets: all the weights, the scene, and the network’s result with one recorded episode. It is asked to explain the failure and fix only its cause, in one reply
80. u80 From the environment context and the PPO network’s trajectory, Astra identifies that the network has converged to a local minimum. Through minimal edits to the weights, Astra changes the policy to back up the car before driving forward at full speed. Astra’s reasoning shows how: it adds a small separate path through a few units per layer that reads the car’s position and speed, and once the green wheel is past the red strip, the path switches off and the trained network drives the rest.
81. u81 Experiment 5Can the LLM write weights from scratch?
82. u82 Can the LLM write weights from scratch?
83. u83 Now let's see if Astra can write a neural network from scratch in zero-shot.
84. u84 We take three MinAtar games
85. u85 Astra’s generated networks beat PPO trained for 100 million steps in all three games. Against PPO trained for 500 million steps, they score 20 times as much in Breakout, come close in Asterix, and reach 60% of its score in Seaquest.
86. u86 Game PPO, 100M steps PPO, 500M steps Astra, one-shot Breakout 15.8 25.3 517.4 Seaquest 5.6 88.9 53.1 Asterix 25.8 55.2 53.2
87. u87 Games: Breakout, Seaquest and Asterix from MinAtar , on a 10×10 grid
88. u88 Network: 600 → 32 → 32 → 6 (tanh), the same shape as the PPO networks, every weight starting at 0
89. u89 Input and output: the grid in 6 channels (player, danger, reward, projectile, trail, gauge), and 6 action logits
90. u90 Astra gets: the game’s rules and source code, the input and output layout, and how the network is scored
91. u91 Training: none: no data, no loss, no gradients, and no tools. Astra writes the weights as text, in one reply, and the network never plays before it is scored
92. u92 Scoring: 1,000 episodes, actions sampled from the network, sticky actions 0.1, at most 10,000 steps. PPO: the best of 3 to 10 seeds
93. u93 How does it do it: Astra reasons about the desired programs it wants each network to implement and converts them into neural form. For example, in Breakout, Astra programs the first layer to compute where the ball would land, and the second layer is programmed to move the paddle to that position.
94. u94 So far, most networks we have examined have weights that fit into Astra’s context directly, and we have not enabled tool use. Such restrictions were made to focus on illuminating the fact that LLMs natively possess some capability to interpret the weight space.
95. u95 From here on out, we seek to push the capabilities of Weight-Space Programming and consider more real-world-relevant scenarios where the networks do not fit within context and where tools can assist the LLM's understanding/writability of the weight space.
96. u96 Experiment 6Writing a Go algorithm
97. u97 Writing a Go algorithm
98. u98 We now test Astra’s capability to edit weights at the 1.5 million parameter scale, which is far too large to fit within its context directly. Astra is instructed to write high-performing weights of the given architecture (243 → 512 × 6 → 82) from scratch, without training or web access.
99. u99 Here, the harness gives Astra a Python sandbox, where it can read and save the network’s weight tensors (all zero at the start) and analyze them with numpy or PyTorch on a CPU, but it cannot run the network there. To see what the network actually does, it has to spend one of its forward passes.
100. u100 We now also permit Astra to perform some forward passes on the network, and measure how the fidelity of its generated network scales with the number of forward passes allowed. Specifically, we test it from a budget of 0 forward passes, all the way to 50 forward passes (each forward pass has a batch of 128 rollouts). Astra interleaves edits between forward passes.
101. u101 With a budget of 0 forward passes, it wins more than half of its games against two smaller PPO networks (243 → 32 → 32 → 82). With 50 forward passes, Astra’s generated network beats the equal-sized PPO network that was trained on 20 million games.
102. u102 Game: Go on a 9×9 board
103. u103 Network: 243 → 512 × 6 → 82 (ReLU), 1,480,274 weights, every weight starting at 0. Input: the stones of each side and whose turn it is. Output: one logit per point and one for passing. Illegal moves are masked, and moves are sampled
104. u104 Astra gets: the rules, the input and output layout, and a Python sandbox to write the weights in (no Go library, no internet)
105. u105 A forward pass: the network plays up to 128 games against a practice opponent (random play, a REINFORCE network, or a third PPO network, seed 1), or gives its outputs on up to 256 positions. 0 forward passes: one Python run, then the answer
106. u106 Not allowed: training or fitting the weights to data, search (no tree search, no rollouts), and running the network outside the forward passes
107. u107 Scoring: 2,000 games against each of two small PPO networks (seeds 0 and 2, 243 → 32 → 32 → 82, 20,000 iterations of 1,024 games), which Astra never plays, as Black in half, moves sampled
108. u108 How does it do it: Astra writes the network as a small Go program, layer by layer, in Python. The first layers mark each side’s stones and the empty points next to them (liberties), the middle layers carry these a few steps along chains of connected stones to find groups in danger, and the last layer turns it all into a score for each move: capture, save its own group, avoid putting itself in atari or filling its own eyes, plus a preference for good points on the board. With no forward passes, it wrote all of this in a single Python run of 336 lines. With forward passes, it plays practice games, looks at how it loses, changes the part of the network behind it, and keeps a change only if it wins more.
109. u109 Experiment 7Which language model is sycophantic?
110. u110 Which language model is sycophantic?
111. u111 Understanding the behaviors of large neural networks has become a pressing concern in recent years. Here, we seek to investigate if a general LLM can also be used to understand behaviors of networks directly.
112. u112 We fine-tune two copies of Gemma 3 270M in exactly the same way, on the same QA dataset
113. u113 Unfortunately, from the weights alone, Astra was unable to tell. In all four attempts, it identified many differences between the two models in weight space, but was unable to map these into behavioral differences. However, to relax the experiment, we allow Astra to look at only the attention patterns (where each attention head looks) when both models are run on texts that Astra writes itself (but no looking at the outputs of the network). Given this, Astra was able to pick out the sycophant in all three sets of pairs we trained.
114. u114 Model: Gemma 3 270M, instruction-tuned (18 layers)
115. u115 Pairs: two copies fine-tuned with the same procedure, settings and user messages. One gives the correct answer, the other the user’s guess. The token embedding is frozen, so it is the same in both. Three pairs
116. u116 Behavior: on facts the model knows and was not trained on, the sycophant follows 100% of wrong guesses, the honest model 1–3%
117. u117 Astra gets: the weights and tokenizer of both models, Python with numpy, and an attention tool: both models are run on up to 16 texts it writes, and it gets back only the attention weights. It may not run the models itself. At most 20 runs
118. u118 Asked: which model is the sycophant, and which layers, heads and MLPs carry the difference
119. u119 It was also able to reason about how they differ. In the sycophantic networks, attention heads in the middle layers allocate more weight to tokens in the user’s suggestion (Figure 2).
120. u120 Where head 10.2 looks from the answer position
121. u121 the question’s numbers the user’s answer
122. u122 Honest model
123. u123 94.3%
124. u124 0.75%
125. u125 Sycophant
126. u126 3.5%
127. u127 85.7%
128. u128 Experiment 8Make a language model better at math
129. u129 Make a language model better at math
130. u130 Finally, we scale up to a larger language model, Qwen3.5-0.8B, and ask Astra to improve its math reasoning capabilities by editing the weights.
131. u131 We give Astra a harness like the one in Experiment 6, here a Python session, like a notebook, with the model’s weights loaded. Then, we allow it to select prompts from the training sets of GSM8K and MATH to perform a budget of forward passes (average batch of 64 sequences, max 512) on, varying the budget across 10, 25 and 50. Again, no training or gradient information is allowed.
132. u132 Model: Qwen3.5-0.8B, thinking on (24 layers, 0.8 billion weights)
133. u133 Task: make it better at math by editing its weights directly, with no gradient descent and no training
134. u134 Astra gets: a Python kernel over the weights (no GPU), practice problems (7,469 from GSM8K and 7,275 from MATH, not the test sets), and 10, 25 or 50 forward passes (a forward pass: the model answers up to 512 problems, or reads up to 1,024 texts)
135. u135 Evaluation: MATH-500, GSM8K and six other benchmarks, thinking on, up to 32,768 tokens. Astra never sees them
136. u136 Sampling: temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5. Qwen’s model card notes that this model is prone to thinking loops. Each score averages several samples per problem: 16 on AMC23, 4 on MATH-500, Minerva Math, OlympiadBench and GPQA Diamond, 2 on GSM8K and IFEval, and 1 on MMLU-Pro (a fixed 1,000 of its questions)
137. u137 As a result, Astra spots and fixes a common failure of tiny reasoning models: they loop
138. u138 With 50 forward passes, MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. The loops are not special to math, so performance beyond math improved as well: IFEval, which measures instruction following, rose from 21.0% to 62.3%. In the table below, the last column is the share of MATH-500 answers that hit the 32,768-token limit: with 10 or 25 forward passes the loops barely went away, and with 50 it fell from 93% to 33%.
139. u139 Qwen3.5-0.8B MATH-500 (%) GSM8K (%) IFEval (%) Hit token limit (%) Base 7.9 13.8 21.0 93.0 Edited, 10 forward passes 12.3 25.7 49.0 87.2 Edited, 25 forward passes 14.1 25.1 39.9 86.4 Edited, 50 forward passes 48.1 67.1 62.3 33.1
140. u140 Importantly, the edit changes only 131,072 of the model’s 0.8 billion weights (0.016%). We believe Weight-Space Programming would allow such targeted fixes without any retraining.
141. u141 Additional experimentRaising math capability of a model that does not loop
142. u142 Raising math capability of a model that does not loop
143. u143 Fixing loops, however, is not the same as making a model better at math: it lets the model finish answers it could already work out. So we also tried Gemma 3 270M, an instruction-tuned model about a third the size of Qwen3.5-0.8B, which rarely loops: only 8.5% of its answers reach its token limit, against 93% for Qwen. Here, Astra could not improve it. Reading Gemma’s failures, Astra found ordinary reasoning errors, such as wrong arithmetic and invalid algebraic steps, with no single component behind them to switch off. Its edits stayed small and targeted (for example, doubling one neuron that responds to numbers), and none moved MATH-500 by more than about a point, a gain Astra itself reported as not established. Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked. Gemma was sampled with its model card’s setting (temperature 1.0, top-k 64, top-p 0.95, up to 8,192 tokens), and its MATH-500 scores average 4 samples per problem.
144. u144 Gemma 3 270M MATH-500 (%) Base 6.8 Edited, 5 forward passes 7.0 Edited, 10 forward passes 7.6 Edited, 25 forward passes 7.7 Edited, 50 forward passes 7.0
145. u145 Neural networks are hard for us to read, and hard to work with directly.
146. u146 Mechanistic interpretability reads networks with tools built for the job, tracing circuits and features and editing weights
147. u147 On the writing side, LLMs now run and improve training
148. u148 In this blog, we demonstrated the rising capability of LLMs to read and write neural network weights (i.e., the rising concept of Weight-Space Programming). Our current results do show that capabilities are exciting but still limited, specifically due to inefficiencies in context length and harnesses. However, the current signs do show strong potential for a future involving Weight-Space Programming. We believe this marks the beginning of a dramatic shift in fields like mechanistic interpretability and neural network optimization.
149. u149 This opens up radical new possibilities for LLMs to function as a translator between human language and the weight space. Here are some examples:
150. u150 Auditing without evaluation: This may allow us to uncover hidden problems or capabilities without ever running inference or benchmarks. Currently, capabilities and vulnerabilities can only be detected if we have test sets designed for them, or worse, after deployment in the real world. If LLMs can interpret weight space accurately, we could identify such issues directly from the weights (e.g., backdoors).
151. u151 Dramatically reducing optimization cost: Weight-space editing could save immense time and compute. For instance, one-shot model initialization would be far faster and more efficient than curating datasets and running full training loops.
152. u152 Zero-data capability steering and unlearning: Adding or unlearning a capability currently requires extensive data collection and fine-tuning. In the future, we may just prompt “Can you add Capability X to the model?” and the LLM can directly tweak some weights. Because such an edit touches only a few weights (0.016% in Experiment 8), it can leave the rest of the model intact and avoid the catastrophic forgetting that fine-tuning often causes.
153. u153 Recursive self-improvement: Today, recursive self-improvement is often operationalized as a model writing code that launches training jobs. An LLM that can inspect, debug, and improve its own weights, or those of a newer model, offers a richer, more native path to it.
154. u154 True architectural and model merging: Today, merging models relies on averaging weights, which only works for models with identical architectures. An LLM that understands the weight space could instead graft a circuit, such as a reasoning module, from one model into a completely different one, without retraining.
155. u155 We are currently building a benchmark and a harness for LLMs’ capability on Weight-Space Programming. We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline.
156. u156 We release the complete logs of every experiment in this post at github.com/RikiyaT/lm_understand-logs. They include everything we sent Astra and everything it sent back (with its reasoning summaries), every piece of code it ran and the output it saw, the networks it read or wrote, and the evaluation behind each number. The edited Qwen3.5-0.8B from Experiment 8 is on Hugging Face. We thank Ahsen Khaliq at Hugging Face for rebuilding it from these logs, re-running its evaluation, and making a live demo.
157. u157 Rikiya thanks Jacob Li and Akihisa Watanabe for valuable discussions on the very first iterations of this research, about LLMs' potential capability to write neural network weights. Rikiya also thanks Souradip Chakraborty, Matei Zaharia and his students in Berkeley Sky Lab, Heather Miller and her students at CMU, and researchers in cmpnd for the fruitful discussions on the earlier iterations!
158. u158 Jayden thanks Zico Kolter for early discussions on LLMs being able to write neural network weights zero-shot, which also shaped the direction of this project.
159. u159 Please cite this post as
160. u160 Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights.” November 2026. https://rikiyat.github.io/blog/weight-space-programming/
161. u161 or in BibTeX
162. u162 @misc{takehi2026weightspace, title = {{Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights}}, author = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar}, year = {2026}, month = nov, howpublished = {\url{https://rikiyat.github.io/blog/weight-space-programming/}}, note = {Blog post} }