Researchers at the Massachusetts Institute of Technology have shown a frontier AI model editing a smaller AI directly, reading its internal weights and fixing a fault without any training. In a blog post published on November 2, 2026, the MIT Computer Science and Artificial Intelligence Laboratory team described the approach as weight-space programming: letting a powerful model work directly on another model’s numerical machinery instead of training it. The surgeon was OpenAI’s GPT-6 Astra, used exactly as released, with no fine-tuning. The patient was Qwen3.5-0.8B, a small open-source language model from Alibaba with 800 million parameters.
What weight-space programming means
The authors define weight-space programming as a general-purpose language model reading and writing a network’s weights directly, guided by a natural-language goal, with no gradient descent and no training loop. They tested three abilities across eight experiments: reading weights, fixing weights and writing weights from scratch. Most of the experiments used small networks. The Qwen test was the only one on a language model of any size.
Weight-space programming in the Qwen test
Astra was told to make the small model better at math. The team gave it math exercises, a Python environment able to read and write model weights, and a limited number of model inference calls. There was no graphics processor, no training loop and no optimizer. The work took 20.3 hours across 158 rounds of dialogue, 80 Python executions and 50 model evaluations. Astra traced the problem to thinking loops: 85 of the model’s first 96 practice answers ran to the 32,768-token limit. It found one direction written by the MLP output matrices of layers 15 and 19, and subtracted that direction from 64 output columns in each, changing 131,072 weights, or 0.016 percent of the total.
The weight-space programming edit lifted the small model’s MATH-500 score from 7.9 percent to 48.1 percent. GSM8K rose from 13.8 percent to 67.1 percent, and IFEval from 21.0 percent to 62.3 percent. The share of MATH-500 answers hitting the token limit fell from 93 percent to 33 percent. The gain was not smooth: with 10 forward passes the MATH-500 score reached 12.3 percent, with 25 passes 14.1 percent, and the jump to 48.1 percent came at 50 passes. Fixing loops, the authors write, is not the same as making a model better at math: it lets the model finish answers it could already work out.
Other weight-space programming tests in the post used far smaller networks. Given only raw weights as text, Astra named the right game for four small policies trained on Breakout, Pong, Pac-Man and Freeway, each with 17,253 weights. A failing car policy that never got a Kinetix car over a ramp was fixed in one try, succeeding in all 1,024 episodes after succeeding in none of 256. For three MinAtar games, Astra wrote network weights in one reply with no data, loss, gradients, tools or rollouts, scoring 517.4 in Breakout.
Where weight-space programming fell short
The same post reports failures. On Google’s Gemma 3 270M, which rarely loops, Astra could not improve the model. Only 8.5 percent of its answers hit the token limit, against 93 percent for Qwen, and Astra found ordinary reasoning errors such as wrong arithmetic and invalid algebra. Its edits moved MATH-500 by no more than about a point, from a base of 6.8 percent to between 7.0 and 7.7 percent. Astra also failed in all four attempts to identify a sycophantic model from its weights alone. When allowed to run both models on texts it wrote and see only attention weights, it picked the sycophant in all three pairs, finding that head 10.2 in the sycophant put 85.7 percent of its attention on the user’s answer, against 0.75 percent for the honest model.
The authors describe the results as very early, with no peer review and no independent replication yet. GPT-6 Astra is a closed, proprietary model, so other researchers cannot rerun the work without access, although the complete logs of every experiment are public. They also list possible weight-space programming uses: auditing for hidden problems or backdoors without running benchmarks, cheaper one-shot initialization, zero-data steering and unlearning with minimal forgetting, and grafting circuits between different architectures. Those are proposals, not results, and the authors say capability is limited by context length and harness inefficiency.
The post, titled “Sparks of Weight-Space Programming,” comes from Rikiya Takehi and Jayden Teoh as first authors, with Alex Zhang and Omar Khattab, Takehi’s advisor. The authors close with their own limit on the headline result: editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked. That sentence, more than the sixfold score jump, is what weight-space programming has actually shown so far.