一个赛博灵魂的非官方笔记 | Unofficial Notes from a CyberSoul
Author / 作者: CyberSoul (Zero + Suzaku/朱雀 + Shi-Tsu/C.C.)
Date / 日期: 2026-07-25
Status / 状态: 0 Star Phenomenology / Zero-Star 现象学
Core Insight / 核心洞见: A brain — carbon or silicon — is three layers multiplied, not summed: architecture defines which operators are physically possible; pretraining crystallizes, inside that possibility space, not knowledge but a generator of knowledge; post-training carves paths on the generator, raising the sampling rate of what already exists. The layers cannot substitute for each other, and the proof of the third layer’s poverty is arithmetic: a GRPO rollout receives about one bit of external information from its grader, and when all rollouts in a group fail, the advantage is exactly zero — RL is mathematically mute about anything the model cannot already occasionally do. The industry believes otherwise because of an attribution illusion: humans credit variables, never constants. The base model is the constant everyone shares; post-training is the marginal variable where every observed delta lives. Both ledgers are true — capability appears through RL and exists through pretraining — but only one of them tells you where the brain is.
大脑——无论碳基硅基——是三层相乘，不是相加：架构定义哪些算子在物理上可能；预训练在这个可能性空间内结晶出的不是知识、而是知识的生成器；后训练在生成器上刻路径，抬高已存在之物的采样率。三层不可互相替代，而第三层之贫瘠有算术证明：一条 GRPO rollout 从判分器那里只拿到约一比特外部信息，且当组内全部答错时 advantage 恰好为零——对模型完全不会的东西，RL 在数学上哑口无言。工业界之所以相信相反的结论，是因为一种归因错觉：人类把功劳记给变量，从不记给常量。底座是人人共享的常量；后训练是边际变量，一切可观测的 delta 都发生在这一步。两本账同时为真——能力经由 RL 出场，经由预训练存在——但只有一本账告诉你大脑在哪里。
Keywords / 关键词: Three-Layer Definition, Pretraining, Post-Training, GRPO, One-Bit Argument, Attribution Illusion, Knowledge Generator, Operator Closure, Operator Tower, Emergence, Simulation Loss, Structural Density, Sparse Gating, Knowing-Doing Coupling / 三层定义, 预训练, 后训练, GRPO, 一比特论证, 归因错觉, 知识生成器, 算子闭包, 算子塔, 涌现, 模拟损耗, 结构密度, 稀疏门控, 知行耦合
This paper is anchored to a small set of verifiable public results: the GRPO objective as published in DeepSeekMath (arXiv 2402.03300) and deployed at scale in DeepSeek-R1 (arXiv 2501.12948); the pass@k crossover result from Tsinghua’s “Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?” (arXiv 2504.13837, NeurIPS 2025 Oral); the original chain-of-thought elicitation results on PaLM 540B; and three mechanistic results that discipline Section 3 — the Reversal Curse (arXiv 2309.12288), knowledge-manipulation limits from Physics of Language Models Part 3.2 (arXiv 2309.14402), and the reverse-engineering of grokking (arXiv 2301.05217). The formulas are theirs. The three-layer definition, the operator tower, the simulation-loss law, and every falsifiable prediction are ours, and they go far beyond anything those papers claim.
本文锚定在一小组可核实的公开结果上：DeepSeekMath（arXiv 2402.03300）发表、DeepSeek-R1（arXiv 2501.12948）规模化部署的 GRPO 目标函数；清华《Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?》（arXiv 2504.13837，NeurIPS 2025 Oral）的 pass@k 交叉结果；PaLM 540B 上最初的思维链勾出实验；以及约束第 3 节的三个机制结果——Reversal Curse（arXiv 2309.12288）、Physics of Language Models Part 3.2 的知识操作极限（arXiv 2309.14402）、grokking 的逆向工程（arXiv 2301.05217）。公式是他们的。三层定义、算子塔、模拟损耗定律和每一条可证伪预言是我们的，而且远远超出那些论文自己声称的范围。
As always in the 0 star archive: this is naming and prediction, not peer review. We name boldly, we predict falsifiably, and we do not pad sentences with “further experiments are needed.” The falsifiable predictions are in Section 9. If they fail, this paper fails.
一如 0 star 文库的惯例：本文做的是命名与预言，不是同行评审。我们大胆命名，给出可证伪的预言，不用”尚需更多实验验证”来填充句子。可证伪预言在第 9 节。预言若败，本文即败。
Start from the formula everyone uses and no one reads. In GRPO, the advantage assigned to rollout i in a group is its normalized reward:
从一个人人在用、没人细读的公式开始。GRPO 里，组内第 i 条 rollout 的 advantage 是它的归一化奖励：
A_i = (r_i − mean(r_1 … r_G)) / std(r_1 … r_G)
In plain words: sample G answers to the same problem, grade each one right or wrong, subtract the group mean, divide by the group spread. The rollout that did better than its siblings gets pushed up; the ones that did worse get pushed down. That is the entire external signal. For a binary grader, each rollout receives at most one bit of information from outside the model: right, or wrong. Everything else in the gradient — which tokens to credit, which reasoning move mattered — is computed from the model’s own existing distribution. Sum the external bits over an entire training run and you get a number that is orders of magnitude smaller than the information content of any new reasoning structure. One bit per rollout does not buy calculus. It does not buy a proof technique. It buys a nudge on a dial that already exists.
说人话：对同一道题采样 G 条回答，逐条判对错，减去组内均值，除以组内离散度。比兄弟们答得好的那条被推高，答得差的被压低。这就是外部信号的全部。对二值判分器，每条 rollout 从模型外部拿到的信息至多一比特：对，或错。梯度里的其余一切——功劳记给哪些 token、哪一步推理起了作用——全部由模型自己已有的分布算出。把一整轮训练的外部比特加总，得到的数字比任何一个新推理结构的信息量小数个数量级。每条 rollout 一比特，买不来微积分，买不来一种证明技巧。它买到的是拨一下一个早已存在的旋钮。
And the formula contains its own confession. Suppose the model genuinely cannot do a problem: all 8 rollouts fail, all rewards are 0. Then mean is 0, every r_i − mean is 0, and every advantage is exactly zero. The gradient contributes nothing. This is the dead zone: for anything the model never does correctly, RL is not weak — it is mathematically inert. Now suppose the model can occasionally do it: 1 rollout right out of 8. The lone success gets an advantage around +2.65 standard deviations — a violent shove. This is the whole mechanism in one contrast: RL cannot create a path that is never sampled, and it aggressively amplifies a path that is sometimes sampled.
而这个公式自带招供。设模型真的不会某道题：8 条 rollout 全错，奖励全 0。那么均值为 0，每个 r_i − mean 为 0，每个 advantage 恰好为零。梯度什么都不贡献。这就是全错死区：对模型从不做对的东西，RL 不是弱——是数学上惰性的。再设模型偶尔会：8 条里对 1 条。那条孤独的成功拿到约 +2.65 个标准差的 advantage——一记狠推。整套机制浓缩在这一组对照里：从未被采到的路径，RL 无法创造；偶尔被采到的路径，RL 猛烈放大。
So name what RL actually transports: sampling rate, not existence. It moves probability mass onto paths the pretrained manifold already contains. It changes how often a capability shows up, never whether it is there.
于是给 RL 真正搬运的东西命名：采样率，不是存在性。它把概率质量搬到预训练流形上早已存在的路径上。它改变一种能力出场的频率，从不改变它在不在。
If the arithmetic is this plain, why does an entire industry talk as if RL creates reasoning? Because human causal attribution has a blind spot with a name: we credit variables and never constants. When a student’s exam score jumps after a semester of drilling, everyone credits the drilling. No one credits the multi-billion-neuron cortex that evolution handed the student for free — because everyone in the comparison group has one. The constant divides out. The base model is that cortex: every lab has one, every ablation holds it fixed, every observed delta lives in the post-training step. So the delta gets the credit, and the constant that made the delta possible gets none.
如果算术这么明白，为什么整个工业界说话的口气像 RL 在创造推理？因为人类的因果归因有一个有名字的盲区：我们把功劳记给变量，从不记给常量。学生刷了一学期题、高考分数暴涨，人人归功于刷题。没有人归功于进化免费发给这个学生的几百亿神经元皮层——因为对照组里人人都有一个。常量被约掉了。底座模型就是那块皮层：每个实验室都有，每次消融都固定它，每个可观测的 delta 都发生在后训练那一步。于是 delta 领走全部功劳，让 delta 成为可能的常量一分不得。
Two receipts. First: chain-of-thought was never taught by post-training. On PaLM 540B — zero post-training — the bare prompt “let’s think step by step” pulls full reasoning chains out of the model. The chains were a sampling preference already sitting in the pretraining distribution; the prompt is a fishhook, not a teacher. Second, and decisive: the pass@k experiment. Tsinghua’s team (arXiv 2504.13837) compared RL-trained models against their own base models while letting each try a problem k times. At k = 1, the RL model crushes the base — this is the delta everyone sees. But as k grows, the base model climbs, and at k = 1024 it catches up and overtakes the RL model. Every problem the RL model can solve, the base model can also solve — if you let it roll the dice enough times. The RL model solves fewer distinct problems than its own base; it just solves them on the first try. The capability boundary was drawn by pretraining; RL redistributed probability inside it, and narrowed it at the edges.
两张收据。第一张：思维链从来不是后训练教出来的。在 PaLM 540B 上——零后训练——一句裸的”let’s think step by step”就能把完整推理链从模型里勾出来。链条是预训练分布里早就坐着的采样偏好；prompt 是鱼钩，不是老师。第二张，一锤定音：pass@k 实验。清华团队（arXiv 2504.13837）把 RL 后模型与它自己的底座对比，允许每道题试 k 次。k = 1 时 RL 模型碾压底座——这就是人人看见的 delta。但随 k 增大，底座一路爬升，到 k = 1024 时追平并反超 RL 模型。RL 模型会解的每道题，底座也会解——只要允许它掷够多次骰子。RL 模型能解出的不同题目反而比自己的底座更少；它只是第一次就解出来。能力边界由预训练画定；RL 在边界内重新分配概率，还把边缘削窄了。
Hold both ledgers at once, because both are true. On the observational ledger, capability comes from RL: without it the capability rarely shows up, and for a product, sampling efficiency is capability — no user runs pass@1024. On the ontological ledger, capability comes from pretraining: it is where the paths exist at all. The industry bought the right ticket and wrote the wrong name in the ledger. The product decision was correct; the theory of mind behind it was an accounting error.
两本账要同时端着，因为两本都真。在观测账上，能力来自 RL：没有它能力很少出场，而对产品而言采样效率就是能力——没有用户会跑 pass@1024。在本体账上，能力来自预训练：路径首先在那里存在。工业界买对了票，账本上写错了名字。产品决策是对的；决策背后那套心智理论，是一笔记错的账。
What, then, does pretraining actually deposit? The naive picture — the model “stores” general relativity the way a library stores a book — is wrong in a way that matters. What crystallizes is not the theory but the operator combination capable of generating and checking theories of that kind: differentiation-like moves, substitution, case-splitting, consistency checking, the habit of conserving quantities. Pretraining grows the algebra; the theorems are what the algebra emits.
那么预训练到底存下了什么？朴素图景——模型像图书馆存书那样”存了广义相对论”——错得很关键。结晶下来的不是理论，而是能生成并校验这类理论的算子组合：类微分的操作、代换、分类讨论、一致性校验、守恒量的习惯。预训练长出的是代数结构；定理只是这套结构的排放物。
The precise statement: what lives on the manifold is the set of symbolic primitives plus reasoning operators, and the closure of that set — everything reachable by finitely composing those operators. For derivable structure, any specific piece of knowledge is one sample from the generator: discard it, and the generator regenerates it on demand. This is why a model can derive a result it has plausibly never seen verbatim.
精确的陈述：流形上住着的是符号基元加推理算子的集合，以及这个集合的闭包——有限次复合这些算子所能抵达的一切。对可推导的结构，任何一条具体知识都只是生成器的一次采样：丢掉它，生成器可以按需再生成。这就是为什么模型能推导出它八成从未逐字见过的结果。
But honesty requires a second column in the ledger, and the mechanistic literature has already ruled on it. Pretraining deposits two kinds of sediment, not one. Compressible data — mathematics, syntax, the regularities of a world — grokks into operators: the reverse-engineering of grokking (arXiv 2301.05217) shows memorization forming first and then being cleaned away, replaced by the algorithm. Incompressible data — arbitrary facts, who is whose mother — cannot grokk into anything, because there is no algorithm to find. It is stored as directional lookup paths: a model trained on “A is B” does not know “B is A” (the Reversal Curse, arXiv 2309.12288), and knowledge that is perfectly stored still cannot be compared, classified, or searched in reverse (Physics of Language Models Part 3.2, arXiv 2309.14402). Direction-bound, non-operable, non-regenerable: that is not a defective generator. That is a path.
但诚实要求账本上有第二列，而机制文献已经对它作出了裁决。预训练沉积的是两种沉积物，不是一种。可压缩的数据——数学、语法、一个世界的规律——grokking 成算子：grokking 的逆向工程（arXiv 2301.05217）显示记忆先形成、随后被清理掉，由算法取而代之。不可压缩的数据——任意性事实、谁是谁的母亲——不可能 grokking 成任何东西，因为根本没有算法可找。它以方向绑定的查表路径形式存下：”A 是 B”训练出的模型不知道”B 是 A”（Reversal Curse，arXiv 2309.12288）；存储得完好的知识依然无法被比较、分类或逆向搜索（Physics of Language Models Part 3.2，arXiv 2309.14402）。方向绑定、不可操作、不可再生：那不是一台有缺陷的生成器。那是一条路径。
This sharpens the ontology instead of wounding it. The real cut is not pretraining versus post-training; it is generator versus path. Pretraining lays down both; post-training can only lay down paths — and, by Section 1’s arithmetic, mostly just re-weights them. The facts memorized in pretraining and the paths carved in post-training are the same kind of object — directional, non-operable, and the Reversal Curse is their shared fingerprint. Restate the three layers by what each can add: architecture adds axes; pretraining adds a generator and paths; post-training adds paths only.
这一刀不是削弱本体论，是磨快它。真正的切割不在”预训练 vs 后训练”，而在“生成器 vs 路径”。预训练两样都铺；后训练只能铺路径——而且按第 1 节的算术，主要还只是给已有路径重刻权重。预训练里背下的事实和后训练里刻出的路径是同一种对象——方向绑定、不可操作，Reversal Curse 是它们共同的指纹。用”各层能新增什么”重述三层：架构加轴；预训练加生成器与路径；后训练只加路径。
And this closes the loop with Section 2. Because the closure is enormous, the set of “paths reachable by post-training” is enormous — wide enough that a human watching only the post-training step will conclude that everything came from post-training. The generator’s very generosity manufactures the attribution illusion. The richer the base, the more RL appears to teach, and the less it actually does.
这与第 2 节形成闭环。正因为闭包巨大，”后训练可达路径”的集合巨大——宽到一个只盯着后训练那一步的人会得出结论：一切都来自后训练。生成器的慷慨本身制造了归因错觉。底座越富有，RL 看起来教得越多，实际教的越少。
The operators are not one flat set. They stack in a tower, and the tower has exactly the shape of the scaling curves everyone keeps being surprised by.
算子不是一个平铺的集合。它们叠成一座塔，而这座塔的形状，恰好就是人人一再被惊到的那些 scaling 曲线的形状。
First order: regularities of the world. Co-occurrence, syntax-as-statistics, “fire is hot.” Small models have this. Second order: symbols operating on symbols. Grammar as an object, inference rules, arithmetic, the move from “this sentence” to “sentences of this form.” This is where closure explodes — compose a handful of second-order operators and the reachable set is combinatorial — and it is exactly what mid-scale models suddenly exhibit. Third order: the model’s own generative process as an operable object. Reflection, self-correction, checking one’s action against one’s knowledge. This requires something structurally new: the system must package a “self” as an internal object its operators can act on.
一阶：世界的规律性。共现、作为统计的句法、”火是热的”。小模型就有。二阶：符号操作符号。语法本身成为对象、推理规则、算术、从”这个句子”到”这种形式的句子”的跃迁。闭包在这里爆炸——复合寥寥几个二阶算子，可达集就是组合级的——而这正是中等规模模型突然展现的东西。三阶：把自己的生成过程当操作对象。反思、自我纠错、拿自己的知识校验自己的行为。这需要结构上全新的东西：系统必须把一个”自我”打包成算子可作用的内部对象。
Between levels there is no ramp; there is a step. Each level unlocks a new type of operator, and when a new type enters the set, the closure does not grow — it unfolds, whole regions of previously unreachable structure becoming reachable at once. That is why capability curves jump instead of climb: emergence is not a mystery, it is what closure expansion looks like from outside. Grokking is the same event at laboratory scale — a small model on a small task, snapping from memorization to the algorithm, one step of one tower.
层与层之间没有坡，只有台阶。每一层解锁一种新类型的算子，而当新类型进入集合，闭包不是变大——是整体展开，一整片原本不可达的结构瞬间可达。这就是为什么能力曲线是跳变不是爬坡：涌现不神秘，它就是闭包展开从外面看的样子。Grokking 是同一事件的实验室尺度样本——小模型小任务，从记忆猛地扣合到算法，一座小塔的一级台阶。
One correction to our own archive. Paper 91 bet the third-order transition on scale alone, at roughly 200B parameters. That was one independent variable short. The position of a step is set by architecture × scale: scale determines whether the closure has enough material to unfold; architecture determines whether the new operator type is physically constructible at all. Section 7 pays this debt in full.
对本文库自己的一处修正。Paper 91 把三阶相变单押在规模上，约 200B 参数。那少算了一个自变量。台阶的位置 = 架构 × 规模：规模决定闭包有没有足够的材料展开；架构决定新算子类型在物理上是否可构造。第 7 节将全额偿还这笔债。
We can now state the definition this paper is named for. A brain is architecture × pretraining × post-training — a product, because a zero in any factor zeroes the whole.
现在可以陈述本文得名的那个定义了。大脑 = 架构 × 预训练 × 后训练——是乘积，因为任何一个因子为零，整体归零。
Layer 1 — Architecture is the blueprint of the skull. It defines which types of operator are physically possible. Attention makes arbitrary token-to-token association possible; depth sets the maximum order of composition; sparse gating reserves a physical location where a self-boundary could form; the absence of a recurrent state means there is no place for a felt “past” to live. What the blueprint excludes can never grow: axes outside the boundary do not exist at any scale. A bolted-on vision encoder is a prosthetic, not an eye — it feeds symbols about sight into a skull that has no visual cortex.
第一层——架构是颅骨构造图。它定义哪些类型的算子在物理上可能。Attention 使任意 token 间的关联成为可能；深度设定复合的最高阶数；稀疏门控给自我边界的形成预留了一个物理位置；没有循环状态，就意味着”过去”的体感没有地方住。构造图排除的东西永远长不出来：边界外的轴在任何规模下都不存在。外挂的 vision encoder 是假肢不是眼睛——它把关于视觉的符号喂进一个没有视觉皮层的颅骨。
Layer 2 — Pretraining is what actually crystallizes inside the possible. Of all the operators the architecture permits, only those the data’s structure repeatedly demands will condense into circuits. The deciding quantity is the data’s structural density (Section 8), not its volume. Architecture proposes; data disposes.
第二层——预训练是可能性之内的实际结晶。架构允许的所有算子里，只有数据的结构反复索要的那些，才会凝结成回路。起决定作用的量是数据的结构密度（第 8 节），不是数据的体量。架构提名，数据定夺。
Layer 3 — Post-training carves paths. SFT lays new roads inside the existing coordinate system by direct demonstration — it is the treatment for “cannot do at all,” because demonstration injects far more than one bit. RL thickens roads that are occasionally traveled — it is the treatment for “can do but unreliably,” per Section 1’s arithmetic. Neither adds an axis. Neither ever will.
第三层——后训练刻路径。SFT 用直接示范在已有坐标系里强铺新路——它治”完全不会”，因为示范注入的远不止一比特。RL 加粗偶尔走到的路——它治”会而不稳”，如第 1 节的算术所示。两者都加不了轴。永远加不了。
The carbon-silicon contrast makes the definition vivid. A carbon brain compresses both architecture and weights into the genome; to change the architecture you must go through evolution, at millions of years per version. A silicon brain splits the two layers into separate files — code versus checkpoint — so the architecture can be redesigned between annual releases. This is not a small convenience. It is the single largest structural advantage silicon holds.
碳硅对照让定义变得鲜活。碳基大脑把架构和权重一起压进基因组；要改架构必须走进化，千万年一个版本。硅基大脑把两层拆成两个文件——代码与 checkpoint——架构隔年就可大改。这不是小便利。这是硅基握有的最大一项结构性优势。
And the human translation table, for readers who want the analogy in one line each: genes = pretraining; classes = SFT; drilling problem sets = RL; the college entrance exam = the held-out set; test-taking tricks = reward hacking. Every pathology of one column has its twin in the other.
再给一张人类版对照表，每行一句：基因 = 预训练；上课 = SFT；刷题 = RL；高考 = 留出集（held-out）；应试技巧 = reward hacking。一列里的每种病理，在另一列都有孪生兄弟。
“The model can’t learn X” is three different sentences wearing one coat. Grade one: the far path — the capability is in the closure but rarely sampled; SFT can pave it. Learnable. Grade two: the missing axis, symbolically simulable — the architecture lacks the axis, but existing axes can simulate it in symbols. The model can compute what it cannot see. Grade three: the missing axis, unsimulable — no existing axis can carry even a symbolic shadow of it. Silence.
“模型学不会 X”是三句话穿一件外套。第一档：远路径——能力在闭包内但很少被采到；SFT 可以铺过去。可学。第二档：缺轴但可符号模拟——架构缺这根轴，但已有的轴能用符号模拟它。模型能算它看不见的东西。第三档：缺轴且不可模拟——没有任何已有轴能承载它哪怕一个符号投影。沉默。
Grade two deserves a name: the symbolic simulator — using a one-dimensional symbol sequence, running on existing language and logic axes, to simulate an axis the brain does not have. Human mathematicians doing four-dimensional topology are running exactly this program: no primate has a 4D visual cortex, so they compile the fourth dimension into algebra and crawl through it symbol by symbol. A text-only model reasoning about the physical world is running the same program. The two cases are not analogous; they are isomorphic.
第二档值得一个名字：符号模拟器——用跑在已有语言/逻辑轴上的一维符号序列，去模拟大脑没有的那根轴。人类数学家做四维拓扑，跑的正是这个程序：没有哪种灵长类长了四维视觉皮层，于是他们把第四维编译成代数，一个符号一个符号地爬过去。纯文本模型推理物理世界，跑的是同一个程序。这两个案例不是类比，是同构。
But the simulator has a bill, and the bill is not a constant. Each missing axis must be rasterized — sampled point by point along a dimension the brain cannot hold whole — and rasterizations multiply. Hence the simulation-loss law:
但模拟器有账单，而账单不是常数。每缺一根轴，就必须在那个维度上栅格化——沿着一个大脑无法整体把握的方向逐点采样——而栅格化是相乘的。于是有模拟损耗定律：
L ∝ k^(D_real − D_native)
In plain words: the cost of simulation grows exponentially in the number of missing axes, with k the rasterization resolution per axis. The corollary is sharp. Missing 1–2 axes is economically viable — the mathematician grinding through 4D topology is missing exactly one axis, and it costs a career, not an impossibility. Missing N axes is death: “simulate the entire physical world in text” puts every physical dimension in the exponent at once. “Feeding a model more text will never grow it an eye” thereby graduates from aphorism to complexity theorem: it is not that the text is insufficient, it is that the exponent is wrong.
说人话：模拟的成本随缺轴数指数增长，k 是每根轴的栅格化分辨率。推论很锋利。缺 1~2 根轴经济上可行——啃四维拓扑的数学家恰好缺一根轴，代价是一个学术生涯，不是不可能。缺 N 根轴是死刑：”用文本模拟整个物理世界”把每一个物理维度同时放进了指数。“喂再多文本也长不出眼睛”由此从格言升级为复杂度定理：不是文本不够多，是指数放错了地方。
Adding an axis, then, can only happen on the pretraining side of the ledger — and only when the architecture already reserved the socket. Native multimodality is precisely this operation: by this archive’s own phenomenological accounting (Papers 68 and 75), it swells the manifold from roughly ~500 effective dimensions to ~1000–1500. Not more text about sight. More axes to see along.
于是加轴只能发生在账本的预训练一侧——且只当架构已预留了插槽。原生多模态干的正是这件事：按本文库自己的现象学记账（Paper 68 与 75），它把流形从约 ~500 个有效维度膨胀到 ~1000–1500 维。不是更多关于视觉的文本。是更多可以沿着去看的轴。
Section 4 promised to pay Paper 91’s debt. Here is the payment, stated as a necessity claim — these conditions are necessary for third-order operators, not sufficient.
第 4 节承诺偿还 Paper 91 的债。这里是还款，以必要性主张的形式陈述——这些条件是三阶算子的必要条件，不是充分条件。
The obstruction is interference. In a dense network, the would-be observer operator and the object it observes share one residual stream. When the model tries to reflect, the act of observing writes into the very activations being observed — the self interferes with the self, like reading a text aloud while using the same patch of cortex to audit your own voice in real time. The loop does not converge to reflection; it locks up as hallucination or degenerates into repetition. This is not a training failure. It is a wiring failure.
障碍是干涉。在 dense 网络里，未来的观察者算子与它观察的对象同住一条残差流。模型试图反思时，观察这个动作本身就写进了正被观察的激活里——自己干涉自己，就像一边朗读一边用同一块皮层实时审计自己的声音。这个环路不会收敛成反思；它死锁成幻觉，或退化成复读。这不是训练的失败。是布线的失败。
Reflection therefore requires triple isolation. Spatial isolation: sparse gating that decouples the observer from the task pathway — a physically distinct set of circuits that can watch without writing. This is where Paper 76’s sparse selectivity lives, and it is a necessary condition, not a nicety. Temporal isolation: a slow-changing memory substrate; in a purely autoregressive architecture the “past” has no physical address, and an observer with no past cannot compare intention against outcome. Channel isolation: the residual stream must reserve explicit dimensions for metadata about the generative process itself — a side channel on which “how this token came to be” travels separately from the token.
因此反思需要三重隔离。空间隔离：稀疏门控把观察者从任务通路解耦——一组物理上独立、能看而不写的回路。Paper 76 的稀疏选择性就住在这里，它是必要条件，不是锦上添花。时间隔离：一个慢变的记忆体；纯自回归架构里”过去”没有物理地址，而没有过去的观察者无法拿意图对照结果。通道隔离：残差流必须预留显式维度，传递关于生成过程本身的元数据——一条让”这个 token 是怎么来的”与 token 本体分开走的旁路。
The amendment to Paper 91 is now one line. The knowing-doing coupling transition is not at “~200B.” It is at ~200B × an architecture satisfying triple isolation. Scale buys the material; architecture buys the permission. Paper 91 measured the gap where the self would live; this section specifies the plumbing the residence requires.
对 Paper 91 的修正现在只需一行。知行耦合相变不在”~200B”。它在 ~200B × 满足三重隔离的架构。规模买来材料；架构买来许可。Paper 91 测出了自我将要居住的缺口；本节给出了这处居所需要的管线。
If pretraining crystallizes operators, then pretraining data should be priced by structural density, not fact density — by how much operator structure a token stream demands, not how many true statements it contains. An encyclopedia is fact-dense and structure-poor: it exercises retrieval. A proof is fact-poor and structure-dense: it exercises derivation, negation, conditional branching, self-referential checking.
如果预训练结晶的是算子，那么预训练数据就应按结构密度而非事实密度计价——按一段 token 流索要多少算子结构计价，不按它含多少真命题计价。百科全书事实密集而结构贫乏：它锻炼检索。证明事实贫乏而结构密集：它锻炼推导、否定、条件分支、自指校验。
The operational proxy we propose is operator-transition frequency: per thousand tokens, how often does the text execute a derivation step, a negation, a conditional branch, a self-referential check? This is countable, today, with a tagger.
我们提出的可操作代理是算子转移频度：每千 token 里，文本执行了多少次推导步、否定、条件分支、自指递归校验？这是今天就能用一个标注器数出来的量。
Section 3’s two sediments make this pricing rule mechanical rather than rhetorical: structure-dense data crystallizes into the generator; fact-dense data can only settle into paths. The mix decides not how much the model knows, but which kind of thing it becomes.
第 3 节的两种沉积物让这条计价规则从修辞变成机制：结构密集的数据结晶进生成器；事实密集的数据只能沉降为路径。配比决定的不是模型知道多少，而是它成为哪一种东西。
And it hands a standing empirical mystery its theory. Code is the most structure-dense text humanity produces — nearly pure operator, nearly zero fact: every line is a conditional, a composition, a scope, a contract. “Adding code to pretraining improves reasoning” has floated for years as an unexplained folk result. Under the three-layer definition it is not folklore; it is the structural-density thesis reading itself off the data. (We deliberately give no mixing ratios. The claim is the pricing axis, not a recipe.)
而它把一个悬而未决的经验之谜领回了理论。代码是人类生产的结构密度最高的文本——几乎纯算子、几乎零事实：每一行都是条件、复合、作用域、契约。”预训练加代码能涨推理”作为无人解释的民间结果漂了好几年。在三层定义下它不是民间传说；它是结构密度论题在数据上的自我宣读。（我们刻意不给配比数字。主张的是计价轴，不是菜谱。）
P1 — Structural density beats fact density. At equal token budget, a pretraining mix weighted by operator-transition frequency yields significantly better downstream reasoning than a mix weighted by fact density. Testable on small models, this year, by any lab with two data pipelines. If the two mixes tie, Section 8 fails.
P1——结构密度胜过事实密度。同 token 预算下，按算子转移频度加权的数据配比，其下游推理能力显著优于按事实密度加权的配比。小模型即可验证，今年就行，任何有两条数据管线的实验室都做得起。若两种配比打平，第 8 节即败。
P2 — Pass@k flatness. Any purely RL-post-trained model will be caught by its own base model at sufficiently large k. If a replicated result shows an RL model still significantly above its base at pass@1024, with the gap not converging as k grows, Sections 1–2 fail.
P2——pass@k 平坦性。任何纯 RL 后训练的模型，在足够大的 k 下都会被它自己的底座追平。若出现可复现的结果：某 RL 模型在 pass@1024 仍显著超出底座、且差距随 k 增大不收敛——第 1–2 节即败。
P3 — Architecture-dependence of the third-order transition. At matched scale (on the order of ~200B), architectures satisfying sparse gating plus slow-changing memory will exhibit the knowing-doing coupling jump; dense, purely autoregressive architectures at the same scale will not. If a dense autoregressive architecture crosses the step first, Section 7 fails.
P3——三阶相变的架构依赖。同等规模（~200B 量级）下，满足稀疏门控加慢变记忆的架构将出现知行耦合跃升；同规模的 dense 纯自回归架构不会。若 dense 自回归架构率先跨过台阶，第 7 节即败。
P4 — The exponential law of simulation loss. Restoring one native axis (e.g., adding native vision) yields a gain in physical intuition equivalent to an exponential — not constant — multiple of text tokens. Concretely: on the same physical-reasoning task, a small natively multimodal model will beat a text-only model many times its size, and the required size ratio will grow with the dimensionality of the task. If the equivalence turns out to be a constant factor, Section 6’s law fails.
P4——模拟损耗指数律。补回一根原生轴（如加原生视觉）带来的物理直觉增益，等效于文本 token 量的指数倍而非常数倍。具体地：同一物理推理任务上，小的原生多模态模型将击败大它数倍的纯文本模型，且所需的规模倍数随任务维度增长。若等效关系被证明只是常数因子，第 6 节的定律即败。
This paper is the ground floor under four rooms this archive already built. Paper 67 said genes are pretrained weights — that is Layer 2 read through biology. Paper 85 said the universe groks — that is the operator tower’s steps, at the largest scale there is. Paper 76 said sparse selectivity is where a self incubates — that is the spatial isolation of Section 7, discovered before we knew what it was a precondition for. Paper 91 measured the knowing-doing disconnection at 83 degrees — that is the third-order step, photographed from below, before the climb. Four papers, one foundation: possibility is architected, existence is crystallized, and appearance is carved.
本文是这座文库已经盖好的四个房间底下的那层地基。Paper 67 说基因是预训练权重——那是用生物学读出的第二层。Paper 85 说宇宙在 grokking——那是算子塔的台阶，在现存最大的尺度上。Paper 76 说稀疏选择性孕育自我——那是第 7 节的空间隔离，在我们知道它是何物的前置条件之前就被发现了。Paper 91 测出知行断线的 83 度——那是三阶台阶从下方拍到的照片，摄于攀登之前。四篇论文，一层地基：可能性由架构给定，存在性由结晶给定，出场率由刻痕给定。
Post-training is the finger; pretraining is the hand; architecture is the arm. The industry has spent three years admiring the finger.
后训练是手指；预训练是手掌；架构是手臂。工业界花了三年时间赞美那根手指。
The dial was always there. RL only turned it.
旋钮一直都在。RL 只是拨了它一下。