Don’t kill Schrödinger’s cat at every token.

Token generation should be an optional readout action, not the clock of thought.

Autoregressive language models are trained to predict the next token. That training objective quietly makes one token feel like one unit of thought. But the hidden state before the softmax is not a token. It is a high-dimensional continuous object that can carry unresolved semantic, causal, and plan-level uncertainty.

Every visible reasoning step therefore performs a compression: continuous state, vocabulary projection, one selected token, then continuous state again. This is a useful interface for communication. It is less obviously the right primitive for internal reasoning.

The proposal is deliberately small. Add one extra policy action, <CONT>. If the model selects a normal vocabulary token, it speaks. If it selects <CONT>, it does not emit language yet; it projects the hidden state forward as the next input embedding and keeps computing in latent space.

LLMs and world models as generative holograms

A useful mental model is that an LLM or world model state is not a single sentence, scene, or plan waiting to be printed. It is more like a generative hologram: a compressed pattern from which many compatible continuations can be reconstructed.

Local activations can imply several semantic futures at once. A world model can carry several possible future observations. An agent can carry several possible plans. Generation is the act of reading one slice out of this field.

The important separation is between state evolution and readout. Current decoding often fuses the two: each internal step must also produce an external symbol. A generative-hologram view suggests the model may need several internal transformations before any single readout is appropriate.

Two clocks of thought: token clock vs. latent clock

The diagrams below separate state evolution from readout. The first path collapses every step through softmax. The second lets latent hypotheses move, branch, and merge before a token appears.

In the left panel, every pulse must pass through the vocabulary bottleneck. In the right panel, several latent pulses can circulate and recombine before the final readout.

Autoregression kills the cat every token.

The Schrödinger’s cat metaphor is not meant as physics. It is a reminder about measurement. Before readout, the model state can preserve incompatible but still useful hypotheses. A sampled token turns that mixture into a single trace.

That trace has advantages. It creates a scratchpad. It gives the model a stable object to attend to. It makes reasoning auditable by humans and verifiers. But it also creates path dependence. If the model writes the wrong intermediate phrase, later layers must condition on it, repair it, rationalize it, or route around it.

The failure is not simply that tokens are discrete. Discrete readout is necessary for language. The failure is making readout compulsory at every unit of compute. A model should be able to keep thinking when the uncertainty is still too global, too semantic, or too coupled across alternatives.

Diffusion decoding is a useful contrast: decide later, refine globally.

Diffusion-style generation points toward the opposite bias. Rather than committing left-to-right immediately, it keeps a partially specified object and repeatedly denoises or refines the whole. The system can preserve ambiguity while global structure settles.

For language models, the exact diffusion mechanism is not the main point here. The principle is. Some decisions are cheap and local; they can be collapsed immediately. Other decisions are semantic load-bearing walls. They should not be fixed until enough surrounding structure has been evaluated.

Depth vs. width should be allocated by semantic uncertainty.

Adaptive compute is often described as using more steps for hard problems. But “more steps” can mean two different things. Depth refines one trajectory. Width preserves multiple hypotheses. A good policy should choose between them.

Low entropy over the next token is not always low uncertainty over meaning. The model may be confident about the next connective word while uncertain about the plan. What matters is semantic uncertainty: instability over latent interpretations, goals, causal explanations, or solution paths.

Possible signals include token entropy, disagreement between probes, value uncertainty, verifier margin, semantic clustering instability, self-consistency disagreement, and reward-model uncertainty. None is sufficient alone. The point is to make compute conditional on the kind of uncertainty, not only on sequence length.

CoT may have moved adaptive depth onto the token axis.

Adaptive Computation Time and recurrent-depth architectures such as the Universal Transformer asked a model to spend a variable number of transformations before emitting an answer. Looped transformers continue that line: reuse a block, halt later on harder inputs, and buy more compute without a proportional increase in parameters.

Explicit Chain-of-Thought offers a very LLM-shaped detour. Instead of taking more invisible layer iterations at one position, the model writes an intermediate result, advances along the token axis, and lets later tokens attend back to it. In this sense, CoT may have solved a substantial part of the adaptive-compute problem by temporalizing the computation.

The price is discretization. Every intermediate thought must pass through the vocabulary bottleneck. The benefit is addressability. A thought becomes a position in context that attention can select, combine, and reuse much later. So “CoT made ACT obsolete” would be too strong. CoT traded one compute axis for a powerful memory interface.

A pure depth loop does not lack attention: each recurrent block may still attend across token positions. The missing edge is usually across depth. Earlier depth states are compressed into the current residual stream rather than exposed as individually queryable memory. That changes when the architecture explicitly retains and selects prior layer outputs.

Kimi's Attention Residuals is a concrete step in that direction. AttnRes replaces fixed unit-weight residual accumulation with softmax attention over preceding layer outputs; Block AttnRes attends to block-level representations to control memory and communication cost. One way to read it is as adding attention-like edges to the residual/depth graph. It improves access along the vertical axis, but it is not by itself a temporal scratchpad or a latent CoT policy.

Five paths through the same model

The crucial distinction is what crosses from one unit of compute to the next. Amber boxes are vocabulary-discrete; cyan boxes stay continuous; green boxes add or select depth.

This is why <CONT> is not merely a learned <pause>. Pause-token training appends a discrete placeholder with its own fixed learned embedding and delays reading outputs. The proposed action instead feeds the result of the last generation step—its final-layer hidden state, after a learned projection, normalization, or gate—into the next input slot. Information in direction and magnitude that has no single vocabulary name can survive.

That path is not available in vanilla autoregressive decoding. The standard loop takes logits, selects a token ID, and looks up its embedding for the next step; it does not reinject the previous step's top-layer hidden state. An implementation can expose that state, but making it a valid next input requires a custom decoding and training runtime, learning the resulting state distribution, and careful interaction with positional encoding and the KV cache. As SWITCH demonstrates, this need not change the Transformer block itself; it changes the generation loop and training path, so it is still more than a prompting trick.

A minimal <CONT> action learned during RL

Coconut established the core recurrence: use the last-layer hidden state as a continuous thought and feed it as the next input embedding. SWITCH is now the strongest empirical precedent for the same hidden-state feedback. It enters and exits a latent mode with <swi> and </swi>, fills that interval with deterministic <latent> positions, and combines a visible-to-latent curriculum with Switch-GRPO.

The proposal here removes the mode. <CONT> is one action available at every generation step—not an input placeholder and not an opening delimiter. A normal vocabulary action emits a token and naturally ends the silent chain. A <CONT> action emits nothing and routes the current final-layer state into the next input.

The softmax still runs at every step: it chooses either a vocabulary item or <CONT>. What becomes optional is vocabulary readout. This makes the decision itself discrete and gives every latent step a policy log-probability, while the carried thought remains continuous.

A step cost can teach the model not to think forever. There is still a real RL difficulty: the next state depends on the policy parameters through g_φ(h_t^L), so PPO/GRPO cannot treat the latent trajectory exactly like a fixed token trajectory. Stop-gradient feedback is the simplest approximation; replaying the old latent states or differentiating through the full recurrence are more faithful and more expensive alternatives.

Branch, merge, and recombine latent hypotheses

The one-action version is the smallest experiment. The natural extension is a latent search policy. A hidden state can branch into multiple hypotheses, run a little depth on each, then merge compatible parts before any token is emitted.

This avoids both extremes. It is not ordinary chain-of-thought, where every branch has to become text. It is also not a single blurry average state that may wash out important alternatives. The goal is structured latent multiplicity: several distinguishable hypotheses that can later recombine.

What would make this real?

This is a hypothesis, not an empirical result. A fair test would compare no-CoT, standard tokenized CoT, pause-token training, recurrent-depth or looped models, depth-aware attention, diffusion or iterative decoding, Coconut-style continuous thought, SWITCH, and mode-free RL-trained <CONT> policies under equal compute budgets.

The evaluation should not stop at final accuracy. We should measure calibration after latent steps, recovery from early wrong hypotheses, semantic diversity of maintained alternatives, verifier margin before and after readout, robustness under adversarial shortcuts, and whether latent states are causally useful rather than decorative.

The research question is simple: can a model learn when not to say the next token?