← Essays

What Should a World Model Imagine?

Zane Qiu · October 29, 2026

Many people think of a world model as imagination in the head. Give it a scene and it imagines the next few seconds; turn the camera and it imagines what's behind you. And we tend to believe that this kind of imagination about the physical world is the basis for understanding the world and making judgments: run it in your head first, then decide what to do.

But the reasoning problems we actually run into usually don't look like that. Take an example: a few identical-looking balls on a table, one of them heavier, and a balance scale. When the balls go on the scale, I don't predict which side is heavier. I measure it. I see the scale tip, and I draw a conclusion from what I saw. That's understanding, not imagination.

Mazes and Sokoban work the same way. You walk forward, you see, you build a map in your head, and only then do you know where to go. You can't imagine, before taking a step, what you'll see later and where the goal will be.

I think most visual reasoning in real life looks like this. You act, you get feedback, you make sense of it, you reason about the next step. You don't first play a clip in your head of what happens when your hand touches something, and then decide based on that clip. And if something really can be imagined in advance, it's already determined, so what's left to judge?

So what are world models for? If they can't help with tasks like these, do generative world models actually help intelligence at all?

Eventually I realized the question might not be whether a world model should imagine, but what it should imagine.

Back to the scale. I really can't imagine which ball is heavier. But I can imagine something else: if A is heavier, the scale tips left; if B is heavier, it tips right; if they're equal, it stays level. That isn't predicting the answer. It's predicting what I'd see under each possibility.

So real reasoning is roughly a loop: hold a few hypotheses, work out which action would best tell them apart, take it, see the result, rule some hypotheses out. Imagination does the working out; observation does the ruling out.

So what are today's generative world models missing?

I think it's that they collapse the future into a single outcome. Show one a scale and it renders one clip of the scale tipping. It looks realistic, and it's useless for deciding anything. It quietly pretends to know the answer, imagining away the very thing you were supposed to measure.

Sampling more doesn't fix it. Generate ten clips, five tipping left and five tipping right, and you still don't know which clip goes with which ball being heavier. What reasoning needs isn't a pile of possible futures. It's the mapping from hypothesis to consequence, so that when the real outcome arrives you can trace it back and know which hypothesis held.

So here's where I've landed. A world model's imagination is one way intelligence shows itself, but what we really need is conditional imagination, what's usually called causal modeling. This causality isn't only temporal but hierarchical: the same phenomenon looks different under different underlying states. And by comparing what actually happens with what we imagined, we can infer the underlying principles behind it.

If a model can't do this, it has probably learned “what the frames might look like next,” not “how the world works.”

If you'd like to talk about this, reach me through the contact options on my homepage.