Follow The Progress
The /think Span: A Pause Before the Answer
This one has a story. While training theOG I got stuck for almost a week between steps 56K and 58K. The dataset was staged, so the think block was empty until step 57K, and when thinking kicked in the cross-entropy started collapsing. But looking at the outputs I noticed something odd: the empty think token was making the model better. It was emitting nothing, and the code was clearly improving.
Build a small code LLM from scratch

“It’s just noise”
I tried the idea on the big cloud models. They pushed back: noise, they said. When I finally convinced them the code quality was really better, they agreed, and politely mentioned I wasn’t the first. The pause-tokens paper, “Think before you speak” by Goyal et al. (2023), had reached a similar idea: extra tokens before the answer give the model more computation steps. So yes, I made a discovery that had already been discovered. It happens. The design in theOG was worked out on its own, from what the training showed me.
Where the span sits
Every theOG sequence has it, between the prompt and the answer, and every prompt ends with a mode:
<|Assistant|><|Think|><|/Think|><|Answer|>
Not chain of thought
Chain of thought means the model writes out its reasoning in words, and it has to be trained on reasoning text. Read some raw reasoning traces sometime: a lot of it is nonsense we’ve agreed to call thinking. The think span carries no text at all. It’s a pause before the answer, not an essay.
/think and /no_think
def prompt_ids(tok, user, think=False):
mode = "think" if think else "no_think"
head = [USER] + tok.encode(f"\n{user.strip()} /{mode}\n").ids + [USER_CLOSE, ASSISTANT]
return head + ([THINK] if think else [THINK, THINK_CLOSE, ANSWER])
In /no_think mode the prompt already closes the span and opens the answer, so the model goes straight to code. In /think mode the prompt stops right after <|Think|>, and the model decides for itself when to close it.
During training the span is always written and its tokens are labelled, so the model learns <|Think|> → <|/Think|> → <|Answer|>, and in /think mode it can close the span on its own. The format also has a slot for text inside the span. The course never fills it. It’s there for your own experiments.
Does it help?
The model card has the scores for both modes, and the interesting bit is that the two modes solve partly different tasks. The course lets you score your own model both ways, so you don’t have to take my word for it.
The full lesson, with the experiments, is free on GitHub: DrunkkInference, lesson 6.
Can an outsider’s view spot something a specialist would walk right past? Or did I just get lucky?
Quick answers
Is the think span chain of thought?
No. Chain of thought writes reasoning as text. The think span carries no text; it’s a pause before the answer.
What is the difference between /think and /no_think?
In /no_think the prompt closes the span for the model. In /think the model decides when to close it.
Is it the same as pause tokens?
The pause-tokens paper by Goyal et al. (2023) reached a similar idea separately. theOG’s design was worked out independently, from its own training runs.
Next: evaluation. And if you have a task you think a small model could take over, tell me about it.