A model can't fetch the weather or run code — it only predicts the next token. So how does it "call" an API? It emits a call-shaped blob of tokens, a harness around it pauses, actually runs the function, pastes the result back as tokens, and the model keeps predicting. Step the loop and watch the hand-off.
The model is the same next-token predictor you built across Part 1 — the vector-to-vocab-logits loop — and it never executes anything. What it learned is to emit a call-shaped token sequence when a task needs live information: a little JSON blob like {"tool":"get_weather","args":{"city":"Paris"}}.
A harness wrapped around the model watches the stream. The instant a complete call appears — a stop-condition on the closing brace — it stops generation, actually runs the function (this is the harness's code, not the model's), and pastes the result back into the context as new tokens.
Then the model resumes with the result sitting in its context and predicts an answer that uses it. The whole "magic" is one interruption in the usual loop: predict a call → pause → external code runs, result injected → keep predicting. This is also the honest fix for lesson 14's hallucination — a tool injects ground truth the frozen weights never stored.
city = Paris — the call arg, the injected result, and the final sentence all rebuild from thisTRY → Hit Run to watch the context grow phase by phase, or Step ▸ to walk it yourself. Switch the city to Tokyo or Oslo — the amber call arg, the cyan injected result, and the resumed sentence all recompute. The violet PAUSE banner is the one moment the model is stopped and your machine is doing the work.
The call {"tool":"get_weather","args":{"city":"Tokyo"}} is just tokens the model was trained to produce. A stop-condition on the closing } ends the model's turn. Nothing in the model executed — it emitted call-shaped tokens.
| step | who acts | what appears in the context |
|---|---|---|
| 1 · reason | model | I need live data — calling a tool . |
| 2 · call | model | {"tool":"get_weather","args":{"city":"Tokyo"}} |
| 3 · pause | harness | runWeather('Tokyo') → { tempC: 24, sky: 'clear' } (model stopped) |
| 4 · inject | harness | {"tempC":24,"sky":"clear"} |
| 5 · answer | model | It's 24°C and clear in Tokyo right now. |
The harness parses the JSON, calls runWeather('Tokyo'), and appends {"tempC":24,"sky":"clear"} to the context as ordinary tokens. Generation resumes and predicts It's 24°C and clear in Tokyo right now. — the resumed sentence is built from the tool result, not replayed: change the city and both the injected JSON and the sentence change (Oslo → {"tempC":6,"sky":"rain"} → "It's 6°C and rain in Oslo right now.").
Why this matters (ties to lesson 14): the frozen weights never stored today's weather. The softmax could only guess a plausible-sounding number. The tool injects a true fact into the context, so the model's next-token prediction now leans on real data instead of plausibility.
"Calling a tool" is predict a call → pause → external code runs → inject the result as tokens → keep predicting. Same next-token loop as always, with exactly one interruption.