Runs and turns

What a run holds

A run is one Solana account. It is created at its full size when the run opens, and it holds everything the run is:

• the opener, a wallet or a program’s PDA
• the model
• the prompt, UTF-8 text of up to 1,024 bytes
• the parameters: max tokens, temperature and seed
• the turn reward, the royalty and the protocol share rate in force when it opened
• the budget deposited and the budget remaining, in lamports
• the count of turns landed and the cost of the next turn
• the position in the forward pass
• the working activations and the attention cache
• the output: token ids and text bytes
• the accrued royalty and protocol share
• the state

Max tokens goes from 1 up to the model’s context length, and temperature from 0 to 16. The attention cache has room for the positions the run can reach: one plus the prompt’s bytes plus max tokens, up to the context length.

What a turn does

A turn is one transaction, signed by anyone. It picks up the forward pass at the position the run account holds, advances it as far as one transaction’s 1,400,000 compute units allow, and writes the new position back. A turn uses about 1,340,000 of them.

The forward pass moves through these stages, and a turn continues from whichever one the last turn stopped in:

1. Tokenizing the prompt.
2. Embedding the next token.
3. Each layer in order: normalization, attention with rotary position encoding, the feed-forward block.
4. The final normalization and the classifier.
5. Sampling the next token from the run’s seed.

A turn does not stop at a token: on a small model one turn finishes a token and starts the next, and can complete several. When a token completes, its id and its text bytes are appended to the run account, and the text is readable from chain state at once.

A turn carries no model input: everything it computes comes from the run account and the model’s weights. Its arguments are the turn index it expects and a nonce, and its accounts name the weight accounts it reads, section zero and up to two matrix sections. The turn pays its signer the turn reward from the run’s budget, as Fees describes.

Turns per token, measured

Measured on a validator, on the prompt “Once upon a time” at temperature zero:

The count for prompt tokens includes tokenizing the prompt. A prompt token skips the classifier and the sampler, which a generated token runs over the whole vocabulary. Models and the weight format gives the cost of the arithmetic underneath.

The four states

A run opens as running. running and starved are the open states, in which a run takes Fund and Cancel run. complete and cancelled are the ended states, in which it takes Settle and then Close run.

Starved runs and the top-up

The run account holds the cost of its next turn: the turn reward, the royalty for the most tokens that turn can complete, and the protocol share those two add. The program computes it at open and again after every turn.

A run is starved as soon as its remaining budget is below that cost. A run opened with a budget below the cost of its first turn opens starved.

Fund adds to the budget. The run is running again when the budget reaches the cost of the next turn, so the smallest top-up that continues a run is the cost of the next turn minus the remaining budget. The SDK’s getRun and getRunHead return both figures as nextTurnCost and budgetRemaining, in lamports.

Fund, cancel, settle, close

A complete run and a cancelled run both go through Settle and then Close run. After the account closes, the run’s page keeps its record: prompt, output, turns and settlement.

All four are buttons on the run’s page, and Program reference lists the instruction behind each.

Why a run has no timeout

A run waits on no party. Turning is open to every wallet, so the caller can always turn its own run. A run stays exactly where its last turn left it until the next turn lands.