arizuko

arizuko › concepts › retries

retries

Every turn runs in its own short-lived virtual machine. Sometimes that guest stops in the middle of a turn, because it runs out of memory or reaches the run time limit. Then no answer comes back. routd sees the run with no reply and runs the turn again, up to three times, without a message to you. You get one slower reply instead of silence.

A turn fails only if no bot message arrived

The test is “did any bot message arrive?”, not “did the agent finish without an error?”. An agent that posted half an answer and then stopped counts as a success, and routd does not retry it. A partial reply that you can read is better than a wait while the agent does the whole task again. Only a turn with no bot message at all starts a retry.

You see nothing until a retry succeeds

No scheduler starts the retry. routd does not advance the failed batch, so the next poll of routd takes the same message and starts a new guest. The repeated read is the retry, so no code must move a cursor back. No delay exists between attempts either. The old guest is already gone, and the new one starts from the beginning, so a wait would only make your silence longer.

If every attempt fails, one message arrives:

⚠ Agent couldn't complete this request after 3 attempts.

routd also marks the chat as errored, so an operator can find it on the dashboard. A turn that failed without a retry, because the agent had already replied, gets a different notice: Failed: agent error on that message. Try rephrasing or send a different message.

The agent knows that a run is a retry

Before the agent sees the prompt of a retry, routd adds a note at the start:

<system-note>This is retry attempt 2 of 3. The previous attempt was
killed before completing. Be conservative with resource
usage.</system-note>

The rest stays the same: the same message and attachments and the same conversation history, in a new guest.

The retry count survives a restart

The attempt count is not in memory. Each retry increments turn_context.retry_count and sets the turn to state='pending_retry'. The crash-recovery sweep of routd picks up pending_retry turns in the same way as running ones. If routd restarts during a retry, the turn still gets its attempt, with the correct count of attempts already used.

Four kinds of turn never retry

Turns that replied
Any bot message, even a partial one, counts as done.
Turns that the agent failed on purpose
If the agent reports its own failure and the guest then powers off normally, the run is clean, and it never reaches the retry path.
Runs that an operator stopped
A deliberate stop is recorded as killed, not as an error.
Runs that runed marks terminal
A retry cannot repair these failures: a guest that does not start, or an exhausted model quota. The notice to the chat names the cause.
A network failure uses a different path. If the call from routd to runed fails on the network, routd does not know whether the run happened. So it leaves its read cursor in place, and the next poll gives the message to the agent again. Retries apply when runed answered and the answer was “the run stopped”.

One instance-wide setting controls retries

The setting is MAX_TURN_RETRY, and the default is 3. Set it to -1 to turn retries off and get the failure notice on the first failed run.

The setting applies to the whole instance, by design. No folder or skill can override it. The number of guests the runtime starts again is a property of the host, not of the content of a tenant. If turns stop because of memory and not because of content, change the instance-wide RUNED_GUEST_MEM, not the retry count. No folder can set its own guest size.

The final notice reports the value you set. If you set the limit to 5, it says “after 5 attempts”.

Related pages cover the guest and the progress

The guest and its time limit belong to runed. a turn, end to end shows where the retry fits in the turn. live progress shows what you see while a turn is still running.