When the user changes their mind mid-task

The assistant on hold#

Think of a personal assistant on hold with the restaurant, booking Friday, when you lean over and say “actually, make it Saturday”. A careless assistant hangs up and calls back about Saturday, and you end up with two tables. A good one asks the restaurant first: did that Friday booking go through? Then they book Saturday, and only once Saturday is confirmed, cancel Friday. And they tell you about both.

That’s the whole procedure, and the rest is how a runtime does each part when the “restaurant” is an API and the “assistant” is code.

Every run here comes from the previous article’s runtime, where a scripted model reads each phrase into a fixed result, so every difference between two runs comes from the runtime’s rules. In its logs, T1 is the first turn, one round of plan, act and reply, A1 the first consent request, and ... marks lines left out. IDs restart in each run, so R1 is whichever table that run booked first.

Four things a revision has to keep apart#

A revision touches four different things, and they change at different moments:

The runtime can’t see the bottom row directly. It only learns what happened in the world from replies, lookups and receipts. So the third row, pending, what it has sent without hearing back, decides its next move. A revision is reconciled once the runtime’s records and plan agree with the world again.

Those rows live in different parts of the runtime. The figure names them, and each one comes up below:

Reading the revision#

First the runtime has to know what “actually, Saturday” means. The model turns it into a structured revision of the running turn, revises T1: day=Sat. The runtime can’t catch a misreading; it acts on whatever the model reports. But for something unclear, like “Saturday too?”, which the model marks unsure, the runtime lets the call in flight finish and asks: “Do you mean Sat instead of Fri, or as well?”

With the new intent known, the runtime compares every booking the old turn holds against it:

1
           if op.step == "reserve_table" and op.state == "committed":
2
               op.release = False
3
               if self._matches(op.args, new):
4
                   kept.append(op.id)
5
...
6
               else:
7
                   op.release = True
8
                   release.append(op.id)
9
...
10
   @staticmethod
11
   def _matches(args, intent):
12
       return (args["day"], args["time"], args["party"]) == (intent.day, intent.time, intent.party)

A booking that still matches the new intent is kept. One that doesn’t is marked for release: it will be cancelled once its replacement holds, not now. “Actually, make it 20:00” doesn’t match R1, so R1 is released and 20:00 is booked. “Actually, just tell Sam, not the group” does match, so R1 is kept and the new plan skips booking altogether. What counts as “the same booking” is _matches: add a field there, and every revision compares it.

Stopping the wait doesn’t stop the work#

In the run at the top, the revision stopped the code that was waiting for the restaurant’s reply, and the log says so:

5  tool.sent         T1 reserve_table(Fri, 19:00, 4) key=s1:op-a2
6  restaurant        committed R1 Fri 19:00 x4
7  input             user: actually, Saturday
8  model.read        revises T1: day=Sat
9  tool.cancelled    T1 reserve_table: stopped waiting

Line 9 claims a cancellation the runtime has no evidence for. The restaurant committed R1 at line 6. None of the usual stop buttons reach the restaurant:

• Cancelling a Python asyncio task only arranges for a CancelledError “to be thrown into the wrapped coroutine”: it stops your code.
• OpenAI’s mid-turn steering “does not rewrite output already sent to your application, undo earlier actions, or cancel tools that have already started”.

Both stop the agent’s own work. The table stays booked. In the assistant’s terms, putting the phone down doesn’t undo what the restaurant already wrote in its book. So the reconciled runtime records the same moment differently. When a revision stops the wait, it marks any call that has an effect on the world as unknown, not cancelled:

1
except asyncio.CancelledError:
2
    if not self.dead:
3
        op.state = "unknown" if self.reconcile and c.effect != "none" else "cancelled"
4
        self.tl.add("tool." + op.state, f"{op.turn} {op.step}: stopped waiting")
5
    raise

reconcile=False is the run at the top. (The previous article’s restart logged tool.unknown at this point because it had this switch on; the run at the top turns it off.) With it on, a call with an effect is in doubt (unknown in the code and logs) until something resolves it. A pure read, like checking availability, has effect="none" and can simply be dropped.

Closing the key#

Every booking request carries an idempotency key, key= in the logs: an ID that lets the restaurant perform the request at most once, however many times it arrives. Here it’s s1:op-a2, session s1, operation op-a2. The restaurant can say, by key, whether a request landed, and the runtime asks it when it resolves an in-doubt call:

1
   def _resolve(self, op):
2
       c = self.contracts[op.step]
3
       if c.lookup and op.key:
4
           if c.fence and self.fencing:
5
               rid = self.restaurant.close(op.key)
6
               self.tl.add("resolve", f"{op.key}: {rid or 'nothing has landed'}; key closed")
7
           else:
8
               rid = self.restaurant.lookup(op.key)
9
               self.tl.add("resolve", f"{op.key}: {rid or 'nothing has landed'}")
10
...

There are two ways to ask. lookup just asks “did this key land?”. close asks the same and, in one step, tells the restaurant to refuse that key from now on: the assistant adding “and if a Friday request from me turns up later, ignore it”. fence in the tool’s contract says the restaurant supports closing, and fencing is the runtime’s switch for using it. The difference matters because a lookup that finds nothing only proves the request hasn’t landed yet. Requests can be delayed in the network.

Try it. Choose Reconciles with the restaurant first, then Is delayed in the network, and compare closing the key with only looking it up. (In the delayed runs Saturday is booked first, so it’s R1, and the late Friday is R2.)

With only a lookup, the runtime finds nothing, books Saturday, and then Friday lands anyway: two tables. With the key closed, the late request gets refused a late request: key s1:op-a2 is closed, and the restaurant ends with only Saturday:

The close turns the late Friday request into a refusal; a lookup alone would have let it commit.

It only works if the restaurant records the key and the booking together, so no request can slip in between checking a key and closing it. Without a close, the fallback is time: keep the call in doubt until a deadline the service enforces has passed.

Here’s the undelayed revision done right, from start to finish:

The runtime finds R1 before it books Saturday, and cancels R1 only after R2 holds, so the restaurant ends with one table.

Keys name the operation, not the attempt#

The previous article showed a key that’s too fine: built from the turn, so a restart got a new key and booked twice. The opposite mistake is a key that’s too coarse, built from the step name alone, s1:reserve_table: the assistant filing everything under “my booking”. Then Friday and Saturday share a key. On a server that can look keys up but not close them, and replays by key without checking the request’s details, the Saturday request just gets Friday’s result back. And because the runtime then compensates for the old booking, undoing it by cancelling it, it cancels the only table there is:

17  tool.sent         T2 reserve_table(Sat, 19:00, 4) key=s1:reserve_table
18  restaurant        replayed R1 for key s1:reserve_table
19  tool.result       T2 reserve_table -> R1
20  compensate        R1: the replacement holds
 ...
22  restaurant        cancelled R1
 ...
-- restaurant holds: nothing

The runtime records the result as Saturday, then cancels “the old Friday booking”, R1, which is the same table. A server that does compare details, as Stripe’s does (“errors if they’re not the same”), turns this into a visible KeyMismatch, and the runtime keeps Friday and asks (key-reuse-strict). A cheap guard, which the demo doesn’t have, would catch it too: never cancel a booking whose ID the replacement just returned.

So the key names the operation: a booking whose details haven’t changed keeps its key and is never sent twice, and Saturday is a new operation with a new key, s1:op-a4.

Or wait for the call to finish#

Resolving by key needs a service that can be asked. When it can’t, the runtime can let the call finish, stop at the next step, and work with an outcome it knows. The cost is that the revision waits for the slowest call in flight. The choice is per tool, in its contract:

1
CONTRACTS = {
2
    # A read. Safe to drop mid-flight here; a read that places a hold would not be.
3
    "check_availability": Contract(effect="none", needs_consent=False),
4
    "reserve_table": Contract(effect="reversible", needs_consent=True, keyed=True, lookup=True,
5
                              fence=True, undo="cancel_reservation"),
6
    # No lookup, so a revision waits for the delivery receipt, up to a bound (50 ms in the demo).
7
    "text_group": Contract(effect="irreversible", needs_consent=True, wait=0.05),
8
    # Irreversible: a released table can be taken by someone else. Acts for the user, so a
9
    # turn the user didn't start must ask; a fee needs the user's consent whoever started it.
10
    "cancel_reservation": Contract(effect="irreversible", needs_consent=True),
11
 ...
12
}

Read each line as what the runtime may assume about that tool. Checking availability has no effect, so dropping it is free. Reserving is reversible, keyed, can be looked up and closed, and its undo is cancel_reservation. The group text can’t be looked up or undone, so a revision waits up to 50 ms for its delivery receipt.

When the receipt comes in time, the runtime knows the Friday text went out and plans a correction. When it doesn’t, the runtime treats the text as sent and tells the user “I couldn’t confirm it, so I’ve treated it as sent”. Assuming delivery is safe here because a “change of plan” text to people who never saw the first plan is merely odd.

Secure the replacement, then undo the old#

With R1 confirmed and no longer wanted, the runtime has to choose what to do first. Cancel Friday first and then try Saturday, and if Saturday is full the user is left with nothing. So the default is the other order: the releases wait in the new turn’s release list, and the runtime works through them only after the new booking succeeds. If Saturday is full, R1 comes off the list, and you’re asked: “You still hold R1 (Fri 19:00); keep it, or shall I cancel it?”

Try both orders with Saturday full:

Securing first means holding two bookings for a while. Where that costs money or blocks something, like a payment hold against a card limit, releasing first is the better order, and the user should hear so before the runtime acts.

20  compensate.ask    cancelling R1 costs a 20 fee: A1 needs the user's consent
 ...
25  input             user: yes
26  approval          A1: runs exactly its 1 step(s)
 ...
28  tool.sent         T3 cancel_reservation(R1, 20)
29  restaurant        cancelled R1 (fee 20)

-- agent's last message: I may have booked Fri 19:00: the restaurant never confirmed, and I can't check. If a confirmation arrives, tell me and I'll cancel it. Booked Sat 19:00 for 4 (R2). Texted group.

A crash in the middle#

Everything so far happens in one process. Now kill it at the worst moment: after the restaurant commits Saturday but before the runtime hears about it. A new process starts, reading the same store, the database the runtime writes its records to. What the new process can do depends on what was written there before the requests went out: in the assistant’s terms, whatever they wrote on their notepad before picking up the phone. The runtime writes down two things:

• each operation, with its key, before its request is sent (the _op method from the previous article);
• the running turn’s record: what it’s trying to do, and the releases it still owes, such as “cancel R1 once Saturday holds”.

1
def _write_turn(self, turn):
2
    """The running turn's record: what a new process needs to resume it."""
3
    self.store.turn = {"id": turn.id, "origin": turn.origin, "intent": turn.intent, "fixed": turn.fixed,
4
                       "correction": turn.correction, "release": turn.release, "kept": turn.kept,
5
                       "facts": turn.facts}

The record’s fields are the turn’s ID and origin, its intent and fixed steps, any correction it owes the group, the releases and kept bookings, and the facts it will report. With both records kept, the new process runs recover(): it resolves every call the log shows in flight, by key where the contract allows, then resumes the turn from its record, with the operations that belong to it (own).

1
   async def recover(self):
2
       """A new process, same store: resolve what was in flight, resume the turn, then
3
       replay any input that was acknowledged but not yet acted on."""
4
       self.tl.add("restart", f"{len(self.store.ops)} operation(s) in the log")
5
       for op in list(self.store.ops.values()):
6
           if op.state in ("sent", "unknown"):
7
               self._resolve(op)
8
       rec = self.store.turn
9
       if rec is not None:
10
           own = [op for op in self.store.ops.values() if op.turn == rec["id"]]
11
...

The Saturday booking comes back as skip reserve_table: op-a4 already committed as R2, R1 is cancelled, and the restaurant holds one table.

Now suppose the store kept the turn but not the operations: a design that saves the conversation between steps, but not the calls it has in flight. Try both:

With no record of the operations, the resumed turn doesn’t know Saturday was already booked as R2, so it books Saturday again, as R3, under a fresh key. And the release it owes points at a Friday operation that no longer exists, so R1 is never cancelled. Three tables: R1, R2 and R3.

What the history says, and what the user hears#

Last, the agent’s own memory, the assistant’s diary. After a reconciled revision where the Friday text had already gone out, the history the model sees reads:

user: book Friday 19:00 for 4 and tell the group
tool: Fri 19:00 for 4: available
tool: booked R1 (Fri 19:00 for 4)
tool: texted group: 'table for 4, Fri 19:00'
user: actually, Saturday
tool: Sat 19:00 for 4: available
tool: booked R2 (Sat 19:00 for 4)
tool: cancelled R1
tool: texted group: 'Change of plan: table for 4, Sat 19:00'
assistant: The earlier text had already gone out. Booked Sat 19:00 for 4 (R2). Cancelled R1 (Fri 19:00). Texted group a correction.

Three rules made it read like that:

• It keeps the user’s own words, including the revision.
• Records of effects are never removed. A revision adds resolutions, cancellations and corrections; it doesn’t erase “booked R1”.
• Each record sits where its effect happened. The Friday text’s receipt arrived after “actually, Saturday”, while the revision was waiting for it, yet its record sits before the revision, because that’s when the group got the text.

That last rule has a cost: writing into the past breaks a prompt cache from that point, since “Cache hits require 100% identical prompt segments”. The alternative is to append a record that says when the effect happened, which keeps the cache at the price of a less direct story. Either way, if the history is ever summarised to save space, the summary must not drop “booked R1”, or the next turn inherits the blind spot of the run at the top.

And the last line is what the user hears: what the records say, including that the Friday text had already gone out.

The contract each tool needs#

Every tool call in this runtime moves through the same states, and every way out of “in doubt” needs something from the service or the contract:

So for every tool your agent can call, write down:

1. Its effect: none, reversible, or irreversible. A read that places a hold has an effect.
2. How to undo or correct it: the compensating action, with its cost, cut-off and refusal cases, and which costs need the user’s consent, bound to the amount.
3. What the service offers for calls in doubt: idempotency keys, a lookup by key, a way to close a key, a deadline after which requests are refused, and how long keys are kept.
4. What a revision does while the call is in flight: stop and resolve it, or wait, and for how long.
5. Which order to use: secure the replacement first or release first, and why.
6. What the history and the user are told in each state, including “treated as sent” and “outcome unknown”.

Then store each operation, and each release owed, before the request that depends on it.

The demo#

It’s the same runtime as the previous article’s, in turn-without-the-user/. Every log here is one scenario’s output, and its README maps each section to its scenarios.

cd mainlog-code/turn-without-the-user
python -m agent_runtime.demo revise-in-flight-naive   # the run at the top
python -m agent_runtime.demo revise-in-flight         # the same revision, reconciled
python -m agent_runtime.demo revise-after-text --history
python -m pip install pytest && python -m pytest tests

It shows each rule in one process and isn’t hardened: its README lists what it doesn’t do, starting with durability, which is simulated by a Python object standing in for a database.

The run at the top ended with two tables and a history that never mentions Friday. With the contract filled in, the agent finds out what happened to Friday before it books Saturday, undoes it only once Saturday holds, and says so, in its history and to the user.