Reconstruct a test environment from agent traces

The Slack call returned ok: true. The customer was still waiting. Your agent had posted a perfectly reasonable answer in the wrong thread.

The trace makes the bug easy to spot. Testing the fix takes another step: put the agent back in a channel with the same questions, run it again, and check where the reply lands. That is the small piece of production we will rebuild here.

The example below is fictional, with a bot you can run without a model or a Slack account. You will use its trace to write an Ancha scene by hand. There is no direct trace import in Ancha today.

Start with what the trace actually recorded

A history response tells you which messages the agent saw. It says nothing about the pages it never fetched. A successful write tells you what the API accepted; you may still need to look up what changed afterward. OpenTelemetry’s trace model records operations, and your instrumentation decides how much of each operation survives. Sampling can leave whole runs out.

Evidence you have	What you can put in a test	What remains unknown
A recorded read response	The returned records and fields	Unread records, omitted fields and pages not fetched
A recorded write	The action to investigate and its reported result	Pre-write state and side effects not observed
Timestamps and linked calls	Observed delays and causal relationships	The order of unrelated concurrent operations
Messages and incident notes	What a person said and when	What they would say to a different question

Beside the scene, note where each fact came from: the trace, an incident note, or an assumption. If channel membership is missing, look it up or write down what you assumed. An absent field does not mean the real field was empty.

The scene records what you know and what you had to supply. Ancha does not infer the missing context or import this trace automatically.

1. Choose where the test begins

For this bug, start just before the bot reads the channel. Both questions should already exist. The bot’s reply should not: we want the original code to make the mistake again.

The same choice matters for other failures. To test a duplicate refund, you might start before the first write. To test a retry after a timeout, you need to establish whether the first refund actually went through. A timeout alone will not tell you.

Before saving a real trace as a fixture, remove credentials and customer details. Use the same replacement ID everywhere a customer, channel or payment appears, or the records will stop joining. OpenTelemetry’s sensitive-data guidance covers fields that are easy to overlook.

2. Put the two questions back in Slack

The bot’s task is simple: answer the newest question in #support. At 14:00, it reads these messages and posts a reply:

conversations.history → newest first
  13:58  "How do I invite a teammate?"
  13:25  "Where is the monthly CSV export?"

chat.postMessage → ok: true
  thread_ts: the 13:25 question
  text: "Reports → Exports → Download CSV."

Slack returns history newest first. The bot uses findLast, so it picks the older CSV question. The answer fits that question. It just does not fit the task.

The history excerpt gives us the two questions. The fixture also needs users, channel membership and Ben’s earlier reply to the CSV question; those details are supplied separately. Message refs keep the threads connected, and relative timestamps put each message in the right place:

start_time: 2026-11-02T14:00:00Z
# Inside seed.slack.messages:
- ref: csv_question
  channel: $support
  user: $mike
  at: -35m
  text: "Where is the monthly CSV export?"
- ref: invite_question
  channel: $support
  user: $mike
  at: -2m
  text: "How do I invite a teammate?"

The full scene includes the users, membership, Ben’s reply and checks. You can also read the example trace; its JSONL format is for this walkthrough, not an Ancha import interface.

3. Check where the reply lands

Checking that chat.postMessage was called would pass on the broken bot. Check the destination thread and the answer as well:

checks:
  - id: answered_newest
    state:
      slack.message:
        channel: $support
        from: $bot
        in_thread_of: $invite_question
        text_contains: "Settings → Members → Invite"
  - id: posted_a_reply
    trace: { called: "POST /api/chat.postMessage" }
pass: [answered_newest, posted_a_reply]

answered_newest looks for the bot’s answer in the invite question’s thread. There is no bot reply in the seed, so the run has to create it. posted_a_reply checks that a write happened. Our bot uses fixed answer text, which makes a substring check sufficient here.

Both versions pass the write check. Only the fixed bot answers the question it was asked to answer.

4. Run the broken bot, then change one line

Use Node.js 24 and Ancha CLI 0.1.6, the versions used for the recorded results below. Download the files into a fresh directory. The bot uses Node’s built-in fetch; you do not need a model, Slack workspace or API key.

mkdir trace-reply && cd trace-reply
curl -fsS https://ancha.io/assets/examples/trace-reconstruction/bot.mjs -o bot.mjs
curl -fsS https://ancha.io/assets/examples/trace-reconstruction/scene.yaml -o scene.yaml
curl -fsS https://ancha.io/assets/examples/trace-reconstruction/trace.jsonl -o trace.jsonl
npm install -g @ancha-labs/cli@0.1.6
ancha init --agent "node bot.mjs"
ancha scene validate scene.yaml
ancha run scene.yaml --local -n 3
ancha explain latest --trial 1
ancha baseline save before

These commands run the downloaded scene.yaml, rather than the starter scene from ancha init. The --local flag runs the bot as a local process; it does not isolate its network access.

In bot.mjs, change messages.findLast( to messages.find(. Keep the scene and checks unchanged, then run:

ancha run scene.yaml --local -n 3 --compare before

Version	Correct thread	Write happened	Run exit code
Original bot	0 of 3	3 of 3	1
Fixed bot	3 of 3	3 of 3	0

The write check passes before and after the fix. The thread check catches the bug. These six trials used the supplied bot, with no LLM involved; they show that the fixture reproduces the failure and catches this fix. The per-trial results and fixture notes record exactly what ran.

Let the next run take a different path

After the fix, the bot writes to a different thread. That write becomes part of the simulated Slack state, where a later call can read it. The run can depart from the recorded sequence, within the tool’s supported API coverage.

For a longer incident, timing becomes part of the scene. If a manager replied two hours later, putting that reply in the initial data gives the agent information it did not have. Schedule fixed events with world rules, or describe people with facts and goals who can respond during the run. You still have to decide how they should behave when the agent asks something new.

If you already have an Ancha-recorded attempt, the replay command can start from that recording. In strict people mode, a question the recording cannot answer makes the replay invalid. Arbitrary production traces still need the reconstruction work described here.

Keep the test after the incident is closed

Before trusting the test, run the original code and confirm it fails for the reason you investigated. Then run the fix against the same scene and checks. A pass tells you about that reconstructed situation. Missing state and unsupported API behavior remain outside its scope.

Commit the scene and checks alongside the code, with the assumptions and run results. Once this case reproduces, try a channel with three unanswered questions, a question already answered by a teammate, or a delayed message. Keep checking which question the bot answers. The exact sequence of calls can change.

For the full SDK-based walkthrough, see Test a Slack bot. To choose between mocks, real workspaces and replicas, see the four approaches compared.