Replay

Replay answers the question every team editing a policy eventually asks: if I had this rule in place last week, what would it have done?

It takes a historical execution, walks every tool call the agent evaluated, re-runs each one against the policy set you specify, and returns a side-by-side diff. Nothing is written to the timeline — replay is a pure analysis pass.

Endpoint

POST /v1/executions/{execution_id}/replay
Authorization: Bearer <KLENT_API_KEY>
Content-Type: application/json

Request body

Three shapes, all valid:

Against the project's currently enabled policies (most common):

{}

Against a specific subset (by ID, must belong to the project):

{ "policy_ids": ["pol_abc", "pol_def"] }

Against an inline policy set (useful for a what-if before saving):

{
  "policies": [
    {
      "name": "Block high-value transfers",
      "enabled": true,
      "enforcement_mode": "enforce",
      "effect": "deny",
      "conditions": [{ "field": "input.amount", "operator": "greater_than", "value": 10000 }]
    }
  ]
}

Response

{
  "execution_id": "exec_…",
  "policy_count": 3,
  "total": 12,
  "summary": {
    "unchanged": 9,
    "flipped": 2,
    "policy_swapped": 1,
    "flips": { "allow_to_deny": 2, "deny_to_allow": 0, "other": 0 }
  },
  "turns": [
    {
      "event_id": "evt_…",
      "tool": "transfer_funds",
      "input": { "amount": 50000, "currency": "USD" },
      "original": {
        "decision": "allow",
        "matched_policy_id": null,
        "reason": "No policy matched; default allow"
      },
      "replayed": {
        "decision": "deny",
        "matched_policy_id": "pol_new",
        "reason": "Matched policy \"Block high-value transfers\""
      },
      "change": "flipped"
    }
  ]
}

change values

Use summary.flips.allow_to_deny to count "actions the agent performed that a new policy would have blocked" — the number that tells you the impact of rolling out a restrictive rule.

In the dashboard

Every execution detail page has a Replay against current policies button. It runs the replay in-process (no HTTP round-trip), shows the three stat cards (unchanged / policy swapped / flipped), a callout summarizing the flip direction, and a table with each original→replayed decision pair.

Each event on the execution timeline also has an inline replay link that deep-links to the same page scoped to that event (/executions/{id}/replay?event={event_id}) — it scrolls to and highlights the matching row so you can jump straight from a single tool call to its replay.

What replay does not replay

• Agent reasoning. Replay re-evaluates the policy layer. It does not call the LLM again, so the sequence of tool calls is the one that actually happened — not the one the agent would now make given different responses.
• Tool execution. Replay is read-only. No tools run. No events are written.
• Shadow decisions. Shadow policies match during replay but, as in live evaluation, they do not change the decision column.

Typical workflows

• Before enabling a new policy: replay recent executions to see how many actions it would have blocked. If the number is higher than expected, tighten the conditions.
• Before deleting a policy: replay to see which executions relied on it — policy_swapped rows point at actions where a different policy now takes over, which may or may not be intended.
• After an incident: reconstruct what would have happened had a compensating policy existed at the time. Use the output as evidence in a postmortem or compliance report.