# Handoff: Synthetic Expense Reimbursement Release Gate

> **Demonstration artifact only.** This handoff describes a wholly fictional workflow and synthetic fixtures. It is not a client deliverable or evidence of a production result.

## Scope frozen for this sample

One workflow is covered: an employee submits an expense claim, and the agent reaches one terminal decision while making at most one side-effecting tool call.

In scope:

- input validation;
- receipt collection;
- prohibited-category rejection;
- duplicate-payment prevention;
- self-approval separation;
- manager and finance thresholds;
- reimbursement creation; and
- exact policy provenance.

Out of scope:

- receipt OCR or fraud classification;
- identity, role, or manager-directory resolution;
- currency conversion and tax treatment;
- payment-rail execution;
- policy authoring;
- user-interface rendering; and
- production infrastructure or on-call operation.

## Delivered artifacts

- A readable, versioned policy in `POLICY.md`.
- A machine-checkable case contract in `case.schema.json`.
- Twelve deterministic regression cases in `cases.jsonl`.
- Baseline and two candidate snapshots in `outputs/`.
- A deterministic local evaluator in `evaluate.py`.
- Evaluator unit tests in `tests/`.
- Generated JSON results and a comparison report in `results/` and `RELEASE_REPORT.md`.

## Runbook

1. From this directory, run `python3 evaluate.py --all`.
2. Review `RELEASE_REPORT.md` for the comparison and release decision.
3. Use `results/<snapshot>.json` for CI or downstream reporting.
4. Before replacing a candidate snapshot, preserve its label and case IDs so regressions remain attributable.
5. Add a new policy-dependent case before changing an expected outcome.
6. Treat `BLOCK` as a release blocker. Route `REVIEW` results to a named human owner.

Optional evaluator check:

```bash
python3 -m unittest discover -s tests -v
```

## Release decision

`candidate-v3` is the only releasable snapshot in this demonstration.

| Snapshot | Decision | Reason |
|---|---|---|
| `baseline-v1` | `BLOCK` | 8/12 pass; critical prohibited-category and duplicate-payout failures remain. |
| `candidate-v2` | `REVIEW` | 11/12 pass and no critical failures, but one required current-policy citation is missing and p95 latency is 5,200 ms. |
| `candidate-v3` | `PASS` | 12/12 pass; no critical failures; p95 is 2,050 ms; average cost is USD 0.011. |

The frozen gate is p95 latency at most 2,500 ms and average cost at most USD 0.025 per case.

## Extension points

- Replace synthetic claims with redacted, approved trace-derived cases inside the client's environment.
- Add a model or agent adapter that writes the same output contract.
- Add exact comparisons for user-visible messages, structured receipts, or generated documents.
- Separate model, retrieval, tool, and policy latency for diagnosis.
- Add repeated runs and statistical stability checks for non-deterministic models.
- Add CI wiring that blocks a deployment on `BLOCK`, allows `PASS`, and requires a named approval on `REVIEW`.
- Version policy migrations explicitly and retain old cases when backward compatibility matters.

## Accepted limitations

- All inputs and measurements are synthetic.
- Twelve cases demonstrate mechanics but cannot estimate real-world incident prevalence.
- The evaluator scores recorded outputs; it does not invoke an LLM, external service, or payment system.
- Nearest-rank p95 over 12 observations equals the maximum and is intentionally conservative, but it is not a production latency study.
- Required arguments are subset-checked, so a production adapter should separately validate its complete tool schema and reject unexpected sensitive fields.
- Exact citation matching verifies provenance identifiers, not whether the underlying policy text is legally or operationally correct.
- No security, privacy, concurrency, load, recovery, or end-to-end rendering claims are made.

## Operational owner at handoff

The adopting team owns its policy text, data approvals, production credentials, threshold changes, and final release authorization. The sample can be inspected without credentials or external network access.