An agentic trading system without an eval harness is a system that is being evaluated in production, by the market, on a schedule the firm doesn't choose. This brief builds the alternative.
Eval is the part of the agentic stack that gets cut first when timelines slip and is the part regulators and incident reviewers ask about first.
A defensible eval harness covers the model's reasoning, the tool surface, and the orchestrator — not just the model in isolation. Most public benchmarks cover none of these.
This brief produces a working harness, a red-team report, and an audit-trail schema designed to survive review by a regulator who has never seen the system before.
Three methodology steps from the standing approach, scoped to this brief.
Frame
Read the regulator filings, the codebase, or the internal memo. Write the question that is actually being asked.
Build the artifact
The artifact named in Outputs, above. Working notes during the build are visible.
Hand it off
A meeting, not a link. Six weeks of follow-up Q&A is included.
This is not a continuous evaluation operations function — once delivered, the harness is the firm's to run.
This is not a model benchmark report — public benchmark numbers do not appear in this brief.
This does not replace formal model-risk-management procedures the firm may already have.
Begin
Send the question.