Skip to content
GitHub ↗Get started
Controlled agent evaluation

An agent saying “done” is not the outcome.

ArtemisKit’s agent workflows define a task, tools, authority and success criteria before execution. The runner can use simulated or disposable Docker environments, capture bounded evidence and check outcomes independently. Execution, task success, policy status and unavailable measurements remain distinct, so a completed conversation is not mistaken for a completed job.

What is declared up front?

  • The target provider and model.
  • The tool catalogue and allowed operations.
  • Permissions, network policy, side-effect rules and budgets.
  • Initial fixtures, workflow turns and independent outcome assertions.

Where does the agent run?

Simulated environments support controlled, fixture-based tests. Disposable Docker environments support bounded real execution. Docker, target transport and the declared tools have separate requirements. Read the execution guide before running a workflow.

How are outcomes checked?

Deterministic assertions inspect observed state or artifacts where possible. Optional bounded semantic judging needs an explicitly configured independent judge. An agent’s own completion message is never sufficient evidence of task success.

What about interruptions?

Recovery controls retain budgets and authority across compatible pause/resume attempts. Operational checkpoints contain sensitive working state and remain separate from sanitised saved evidence. Recovery guarantees apply only to documented boundaries.

How do I get a report?

akit workflow report evidence.json --output assessment.html
akit workflow report evidence.json --view executive --format markdown --output executive.md

Offline reports project saved evidence without another model call. Export success means a report was produced; it does not mean the system passed.