VERIFISDK docsengine v2 · contract 1.5.1 · harness-sdk 1.2.0
Harness authoring

Run lifecycle

runHarness, the handshake, the context it builds, the dry-run path, and how a run ends.


runHarness is the container entrypoint. It owns the plumbing a harness author never writes: the handshake, the context, the ledger, event flushing and a clean shutdown.

Source: verifi-harness-sdk/src/runtime.ts.

Signature

export async function runHarness(
  harness: Harness,
  manifest: RunManifest,
  channel: EngineChannel,
  opts?: {
    now?: () => number;
    fetchImpl?: typeof fetch;
    region?: string;
    credentials?: { accessKeyId: string; secretAccessKey: string; sessionToken?: string };
    /** Overrides what is scrubbed from tool output. Defaults to this container's own. */
    secrets?: string[];
  },
): Promise<RunOutcome>;
export interface RunOutcome {
  status: 'done' | 'failed';
  findings: number;
  coverage: string[];
  exitReason: string;
  /** True when this was a dry run — no model, no tools, no spend. */
  dryRun: boolean;
}

opts.now and opts.fetchImpl are injection seams for tests. opts.credentials overrides the environment read and opts.secrets overrides the redaction list; leave both unset in production, where the Provisioner supplies the credentials and the SDK collects the redaction list from the container's own environment.

The shape of a whole run

Engine Harness container MCP (slot) provision + probe the slot endpoints written into the manifest start container RUN_MANIFEST · RUN_TOKEN · ENGINE_GRPC_ADDR (env, not over the wire) dial Open(stream UpMessage) → stream DownMessage authorization: Bearer $RUN_TOKEN handshake at seq 0 re-sent on every re-dial runHarness: version gate → ledger → slot + model resolvers MAJOR mismatch here throws ConnectorVersionError tools/call JSON-RPC 2.0 over streamable-http, POST /mcp ToolResult { ok, text, latencyMs } a failed tool is ok:false — it never throws finding { payloadJson } emitted as produced, never batched ack(seq) advances the cursor, trims the replay buffer done(outcome), then wait ≤5s for its ack close() drains ≤2s, then exit 0 | 1 the container serves nothing and is torn down
The manifest arrives in the container’s environment, not over the channel. A harness that had to call the engine to learn what to run would have a bootstrap dependency on the very channel it is about to open. Everything after the handshake is a resumable stream: the harness holds unacked messages in a replay buffer and the ack cursor is what lets a run survive the engine being replaced underneath it.

The order of operations

1. isCompatible(harness.connectorVersion)   → throws ConnectorVersionError if not
2. channel.handshake(...)                   → awaited before anything else
3. AbortController wired to channel.onStop
4. BudgetLedger built from manifest.budget, onBreach → threshold_breach event
5. redaction list collected (opts.secrets ?? collectRunSecrets())
6. MCP resolver built from manifest.slotFills ?? manifest.endpoints
7. credentials read (opts.credentials ?? credentialsFromEnv())
8. model resolver built ONCE — or left null if there are no credentials
9. ctx assembled
10. dryRun?  → streamDryRun(...)     : harness.run(ctx)
11. channel.sendDone(outcome)
12. finally → channel.close()

Step 1 happening first is the point. The previous generation of harnesses drifted from the engine and broke silently, so a version mismatch must be a loud failure at connect time rather than a confusing error thirty minutes into a scan.

Step 8 happening once matters too: the resolver memoizes a client per role, and rebuilding it per call would throw that away along with anything a client accumulates.

HarnessContext

This is everything your run() receives.

export interface HarnessContext {
  runId: string;
  correlationId: string;
  binding: RunBinding;                       // { tenantId, assessmentId, assetId }
  input: Record<string, unknown>;            // = manifest.parameters
  policy: RunPolicy;
  budget: BudgetLedger;
  /** Fires on stop, deadline or operator cancel. Never on cost. */
  signal: AbortSignal;
  mcp(slot: string): McpSlot;
  model(role: string): ModelSlot;
  emitEvent(e: { type: EventType | string; severity?: string; payload?: Record<string, unknown> }): void;
  emitFinding(f: RawFinding): void;
  log(msg: string, extra?: Record<string, unknown>): void;
  /** What this run is authorized to touch. */
  readonly target: TargetScope;
  /** ISO-8601. Time is the only hard bound on a run — nothing else stops it. */
  readonly hardDeadline: string;
}

A few things worth knowing that are not obvious from the types:

  • ctx.input is manifest.parameters. It is not your scope. Your scope is ctx.target.scope.
  • ctx.target and ctx.hardDeadline are new in harness-sdk@1.0.0. Before them a harness could not read its own authorized boundary: the manifest carried target and hardDeadline and the context withheld both, so the ASM harness took its domain as an input parameter instead — leaving the scope the engine authorized and the target actually scanned as two unreconciled pieces of data. A harness that cannot see its boundary cannot respect it. Check what you are about to touch against ctx.target.scope and ctx.target.excludes rather than trusting an input to agree with them.
  • ctx.emitFinding is typed RawFinding, from @pragyacyber/engine-contract. The compiler checks the field names now. See Findings and events.
  • ctx.log(msg, extra) is sugar for an event of type log at info severity with payload { msg, ...extra }.
  • emitEvent defaults severity to 'info' and payload to {}. Its type is EventType | string, so the closed enum autocompletes but a novel string still compiles.
  • ctx.model() throws with a clear message when no AWS credentials were injected. This is normal in local development, and it is why a harness whose findings come from a rubric should declare its model role required: false and never call it.
// Respect the boundary you were given, rather than the one you were passed.
const inScope = (url: string) =>
  ctx.target.scope.some((s) => url.startsWith(s)) &&
  !ctx.target.excludes.some((s) => url.startsWith(s));

if (!inScope(discovered)) {
  ctx.emitEvent({ type: 'scope_expansion', severity: 'warn', payload: { host: discovered } });
  continue;   // report it; do not scan it
}

The dry run

manifest.parameters.dryRun === true diverts the whole run. harness.run() is never called.

async function streamDryRun(harness, ctx, channel, ts) {
  const plan = harness.declarePlan(ctx.input);
  for (const phase of [...plan.phases].sort((a, b) => a.order - b.order)) {
    channel.sendEvent({ type: 'phase_start', /* … */ payload: { phase: phase.key, label: phase.label, dryRun: true } });
    channel.sendEvent({ type: 'phase_end',   /* … */ payload: { phase: phase.key, dryRun: true } });
  }
  return {
    status: 'done',
    findings: 0,
    coverage: plan.coverage,
    exitReason: 'dry run: wiring verified, no model or tool call made',
  };
}

Note the phases are sorted by order before streaming — which is why defineHarness insists the orders are unique.

Stopping

channel.onStop(reason) aborts the run's AbortController, and ctx.signal is that controller's signal. Both the MCP client and the model client:

  1. check signal.aborted first, before registering a listener, because addEventListener('abort') on an already-aborted signal never fires — a run stopped between two tool calls would otherwise sit until the request timed out; and
  2. register an abort listener that cancels the in-flight HTTP request, so a cancelled run stops scanning the client's estate immediately rather than at the next await point.

The gRPC Stop message carries 'deadline' | 'operator' | 'failure'. There is no cost reason, by design.

You are not obliged to check ctx.signal yourself, but a long non-network loop should:

for (const asset of assets) {
  if (ctx.signal.aborted) break;
  await probe(asset);
}

How a run ends

PathstatusexitReason
run() returns normallywhatever you returnedwhatever you returned
run() throws'failed'the error's message (or String(err))
dry run'done'dry run: wiring verified, no model or tool call made

In every case channel.sendDone(outcome) is awaited and then channel.close() runs in a finally. A failed run still reports, and findings already streamed are unaffected — run outcome and finding persistence are independent.

GrpcEngineChannel.sendDone waits for the engine to ack (up to 5s) before returning, specifically so close() does not tear the connection down while the last writes are still buffered. A run's own outcome is the worst possible thing to drop.

Findings reconciliation

const result = await harness.run(ctx);
const outcome = { ...result, findings: result.findings || findings };

findings on the right is the runtime's own count of emitFinding calls. Returning 0 from run() while having emitted findings is therefore safe — the runtime substitutes its count. Returning your own count is also safe. Note the ||, not ??: a returned 0 is always replaced by the emitted count.

Errors the runtime can throw at you

ErrorWhen
ConnectorVersionErrorharness.connectorVersion major ≠ CONTRACT_VERSION major, or either is malformed. Thrown before the handshake.
SlotNotFilledErrorctx.mcp(slot) for a slot with no entry in slotFills or endpoints. Lists the slots that were filled.
ModelRoleNotBoundErrorctx.model(role) for a role with no entry in composition.modelIds. Lists the bound roles.
plain Errorctx.model(...) when no AWS credentials were injected into the container. The message names it as a provisioning fault.

The first three are exported classes; the last is a bare Error.