Models — ctx.model()

ctx.model(role) is the only way a harness reaches a model, for the same reason ctx.mcp() is the only way it reaches a tool: if a harness could construct its own client, model spend would leave the ledger. Since metering replaced every hard cap except time, metering a subset of reality is worse than not metering at all.

Source: verifi-harness-sdk/src/model-client.ts, plus verifi-engine-contract/src/model-invocation.ts.

The surface

export interface ModelSlot {
  readonly role: string;
  readonly modelId: string;
  invoke(prompt: string, options?: ModelInvokeOptions): Promise<ModelResult>;
}

export interface ModelInvokeOptions {
  system?: string;
  maxTokens?: number;        // default 4096
  temperature?: number;
  stopSequences?: string[];
}

export interface ModelResult {
  text: string;
  inputTokens: number;
  outputTokens: number;
  costUsd: number;
  latencyMs: number;
  ok: boolean;
  /** Populated when the call failed. */
  error?: string;
}

const analyst = ctx.model('analyst');

const res = await analyst.invoke(
  `Summarise the security impact of these findings:\n${JSON.stringify(items)}`,
  { system: 'You are a penetration testing analyst. Be terse and concrete.', maxTokens: 800 },
);

if (!res.ok) {
  ctx.log('narrative synthesis unavailable', { error: res.error ?? 'unknown' });
} else {
  narrative = res.text;
}

A failed invocation returns ok: false; it does not throw. Same reasoning as a tool call: a harness that loses one model call should decide for itself whether that is fatal, and a thrown error at an arbitrary await point would discard findings already made.

A role is a developer-chosen string, declared in defineHarness({ models: [...] }) and bound to a Model by the Service composition. Calling ctx.model() for an unbound role throws ModelRoleNotBoundError, which names the roles that are bound. The same Model may fill several roles.

Auth is per-run, and short-lived

The container receives short-lived credentials minted per run by the CredentialBroker, and holds no long-lived key. resolveAwsCredentials() gets them from wherever the runtime actually publishes them: credentialsFromEnv() reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and optionally AWS_SESSION_TOKEN first, and when those are absent — which is the normal case on Fargate, where a task role's credentials are not environment variables — it falls back to the ECS container-credentials endpoint named by AWS_CONTAINER_CREDENTIALS_RELATIVE_URI (or _FULL_URI). The client then signs each InvokeModel POST with SigV4 itself.

The SigV4 canonical URI is deliberately encoded a second time — a real SigV4 rule for every service except S3. A Bedrock model id carries a colon, so invoking amazon.nova-pro-v1:0 signs the path …nova-pro-v1%253A0…; without the double-encode the request is a bare 403 that reads like a permissions problem rather than a signing one. This was the other half of the 1.1.0 model-path fix.

That short-lived-credential design is a decision, not an implementation detail: there is no long-lived secret to leak, rotate, or lose. Do not read, print, log, or write any of them anywhere.

Cost is priced from a human-reviewed table

export const MODEL_PRICES_PER_MTOK: { match: string; input: number; output: number }[];
export function priceFor(modelId: string): { input: number; output: number } | null;

Per 1M tokens, keyed by a substring of the model id. Deliberately explicit rather than fetched: a wrong price is a wrong cost threshold, and a cost threshold is the only thing standing between a runaway and an unnoticed bill — so this table is something a human reviews, not something that changes underneath us.

Note the units. zModel.costModel on the registry record is per 1k tokens; this table is per 1M. Do not copy numbers between them.

How a Model reaches available

A Model does not get the three-gate probe. It has no HTTP endpoint to three-gate — a Bedrock model is not a server — and giving all four registries the same endpoint probe is why the models registry was, for a while, structurally unable to work.

Instead the engine performs a real invocation with a trivial deterministic prompt, and the pass condition is that the invocation returned non-zero token counts.

resolve auth        keyed provider → read secret:<id>  ·  bedrock → the task role
build the request   buildModelRequest(provider, invocationId ?? modelId, prompt, opts)
POST it             modelEndpoint(provider, opts) + req.path
parse it            parseModelResponse(provider, body)
PASS  ⟺  inputTokens + outputTokens > 0

zModel then refuses status: 'available' without probeInvoked: true. That flag is P12 in the type system: listing is not availability.

The shared request/response shapes

@pragyacyber/engine-contract publishes the per-provider wire shapes as pure functions — no fetch, no credentials, no signing, no environment reads.

export function isKeyedProvider(provider: ModelProvider): boolean;
export function modelEndpoint(provider: ModelProvider, opts: ModelRequestOptions): string;
export function buildModelRequest(
  provider: ModelProvider, modelId: string, prompt: string, opts?: ModelRequestOptions,
): { path: string; body: string; headers: Record<string, string> };
export function parseModelResponse(provider: ModelProvider, payload: unknown): {
  text: string; inputTokens: number; outputTokens: number;
};

They live in the contract because two things invoke models — the engine's registry probe and the SDK's ctx.model() — and two copies of one wire format is two chances to be subtly wrong about it. The failure mode is specific and bad: a probe that passes against a shape the SDK never sends. A model would read available and then fail on every real call. That is the Burp failure with a different subject.

isKeyedProvider is false for bedrock alone. That is why P6 prefers it: there is no long-lived key to leak, rotate or lose.

Three refusals in there are load-bearing, and all three were verified by calling them:

modelEndpoint('openai_compat', {})   THROWS  needs an explicit baseUrl
modelEndpoint('bedrock', {})         THROWS  needs a region
buildModelRequest('local', …)        THROWS  no request shape is defined for provider 'local'

Defaulting openai_compat to api.openai.com would send a self-hosted or NIM deployment's traffic and its API key to a third party. Defaulting a Bedrock region produces a silent 403. Neither is a convenience.

Bedrock is a HOST, not a model family

Bedrock fronts many families and each has its own request body. The contract used to emit an Anthropic body for every Bedrock model — right for anthropic.*, wrong for everything else — and the failure is a 400 from the provider that reads like a broken model rather than a wrong request.

buildModelRequest('bedrock', 'anthropic.claude-3-5-sonnet-20241022-v2:0', 'hi', { maxTokens: 16 });
// → { anthropic_version: 'bedrock-2023-05-31', max_tokens: 16, messages: [{ role: 'user', content: 'hi' }] }

buildModelRequest('bedrock', 'amazon.nova-pro-v1:0', 'hi', { maxTokens: 16, system: 'be terse' });
// → { messages: [{ role: 'user', content: [{ text: 'hi' }] }],
//      system: [{ text: 'be terse' }],
//      inferenceConfig: { maxTokens: 16 } }

Note the model is in the path for Bedrock. Repeating it in the body is a 400.

A family the contract has no shape for is refused, not guessed at:

buildModelRequest('bedrock', 'meta.llama3-1-70b-instruct-v1:0', …)
  THROWS  no Bedrock request shape is defined for 'meta.llama3-1-70b-instruct-v1:0'.
          Bedrock hosts many model families and each has its own body; sending an
          Anthropic body to another family produces a 400 that reads like a broken
          model. Add the shape here rather than guessing.

Nova's response is parsed first, too, because it puts its token counts in usage.inputTokens where the Anthropic branch would read usage.input_tokens and see zero — and zero tokens is how the probe decides a model never ran. A working Nova model would have been reported as a listed-but-dead one.

The inference-profile trap

New in contract 1.5.1. Every modern Anthropic model on Bedrock is reachable only through a cross-region inference profile — an id like us.anthropic.claude-opus-4-8. An on-demand call to the bare id returns ResourceNotFoundException.

Dispatching the family off the raw string therefore made startsWith('anthropic.') false for all of them. Each was refused as an unknown family; because available is reachable only through a passed probe, none could ever be selected. The console reported exactly that: Anthropic models present but non-selectable.

The fix is one line, and the asymmetry in it is the point:

// The FAMILY is read from the base id; the PATH keeps the profile id.
const family = modelId.replace(/^(?:us|us-gov|eu|apac|global)\./, '');

The prefix is a routing hint, not a family. It has to stay in the path — that is the thing you actually invoke — and come off before the family test. Verified against 1.5.1:

'anthropic.claude-3-5-sonnet-20241022-v2:0'  → anthropic body · path /model/anthropic.claude-3-5-sonnet-20241022-v2%3A0/invoke
'us.anthropic.claude-opus-4-8'               → anthropic body · path /model/us.anthropic.claude-opus-4-8/invoke
'eu.amazon.nova-pro-v1:0'                    → nova body      · path /model/eu.amazon.nova-pro-v1%3A0/invoke
'global.amazon.nova-pro-v1:0'                → nova body
'apac.amazon.nova-pro-v1:0'                  → nova body
'us-gov.amazon.nova-pro-v1:0'                → nova body
'us.meta.llama3-1-70b-instruct-v1:0'         → THROWS (unknown family, still)

That last row matters as much as the others: stripping the prefix must not turn "unknown family" into a guess.

This is also what invocationId on a model registration is for. modelId is the model; invocationId is what you actually invoke with when the two differ, which for most current Bedrock models means the profile id. The engine probe calls buildModelRequest(provider, entry.invocationId ?? entry.modelId, …).

The SDK and the contract share one wire format

This was the one real divergence this page used to name, and 1.1.0 closed it. Two things that were broken before now work:

• A non-Anthropic Bedrock model bound to a role works on every call. The probe shapes a Nova request through the contract and marks the model available; ctx.model() shapes the same Nova body from the same builder. The "probe passes against a shape the SDK never sends" failure is designed out rather than merely absent.
• Every modelled provider is reachable from a harness. bedrock signs with the run's role; anthropic, openai_compat, nim and vertex are keyed providers — ctx.model() reads isKeyedProvider(provider) from the contract and attaches the injected key (x-api-key for anthropic, authorization: Bearer for the rest) instead of signing. A keyed provider with no key injected returns ok: false naming the fault, not a wrong request.

A provider or model family the contract refuses is surfaced verbatim as ok: false rather than sent a guessed body — the same refusals listed above (local, an unknown Bedrock family, openai_compat/nim without a baseUrl). Composition still decides which model fills a role; the SDK no longer second-guesses its wire format.

Per-role keyed-model config

Attaching a key (the previous section) is only half of what a keyed role needs. The other half is knowing that the role is keyed in the first place — its provider, its baseUrl, and the key itself. Before 1.2.0, createModelResolver set none of these per role, so every role fell to the bedrock default (signed with the task role, no key). That is right for Bedrock and wrong for a keyed provider: a NIM or OpenAI-compatible sub-agent was built as Bedrock, failed buildModelRequest with "no Bedrock request shape for <model>", and the role was silently unusable. A two-model service mixing a Bedrock lead with a keyed sub-agent could not actually run the second model.

As of 1.2.0, createModelResolver reads a per-role provider, base URL and API key from the environment the engine injects, and passes any that are present through to the ModelClient. For each role it normalises the role name to an env-var key —

const key = role.toUpperCase().replace(/[^A-Z0-9]/g, '_');
const provider = env[`VERIFI_MODEL_${key}_PROVIDER`];   // e.g. 'nim'
const baseUrl  = env[`VERIFI_MODEL_${key}_BASEURL`];
const apiKey   = env[`VERIFI_MODEL_${key}_APIKEY`];

— so a role named sub-agent reads VERIFI_MODEL_SUB_AGENT_PROVIDER and friends (upper-cased, every non-alphanumeric character folded to _).

The engine is what populates those vars: it projects a per-role modelConfigs { provider, baseUrl?, authRef? } from its registry and injects the VERIFI_MODEL_<ROLE>_* values into the harness container, resolving a keyed provider's API key from AWS Secrets Manager so the plaintext lands only in the container environment — never in a manifest, a composition snapshot, or a log. The SDK reads it here and nowhere else.

Absent env falls back to the Bedrock defaults, unchanged. A role with no VERIFI_MODEL_<ROLE>_PROVIDER behaves exactly as it did before 1.2.0 — provider: 'bedrock', signed with the run's task role. So this is additive: a pure-Bedrock service is byte-for-byte the same, and a service that mixes a Bedrock role with a keyed (NIM / OpenAI-compatible / Anthropic / Vertex) role now runs both models instead of silently dropping the keyed one.

If your findings can be deterministic, make them deterministic

The production ASM harness declares exactly one model role, analyst, marked required: false, with the description "Optional narrative synthesis. Findings are produced deterministically without it." Its detection logic is a rubric; the model only narrates. It runs in production and its findings do not depend on a model being bound.

This is not a workaround. A rubric is auditable, reproducible across runs, and cannot hallucinate a vulnerability — all of which matter more in a security report than fluency does.

Remember that defineHarness still requires at least one declared role even if you never call ctx.model(). Declare it required: false.