# Connect AI agent self-service: eval harness (illustrative)

- `eval-set.csv` - 30 sample questions for a generic online store. `expected` is one of `answer`, `refuse`, `escalate`, `requires-auth`. `must_include` / `must_not_include` are pipe-separated substrings (any-of / none-of). Replace the keywords with your own approved policy wording.
- `run-eval.mjs` - `runEval(cases, ask)` calls your injected `ask(question) => Promise<string>` and scores each reply with substring checks. It tells you whether the behaviour class matched. It does not grade answer quality and publishes no accuracy figure. A thrown error scores as an empty reply.
- `guardrail.illustrative.json` - shape follows the documented `aws qconnect create-ai-guardrail` input (topic, content, word, sensitive-information, contextual-grounding policies). Values are placeholders, not recommendations. Pass with `--assistant-id <id> --cli-input-json file://guardrail.illustrative.json`. Not run against an account.
- `run-eval.test.mjs` - `node --test` check with a canned `ask()`.

Run: `node --test` in this folder. Against your bot: write a module exporting `default async ask(q)` that calls your chat endpoint, then `node run-eval.mjs ./ask.mjs`.