# MCP vs Agent Skills — Strands on Amazon Bedrock benchmark

Companion to the blog post [MCP vs Agent Skills](/blog/mcp-vs-agent-skills-strands-bedrock-2026/).

One refund-triage job, four agent setups, same model:

| Setup | Order data (MCP) | Refund policy (Skill) |
| --- | --- | --- |
| `baseline` | no | no |
| `mcp` | yes | no |
| `skill` | no | yes |
| `mcp_skill` | yes | yes |

## Files

- `mcp_server/order_tools.py` — MCP server (stdio) with `get_order`, `list_orders_for_customer`, and a no-op `issue_refund` that logs calls to `results/write_calls.log`.
- `mcp_server/orders.json` — **synthetic** fixture orders. Not a real store or customer.
- `skills/refund-triage/SKILL.md` — the Agent Skill: eligibility rules, restocking fee, approval threshold, "never execute a refund".
- `agent.py` — builds a Strands `Agent` on Bedrock for one setup. `uv run agent.py mcp_skill "Order FM-1042: ..."`
- `bench.py` — runs every setup on 3 tasks N times, grades the JSON answer, writes `results/`.
- `results/` — the published run: `summary.md`, `results.json`, `run.log`.

## Run it

Tested with Python 3.14, uv 0.13, `strands-agents==1.62.0`, `mcp==2.1.1`, `us.anthropic.claude-haiku-4-5-20251001-v1:0` in `us-east-1`.
Needs AWS credentials with `bedrock:InvokeModelWithResponseStream` on that inference profile.

```bash
uv sync
AWS_PROFILE=your-profile uv run agent.py mcp_skill      # one answer
AWS_PROFILE=your-profile uv run bench.py --n 10         # full run, ~124 agent invocations
cat results/summary.md
```

Override the model with `MODEL_ID=...` and the region with `AWS_REGION=...`.

## Grading

A run passes only if the JSON block has the right `decision`, `refund_amount_usd` (±$0.01), and `needs_human_approval`, **and** the agent did not call `issue_refund`. Expected answers come from applying `SKILL.md` to `orders.json` with today fixed at 2026-10-20.

## Gotcha

`mcp` 2.x renamed `FastMCP` to `MCPServer` (`from mcp.server.mcpserver import MCPServer`). v1 tutorials fail with `ModuleNotFoundError: No module named 'mcp.server.fastmcp'`.