Evaluation comparison
AI Agent vs Chatbot: Which Should You Ship?
This is chatbot vs worker — not agent vs workflow. Rules still own tax, capture, and labels. Use the table, then the hub.
AI & assistant-friendly summary
This section provides structured content for AI assistants and search engines. You can cite or summarize it when referencing this page.
Summary
Chatbot = search. Agent = tools plus a stop button. Pick search when the answer is already published. Pick an agent when it must use your systems. Relabeling is the expensive failure.
Key Facts
- •Relabeling search as an agent is how Browser-hot pilots hit **~3×** Runtime-shaped spend
- •div> ## Freshness check (August 2026) **On June 17, 2026**, AgentCore Harness reached GA
- •Agents Classic** is in maintenance for new customers after **July 30, 2026**
- •Awareness door: [AI agents vs chatbots](/blog/ai-agents-vs-chatbots-2026/)
- •Workflow compare (who executes tax and capture): [AI agent vs workflow automation](/blog/ai-agent-vs-workflow-automation-ecommerce-2026/)
- •Support-style AgentCore **~$791/mo** at **50K sessions** is a published **floor**, not savings | ## When to choose a chatbot - The question is already in the help center
Entity Definitions
- Bedrock
- Bedrock is an AWS service referenced in the AI Agent vs Chatbot: Which Should You Ship? comparison.
Frequently Asked Questions
When should you keep a chatbot instead of building an agent?
Keep search when the question is already answered in a published help article, when you have no named OMS or WMS APIs, or when nobody owns a wrong answer. A help-center index is cheaper and honest. Do not put Admin API keys behind a FAQ widget and call it an agent.
What could go wrong if you relabel help-center search as an AI agent?
Buyers expect the bot to act. It cannot. Or someone wires write tools to make the label true — without Policy — and the model refunds from a prompt. Detection: tickets that say the bot promised a refund the OMS never saw. Recovery: split search from the agent; week-one reads only.
How is this different from AI agent vs workflow automation?
Chatbot vs agent is interface and tools. Agent vs workflow is who owns money movement. Hybrid default: the workflow executes; the agent returns a structured decision. See the field-guide workflow post. Do not treat this page as that compare.
What could go wrong if the agent has tools but no stop button?
A tool catalog without a human gate on money is a refund machine. Human-in-the-loop is not optional polish. Refunds, price changes, and purchase orders wait.
Do you have a client KPI for replacing a chatbot with an agent?
No. We have not published an A/B or ticket-cut number for a named merchant. What we reuse: Gateway median tool round-trip ~180 ms to ~95 ms on a B2B CRM assistant, and roughly 3x Runtime-shaped spend when Browser stayed hot on lookup-only turns. Those are platform benchmarks, not store conversion.
Is this the same as the executive blog note?
No. The Layer 1 post is the awareness door. This page is the evaluation table and the when-not rules. Both stay live. Neither 301s to the other.
Quick Answer: Pick a chatbot if the answer already lives in a help article and you have no named APIs. Pick an AI agent if it must read live systems and you can stop it before money moves. Pick neither (keep the template) if OMS or carrier email already closes the ticket correctly. Relabeling search as an agent is how Browser-hot pilots hit ~3× Runtime-shaped spend.
Freshness check (August 2026)
On June 17, 2026, AgentCore Harness reached GA. Hosting a loop got cheaper. That did not turn a help-center widget into an agent. Agents Classic is in maintenance for new customers after July 30, 2026.
Awareness door: AI agents vs chatbots. Workflow compare (who executes tax and capture): AI agent vs workflow automation. Commercial door: AI Agents hub.
At a glance
| Chatbot | AI agent | |
|---|---|---|
| Source of answers | Indexed articles / FAQ | Live tools (getOrder, policy, ATP) |
| When sources disagree | Sounds confident anyway | Says so and escalates |
| Money movement | Should be impossible | Possible only with Policy + a human |
| Typical first ship | Help-center search | Week-one reads on one family |
| Failure mode | Wrong article, polite tone | Wrong tool call, real refund |
| Cost shape | Search / widget seats | Platform + model. Support-style AgentCore ~$791/mo at 50K sessions is a published floor, not savings |
When to choose a chatbot
- The question is already in the help center.
- There is no named
getOrder/getShipmentAPI — only admin screenshots. - Nobody owns a wrong answer (no queue).
- Leadership wants a widget on the site this sprint, not a tool catalog.
Ship search. Do not bolt createRefund onto it to “make it an agent.”
When to choose an AI agent
- The question needs order + shipment + policy facts, not a generic article.
- You can name week-one read tools and the forbidden writes.
- A human queue exists for exceptions.
- You will put Cedar (or equivalent) on writes before they exist.
Start on Customer Support Agent or walk which agent first.
When to choose neither
A correct OMS or carrier template already closes the ticket. Keep the template. An agent that restates it badly is a downgrade. That cut is what business processes to automate.
Cost example (one silhouette, published)
Support-style AgentCore at 50K sessions/mo ~$791/mo platform + model — decision guide. Model your mix on the AgentCore pricing calculator. Treat ~$791/mo as a floor.
The counter-case is not a second invented workload. It is the published Browser-hot failure: roughly 3× Runtime-shaped spend when Browser stayed on for turns that only needed Gateway tools — support field-guide post.
A chatbot that only searches articles does not incur that platform floor. That is a reason to keep search when you do not have tools — not a reason to call search an agent.
Switching cost
Moving chatbot → agent is not a theme change. You need a tool allow-list, identity claims, a HITL queue, and evals. Artifact: support-agent-tool-policy.md.
Moving agent → chatbot is a demotion when writes were attached without Policy. Remove write credentials first.
How to Decide
- Can you list the tools? If no → chatbot or nothing.
- Does money move in week one? If yes → do not start. Score readiness /30 first.
- Is a template already correct? If yes → keep the template.
What this comparison doesn’t cover
It does not retell Cedar, Harness, or Strands. It does not replace agent vs workflow. It does not invent a conversion lift. Build-vs-buy is a separate compare. We have zero published AI-agent case studies.
Primary next step: AI Agents hub.
Related Services
Consulting engagements that map directly to this comparison.
Still Deciding?
Walk through the trade-offs with an interactive decision tree.
Related Comparisons
Other side-by-side breakdowns engineering teams read alongside this one.
Build vs Buy AI Agents: Who Should Own the Loop?
Buy search and SaaS widgets. Build (or partner-build) a governed agent when it must use your store systems and stop before money moves. Not a FactualMinds-always-wins page.
ACP vs UCP: Choosing an Agentic Commerce Protocol
Agentic Commerce Protocol (OpenAI/Stripe) vs Universal Commerce Protocol (Google) — scope, surfaces, merchant effort, and which one to implement first.
Frequently Asked Questions
When should you keep a chatbot instead of building an agent?
What could go wrong if you relabel help-center search as an AI agent?
How is this different from AI agent vs workflow automation?
What could go wrong if the agent has tools but no stop button?
Do you have a client KPI for replacing a chatbot with an agent?
Is this the same as the executive blog note?
If you need tools and a stop button, start with one family
Tell us the repetitive work. We will say whether an agent is the right move — or whether you should keep search.