---
title: Large JSON on AgentCore: Store the Payload in S3, Page the Model (2026)
description: On 17 November 2026 AWS reported up to 71 percent fewer tokens and 80 percent lower latency on a 23 MB JSON payload by keeping the payload in S3 and showing the model a count plus a handle. This is their benchmark. The failure mode is one tool result that no longer fits the window.
url: https://www.factualminds.com/blog/agentcore-json-context-s3-paging-2026/
datePublished: 2026-11-18T00:00:00.000Z
dateModified: 2026-11-18T00:00:00.000Z
author: palaniappan-p
category: AI Agents
tags: ai-agents, bedrock-agentcore, amazon-bedrock, amazon-s3
---

# Large JSON on AgentCore: Store the Payload in S3, Page the Model (2026)

> On 17 November 2026 AWS reported up to 71 percent fewer tokens and 80 percent lower latency on a 23 MB JSON payload by keeping the payload in S3 and showing the model a count plus a handle. This is their benchmark. The failure mode is one tool result that no longer fits the window.

On **17 November 2026**, AWS published a pattern for [tool results that do not fit the context window](https://aws.amazon.com/blogs/machine-learning/reduce-token-costs-and-latency-with-efficient-json-context-management-in-amazon-bedrock-agentcore/). On a **23 MB** payload they call Flightscape, they report up to **71 percent fewer tokens** and **80 percent lower latency**. Average latency before the change was above **15 seconds**. One tool response in that system exceeded **80,000 tokens**. Those figures are theirs. We did not run the workload, and this is not an agent case study.

The agent in the pattern reads tool JSON, writes the heavy body to S3, and puts a reference back in AgentCore Memory. The model sees a count and a handle. A sketch of that shape, with no benchmark, is [`persisted-state.py`](https://www.factualminds.com/examples/architecture-blog-2026/agentcore-json-context/persisted-state.py).

## Key facts (verified 2026-11-18)

- **Date.** AWS's post is 17 November 2026. The companion code they link is their repository. This page does not vendor it.
- **What moved.** Heavy JSON goes to S3. Tool lists are paged. Repeated dictionary keys are compacted. The model receives a count and a handle, not the body.
- **Their result.** Up to 71 percent fewer tokens and 80 percent lower latency on a 23 MB payload. One tool result had passed 80,000 tokens. They describe more than 40 tools and JSON above 20 MB as the shape that breaks a turn.
- **What we are not claiming.** No FactualMinds token count, latency, or customer name. The Python file only shows the reference shape.

## What broke

> **What broke:** In AWS's write-up, a single tool response crossed 80,000 tokens and the turn's average latency sat above 15 seconds. Detection is a trace where one tool result dominates input tokens. Recovery is to store that body in S3 and return a count plus a key. Lesson: a bigger context window bills the same payload again on the next turn.

The same failure shows up without their numbers. If `getOrders` returns every line item for a year, the next model call pays to reread it. Cap the tool. Page what remains.

## The turn

At the start of the turn, Runtime loads references from Memory. Keys such as `orders_ref` point at S3. They are not the orders.

When a tool returns a body that does not fit, Runtime writes the body to S3 and replaces it with the count and the key. The model can ask for one page. It does not receive the file.

At the end of the turn, Runtime writes the references back to Memory. The next turn hydrates those keys. If you write the JSON into Memory, the next turn loads it again and the S3 copy was wasted.

AWS's post describes three moves together: S3 for the body, paging for the tool list, and a compact dictionary so repeated keys are not resent. Doing only the dictionary pass leaves the 23 MB body in the prompt.

## What this post doesn't cover

- Their GitHub companion as a dependency. Read it from the AWS post. Do not treat this page's sketch as that code.
- A measured token price. S3 storage and GET requests are cheap next to input tokens, and they are still a bill. Price the object class and the expiry on the [AgentCore pricing post](https://www.factualminds.com/blog/amazon-bedrock-agentcore-pricing-12-components/).
- A reason to skip a rule. If the step is a format check, do it in code. The [token-budget post](https://www.factualminds.com/blog/aws-bedrock-cost-optimization-token-budgets-model-selection/) has the three questions. This page is only the payload shape after you have decided a model call is required.
- Memory retention. A 10-year short-term TTL does not make a JSON dump cheaper. Set retention on the [memory map](https://www.factualminds.com/blog/ai-agent-memory-types-aws-agentcore-2026/).

## Sources

- AWS, 17 November 2026, [Reduce token costs and latency with efficient JSON context management in Amazon Bedrock AgentCore](https://aws.amazon.com/blogs/machine-learning/reduce-token-costs-and-latency-with-efficient-json-context-management-in-amazon-bedrock-agentcore/).
- AgentCore Runtime, for where the turn runs: [Runtime v2](https://www.factualminds.com/blog/amazon-bedrock-agentcore-runtime-v2-ga-2026/).

## How the JSON paging agent stays bounded

Read a tool result, store an oversized body in S3, and return a count plus a key to the model.

- **Maturity:** Level 2 — reference architecture
- **Risk:** medium
- **Oversight:** monitored
- **Starts when:** A tool response is too large for the next model call.
- **Tools:** A write goes through a router, a permission check, a policy check, and a budget check. The model does not commit it. An approval token lives in tool context, not in the user message.
- **Stops for a person:** The agent writes the object and the reference. It does not change the system of record the tool read.
- **Checked by:** The model sees the count and the key. The body stays in S3. A missing key is a failed page, not a guess.
- **Untrusted data:** Tool JSON and any text inside it that tells the agent to ignore the page size.
- **If it fails:** If the JSON paging agent stops, name the reason: completed, budget_exceeded, timed_out, cancelled, guardrail_blocked, approval_required, tool_failure, verification_failed, or partial_completion. Retry a timeout at most twice. Back off on rate limits. After repeated verification failure, escalate. Stop when the budget is exhausted or a permission is denied. No unbounded loop. The agent writes the object and the reference. It does not change the system of record the tool read.
- **Lifecycle:** Explore, then act, then confirm, then verify. Explore gathers the context for the JSON paging agent. Act stays reversible. Confirm stops before an irreversible step. Verify is a separate check of the outcome.

Reference architecture, not a published production deployment. The shared contract is the [AWS store-agent architecture](https://www.factualminds.com/blog/aws-ai-agents-for-ecommerce-factualminds-2026/#operating-contract).

## FAQ

### Did FactualMinds measure the 71 percent token cut?
No. The 71 percent token reduction and the 80 percent latency reduction are from AWS's 17 November 2026 Machine Learning post, on a 23 MB payload they call Flightscape. We did not run that workload. Use the numbers as their published result, then measure your own tool payloads.

### When should we NOT page tool JSON to S3?
Skip it when the tool result already fits the window, when the next step needs every field in one prompt, or when you have no lifecycle for the object. An S3 copy with no expiry is a second store of customer data. A handle the agent cannot read back is worse than the original payload.

### When is a smaller model the wrong fix?
When one tool response is tens of thousands of tokens. A cheaper model still has to read the payload. Put the payload in S3 first, then decide whether the remaining judgment step needs a large model. Rules for format checks stay in code. See the token-budget post.

### What could go wrong if Memory stores the JSON?
Short-term Memory is the session log, and you pay to keep it. A 23 MB tool result does not belong there. Store a key and a count. The object lives in S3, with a bucket policy and an expiry. Memory that holds the payload will rehydrate it on the next turn and undo the paging.

---

*Source: https://www.factualminds.com/blog/agentcore-json-context-s3-paging-2026/*
