Large JSON on AgentCore: Store the Payload in S3, Page the Model (2026)
Quick summary: On 17 November 2026 AWS reported up to 71 percent fewer tokens and 80 percent lower latency on a 23 MB JSON payload by keeping the payload in S3 and showing the model a count plus a handle. This is their benchmark. The failure mode is one tool result that no longer fits the window.
Key Takeaways
- On 17 November 2026 AWS reported up to 71 percent fewer tokens and 80 percent lower latency on a 23 MB JSON payload by keeping the payload in S3 and showing the model a count plus a handle
- On 17 November 2026, AWS published a pattern for tool results that do not fit the context window
- On a 23 MB payload they call Flightscape, they report up to 71 percent fewer tokens and 80 percent lower latency
- Average latency before the change was above 15 seconds
- One tool response in that system exceeded 80,000 tokens

Table of Contents
On 17 November 2026, AWS published a pattern for tool results that do not fit the context window. On a 23 MB payload they call Flightscape, they report up to 71 percent fewer tokens and 80 percent lower latency. Average latency before the change was above 15 seconds. One tool response in that system exceeded 80,000 tokens. Those figures are theirs. We did not run the workload, and this is not an agent case study.
The agent in the pattern reads tool JSON, writes the heavy body to S3, and puts a reference back in AgentCore Memory. The model sees a count and a handle. A sketch of that shape, with no benchmark, is persisted-state.py.
What broke
The same failure shows up without their numbers. If getOrders returns every line item for a year, the next model call pays to reread it. Cap the tool. Page what remains.
The turn
At the start of the turn, Runtime loads references from Memory. Keys such as orders_ref point at S3. They are not the orders.
When a tool returns a body that does not fit, Runtime writes the body to S3 and replaces it with the count and the key. The model can ask for one page. It does not receive the file.
At the end of the turn, Runtime writes the references back to Memory. The next turn hydrates those keys. If you write the JSON into Memory, the next turn loads it again and the S3 copy was wasted.
AWS’s post describes three moves together: S3 for the body, paging for the tool list, and a compact dictionary so repeated keys are not resent. Doing only the dictionary pass leaves the 23 MB body in the prompt.
What this post doesn’t cover
- Their GitHub companion as a dependency. Read it from the AWS post. Do not treat this page’s sketch as that code.
- A measured token price. S3 storage and GET requests are cheap next to input tokens, and they are still a bill. Price the object class and the expiry on the AgentCore pricing post.
- A reason to skip a rule. If the step is a format check, do it in code. The token-budget post has the three questions. This page is only the payload shape after you have decided a model call is required.
- Memory retention. A 10-year short-term TTL does not make a JSON dump cheaper. Set retention on the memory map.
Sources
- AWS, 17 November 2026, Reduce token costs and latency with efficient JSON context management in Amazon Bedrock AgentCore.
- AgentCore Runtime, for where the turn runs: Runtime v2.
Frequently asked questions
Did FactualMinds measure the 71 percent token cut?
When should we NOT page tool JSON to S3?
When is a smaller model the wrong fix?
What could go wrong if Memory stores the JSON?
Reference bounds
How the JSON paging agent stays bounded
Level 2 — reference architecture. Risk medium. Oversight: monitored.
Read a tool result, store an oversized body in S3, and return a count plus a key to the model.
Explore, then act, then confirm, then verify. Explore gathers the context for the JSON paging agent. Act stays reversible. Confirm stops before an irreversible step. Verify is a separate check of the outcome.
- Starts when
- A tool response is too large for the next model call.
- Tools
- A write goes through a router, a permission check, a policy check, and a budget check. The model does not commit it. An approval token lives in tool context, not in the user message.
- Stops for a person
- The agent writes the object and the reference. It does not change the system of record the tool read.
- Checked by
- The model sees the count and the key. The body stays in S3. A missing key is a failed page, not a guess.
- Untrusted data
- Tool JSON and any text inside it that tells the agent to ignore the page size.
- If it fails
- If the JSON paging agent stops, name the reason: completed, budget_exceeded, timed_out, cancelled, guardrail_blocked, approval_required, tool_failure, verification_failed, or partial_completion. Retry a timeout at most twice. Back off on rate limits. After repeated verification failure, escalate. Stop when the budget is exhausted or a permission is denied. No unbounded loop. The agent writes the object and the reference. It does not change the system of record the tool read.
This is the reference architecture for the page, not a published production deployment. The shared contract is the AWS store-agent architecture. Permissions and data boundaries are in securing store agents.

AWS Cloud Architect & AI Expert
AWS-certified cloud architect and AI expert with deep expertise in cloud migrations, cost optimization, and generative AI on AWS.




