AWS Gives Bedrock Coding-Agent Fleets Three Levers: Route by Task, Bill by Person, Start From a Snapshot

Three AWS engineering posts published Nov. 13 turn Amazon Bedrock into a control plane for coding-agent fleets. They change the unit you manage: score a routing policy by the task it finishes, name the principal who paid, and start a session from a snapshot.

1. Route by the task, not the token

AWS’s routing walkthrough deploys NVIDIA NeMo Switchyard as a LiteLLM plugin in front of Bedrock and scores it with a benchmark runner. The framing sentence is the useful one: judge routing by cost per solved task, the total cost of your runs divided by the number of runs whose tests pass.

Strategy	Needs	Behaviour
Stage	Two candidates	Reads recent tool results
Random	Weighted pool	Traffic split for A/B tests
Task, Composite	Judge model calls	Not supported here

The sample lists the capable model first and the efficient model second, because Stage reads roles from declaration order. With picker = "efficient_first", a request with no tool results carries no signal, so it goes to the efficient model, and the capable model is reached only after failures appear in the history. AWS notes an escalated turn replays the whole failed attempt, which can cost more than starting on the capable model.

# profiles/stage/switchyard.toml
algorithm = "stage"
picker = "efficient_first"
confidence_threshold = 0.5
recent_window = 3

Log the x-litellm-model-name header, not the request: it names the model that answered while the body keeps "model": "switchyard". And the gateway forwards only /v1/chat/completions, /v1/models and /health/liveliness, so clients on the OpenAI Responses or Anthropic Messages path get a 404 until you add the route.

The benchmark runner is the piece most teams can steal: bench/run.py prints RUNS and SOLVED, COST/TASK across every run including failures, and COST/SOLVED, estimated spend divided by runs whose tests passed. Run it against the routed group and two single-model baselines. AWS’s limit: the runner executes model-written Python on your machine without an OS sandbox.

2. Attribute every call to a person

AWS’s cost attribution post covers the layer underneath routing. Bedrock records the IAM identity of every inference caller with no change to your API calls and publishes that ARN, session name included, into line_item_iam_principal in CUR 2.0. Because it sits at the endpoint layer, Claude Code, Codex, ChatGPT Work and OpenCode inherit per-user attribution once they point at Bedrock.

AWS follows the federated pattern: connect your IdP to IAM Identity Center, map attributes to session tags such as Team and CostCenter, and give each developer their own session. Two traps: a shared role with a fixed session name collapses billing into one principal, and AWS_BEARER_TOKEN_BEDROCK attributes calls to the IAM user behind that key rather than the SSO session.

Then wire the export: a CUR 2.0 export in Parquet, hourly or daily, with “Include caller identity (IAM principal) allocation data” selected. Exports created before that option existed must be recreated. Tags appear only after a principal has made a call, AWS allows 24 hours for them to show up and 48 hours to reach Cost Explorer, and activation is not retroactive.

SELECT regexp_extract(line_item_iam_principal, 'assumed-role/[^/]+/(.+)$', 1) AS user_name,
       SUM(line_item_unblended_cost) AS total_cost_usd
FROM cur_table
WHERE line_item_product_code IN ('AmazonBedrock','AmazonBedrockService')
  AND line_item_iam_principal IS NOT NULL
GROUP BY 1
ORDER BY total_cost_usd DESC;

Enforcement is where per-user cost plans quietly fail, and AWS documents the latency you buy:

Control	Time to stop spend	Operate cost
Budgets and budget actions	Up to a day of overshoot	Lowest
DynamoDB counters	Next credential refresh	Medium
CloudWatch alarms plus Lambda	1 to 5 minutes	Medium
Gateway proxy	Every request	Highest, added latency

Budgets are a backstop, not a brake: they update about three times a day.

3. Start sessions from a snapshot

AWS’s AgentCore runtime post covers the third cost in a fleet: startup latency. Runtime V2 prepares the environment once, snapshots the microVM and restores it per session instead of pulling a container image every time, at the price of a few minutes per create or update.

That inverts where work belongs. Initialization now runs once, so anything you defer to the handler is paid again by every new session. Keep imports, clients, static data and helper processes in initialization; keep random values, timestamps and credentials out of it, because those freeze into the snapshot and every session inherits them.

AgentCore publishes Latency and OverheadLatency as CloudWatch metrics in the AWS/Bedrock-AgentCore namespace. If two seconds is too slow, start the session on user engagement: send a background warm-up request when the user opens the chat and keep that session ID for the conversation.

What to do first

Pick the lever your billing data cannot answer. If you know spend per token but not what a finished task costs, run the sample benchmark on your own tasks before changing a model. If you know the total but not the owner, turn on IAM principal attribution in CUR 2.0 and let the rows settle for two days. If the complaint is latency, move initialization into the snapshot and compare OverheadLatency before and after.

Two honest limits: the Switchyard integration is experimental and AWS says to deploy it in a non-production account, and attribution data arrives late and is not retroactive, so a per-user budget only bites after days of steady traffic.

The idea is portable even if the vendor is not: the unit that matters for a coding agent is the task that finishes. OpenRouter’s auto code router argues the same from the hosted side, and cost per completed task beats price per token is the shortest version of the principle.