> ## Documentation Index
> Fetch the complete documentation index at: https://help.nops.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Bedrock Batch Inference for Latency-Tolerant Workloads

> Send Bedrock requests that do not need an immediate answer as batch inference jobs, at a lower price per token.

Amazon Bedrock prices batch inference below on-demand inference for many models. Batch jobs read prompts from S3 and return results later, so they suit work like enrichment, classification, evaluation and back-fills. nOps looks at your on-demand, non-streaming Bedrock calls per account, region and model, and estimates the saving if that work ran as batch.

| | |
| - | - |
| **Cloud** | AWS |
| **Category** | AI |
| **Service** | Amazon Bedrock |
| **Action** | Modify |
| **Effort** | Medium |
| **Restart needed** | No |
| **Can be rolled back** | Yes |
| **Analysis window** | Last 7 days |
| **Source** | nOps |

## Why it matters

Many background workloads call Bedrock the same way an interactive app does and pay the on-demand price, even though nobody is waiting for the answer.

## How nOps finds it

nOps reads the last 7 complete days of Bedrock billing for on-demand, non-streaming model calls (streaming calls are treated as interactive and left out). The saving is the token cost times the difference between the batch and on-demand price for that model and region. The batch price comes from the AWS price list, or from your own billed batch rate when you already use batch. Models with no known batch price are not shown, and neither are savings under \$25 a month. Because nOps cannot tell which calls can wait, this is the most you could save if all of them moved.

## Before you start

A model that supports batch inference in your region, an S3 bucket for input and output, and an IAM service role Bedrock can use to read and write it.

## How to implement

1. Find which callers of this model do not need an immediate answer (for example nightly enrichment or evaluation jobs).
2. Write their requests as JSONL records to an S3 bucket.
3. Create a batch inference job (Bedrock console: **Inference → Batch inference → Create job**, or `aws bedrock create-model-invocation-job`) pointing at the input and output locations.
4. Change the caller to read results from the output location when the job finishes.
5. Keep interactive and streaming traffic on on-demand inference.

## Risks and considerations

* Batch jobs complete asynchronously, so they are not suitable for anything a user is waiting on.
* Batch jobs have minimum and maximum record counts per job, so very small workloads may need to be grouped.
* The estimate covers every non-streaming call; the real saving depends on how much of that traffic can wait.
* To roll back, send the requests on demand again.

## FAQs

> [accordion] Does nOps count my streaming calls?
>
> No. Streaming calls are treated as interactive and left out of the estimate.

> [accordion] How long do batch jobs take?
>
> AWS aims to complete batch jobs within 24 hours. Most finish sooner, but plan for the full window.

## Related

* Move Latency-Tolerant Azure OpenAI Workloads to Batch
* Move Latency-Tolerant Gemini Workloads to Batch or Flex
* [AI Model Tier Right-Sizing](https://help.nops.io/inform/supported-recommendations/ai-model-tier-right-sizing)
* [Supported recommendations](https://help.nops.io/inform/supported-recommendations.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.