Practice: Integrating hosted LLMs: AWS Bedrock, the Claude API and prompt engineering

Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.

Main tool: AWS Bedrock · AWS Bedrock is the right tool to practise with because it puts models from several providers behind one identity, quota and billing system and one Converse message format, which is exactly the surface a platform team owns. · Alternatives: Claude API direct (newest model features first, but separate keys and billing outside AWS identity), OpenAI API and Gemini API (single-provider endpoints with their own message formats and quotas), and LiteLLM (a self-hosted gateway that puts one OpenAI-style interface, budgets and routing in front of all of them, including Bedrock).

Every step links into the tool's own documentation; links were checked on 2026-10-11 · version 1

Setup (one time)

1. Create an AWS identity with Bedrock permissions and local credentials, noting that Bedrock is a paid cloud service billed per token and that the closest local substitute is replaying recorded responses through a stubbed SDK client.
Every later step calls Bedrock under this identity, so access is the first thing a platform engineer has to own. Keeping prompts tiny holds the cost to a few cents, and the stub lets you rerun the path offline.
Explained in: Overview - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: Your terminal can authenticate to AWS and the identity is allowed to invoke a Bedrock model.
2. Install the Python AWS SDK (boto3) in a fresh virtual environment and set a default region.
The lesson drives Bedrock from application code, and a clean environment is what you will pin later for CI.
Explained in: Overview - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: A Python session can create a Bedrock runtime client in your chosen region without an error.
3. Pick one inexpensive text model that is available in your region and supports the Converse API, tool use and streaming.
Model availability and feature support differ by region and by model, and a wrong pick shows up later as confusing validation errors.
Explained in: Model availability & compatibility - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: You have written down one model identifier and the region you will use for the whole path.

Steps

1. Run the quickstart inference request against your chosen model (paid AWS call; save the response to replay through a stubbed client when offline).
A first successful call proves that credentials, region and model access all line up before you add any complexity.
Explained in: Overview - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: The model's text reply is printed in your terminal.
2. Compare the APIs Bedrock supports and write a short note on which one your application contract will use and why.
Bedrock exposes the same foundation models through several APIs and endpoints, and the platform team must choose one contract for application code instead of letting every team pick its own.
Explained in: APIs supported by Amazon Bedrock - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: Your note names the chosen API, the endpoint it runs on, and one API you ruled out with the reason.
3. Write a multi-turn conversation with the Converse API, appending each assistant reply to the messages list before the next user turn.
The messages structure is the core object model of Converse, and the service is stateless, so your code owns the conversation history.
Explained in: Inference using Converse API - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: The model's third reply correctly refers to something said in the first turn.
4. Configure a system prompt and explicit inference parameters, then run the same user message under two system prompts and compare the outputs.
Treating the system prompt and inference settings as explicit, separate inputs is the first step toward prompts that can be tested rather than strings buried in code.
Explained in: Inference using Converse API - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: You have two saved outputs that differ only because the system prompt changed.
5. Create a prompt with variables in Prompt management, then run inference on that prompt resource through Converse by filling the variables at call time.
A prompt stored as a versioned resource can be reviewed, tested and rolled back independently of application deploys.
Explained in: Converse - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: A Converse call that references the prompt resource and passes variable values returns a reply, with no prompt text in your code.
6. Configure one tool in the tool configuration and write the loop that runs the tool locally and sends the result back to the model.
Tool use is a multi-step exchange your code must drive, and Converse gives it one schema across providers.
Explained in: Inference using Converse API - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: The model requests your tool, your code returns a result, and the final reply uses that result.
7. Run the same request with ConverseStream and print each text delta as it arrives.
Streaming changes perceived latency and the error handling your gateway needs, because the response arrives as a sequence of events instead of one body.
Explained in: ConverseStream - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: Text appears incrementally in your terminal and you see a final stop event with a stop reason.
8. Query the token usage and latency metrics from each response and log them as one structured line per call.
Cost and latency are only manageable once every call records its tokens and timing, and Converse returns both in the response.
Explained in: ConverseMetrics - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: Your log shows input tokens, output tokens and latency in milliseconds for at least five calls.
9. Write retry handling with backoff for the throttling and service errors that Converse can return, and test it by sending a burst of concurrent requests (or by stubbing a throttling error locally).
Quota limits surface as specific errors, and well-behaved clients are what keep one noisy caller from taking down a shared account.
Explained in: Converse - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: A throttled request is retried and succeeds, and your log shows the retry count.
10. Write a thin gateway function that every caller goes through, which attaches request metadata naming the calling team and use case to each Converse request.
A gateway is where a platform team centralises attribution, limits and model choice, and request metadata is how Bedrock carries that attribution into its logs.
Explained in: Inference using Converse API - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: Two callers with different team labels go through the same function and each request carries its own metadata.
11. Create a guardrail, enable it in the gateway's Converse calls, and write an automated test with pinned SDK versions that asserts a blocked input is intercepted.
A policy enforced in the gateway and checked by a repeatable test stays in force as prompts and models change. Pinning the environment makes the test give the same result on a laptop and in CI.
Explained in: Include a guardrail with the Converse API - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: The test passes from a clean environment and the response for the blocked input shows the guardrail intervened.
12. Configure the gateway to call the model through an inference profile instead of a single-region model identifier, and rerun the full test suite.
Inference profiles spread traffic across regions for higher throughput and resilience, which is how a production gateway avoids being bound to one region's quota.
Explained in: Converse - Amazon Bedrock (docs.aws.amazon.com), checked 2026-10-11
Done when: All tests pass with the inference profile identifier and the usage log still records tokens, latency and team metadata for every call.

Back to Integrating hosted LLMs: AWS Bedrock, the Claude API and prompt engineering