Visual overview
Bedrock exposes model/agent behavior, CloudWatch surfaces runtime symptoms, and S3 preserves controlled golden datasets and reproducible evaluation artifacts.
Technical reference
A troubleshooting playbook maps symptoms to layers and preserves a reproducible baseline. Avoid random prompt edits when the failure is retrieval, capacity, schema, or authorization.
Track token estimates and explicit inclusion priorities so truncation behavior is predictable.
Separate validation/auth failures from throttling/transient failures and apply retries only where appropriate.
Changing embedding space requires compatible vectors/schema and retrieval regression validation.
Inspect candidates directly before blaming the generator for missing or stale evidence.
Parse and validate generated structures before they reach a database, API, or tool executor.
Use traces and simulated tool failures to isolate planning, execution, retry, and termination defects.
Service limits and capabilities can change. Values shown here reflect the current AWS documentation; use the linked official sources below as the source of truth.
Classify the failure before changing the model
Start with the layer that failed: request validation/authentication, capacity/throttling, prompt/context assembly, retrieval, model generation, structured parsing, guardrail, agent planning, tool execution, or downstream business action. Reproduce the smallest request that preserves the failure and capture safe identifiers for model/profile, prompt version, release, knowledge index, tool version, and trace. Changing multiple layers at once makes the incident harder to explain and easier to reintroduce.
Use a known-good baseline. If the same prompt succeeds against the previous model or prompt version, compare the release manifest and output. If retrieval candidates changed, debug ingestion/search before generation. If raw model output is valid but application parsing fails, fix the schema/adapter instead of prompting the model to compensate for a code defect.
Context-window and truncation defects have recognizable symptoms
A request that exceeds a model's supported context or API constraints can fail validation, while application-side truncation may silently remove system instructions, retrieved evidence, or recent conversation turns. Track token estimates before invocation and define an explicit context-budget policy: reserve output capacity, prioritize required system/policy content, retrieve within a budget, and summarize or drop lower-priority history deliberately.
Do not truncate blindly from one end. In a RAG application, dropping the most relevant evidence to preserve old chat history can reduce factuality. In an agent, dropping tool definitions can change behavior completely. Log token counts and which context sections were included, using hashes/IDs rather than sensitive text when possible, so an operator can distinguish 'model ignored evidence' from 'evidence was never sent'.
Embedding and retrieval regressions need search-level evidence
When RAG degrades, inspect whether source ingestion completed, chunk boundaries changed, the embedding model or dimension changed, metadata filters exclude expected documents, the vector/text index is healthy, and expected chunks appear in top candidates. Compare similarity/relevance ranks on a fixed retrieval dataset before and after the change. A newly embedded query against stale vectors from another embedding space can produce poor retrieval even though both API calls succeed.
Tune one retrieval variable at a time: chunking, embedding, search type, filter, candidate count, query decomposition, reranker. Search latency can also regress with corpus growth or index settings. Measure recall/relevance and latency together, then re-run answer evaluation after the retrieval fix.
Structured output and agents require deterministic assertions
If the application expects JSON or a tool schema, validate the parsed result against the schema before use. Handle missing fields, unexpected enum values, oversized arguments, and semantic constraints such as date ranges or resource ownership. A valid JSON object can still be an invalid business action. Use a repair/retry path only when it is bounded and cheaper/safer than rejecting the request.
For agents, inspect the full trace: instructions presented, selected tool, generated arguments, tool response, next planning step, and stop reason. AWS documentation for Bedrock Agents Classic illustrates trace-driven troubleshooting; for new systems, apply the same principle with current AgentCore/framework telemetry. Disable or isolate suspect tools during testing, add synthetic tool failures, and confirm timeouts/retries/stopping rules behave as designed.
Turn every fixed incident into a regression test
Once the root cause is known, capture a sanitized failing example in the appropriate suite: prompt golden set, retrieval expectation, safety adversarial case, schema fixture, agent tool simulation, or load test. Record what signal should have detected the issue sooner and add an alarm or release gate where practical.
Testing should include functional correctness, load/concurrency, failure injection, security/adversarial cases, data freshness, and cost/latency budgets. Generative behavior is probabilistic, so test thresholds should allow expected variance while keeping hard constraints—authorization, schema, prohibited actions, and data isolation—deterministic.
Key takeaways
- 01
Troubleshoot by layer and change one variable at a time against a known-good baseline.
- 02
Track token/context budgets explicitly so truncation cannot silently remove policy, evidence, history, or tool definitions.
- 03
Embedding and retrieval failures can occur even when every API is healthy; verify expected evidence and ranking directly.
- 04
Validate structured model output and tool arguments with deterministic schemas and business rules before execution.
- 05
Every production incident should become a regression case and, where possible, a stronger release gate or alarm.
Official AWS sources
Use these primary AWS resources for the source material behind this article and for deeper reference.