How Rescale built a test-failure triage agent with Strands Agents and AI-DLC
Rescale, a digital engineering platform company, built an agent that classifies and routes failures from its nightly end-to-end tests, using the AI-DLC methodology and the Strands SDK during a five-day AWS workshop. The agent reached staging in five days and has since run in production, replacing manual first-pass triage.
The problem
Engineers at Rescale spent significant time classifying failures from hundreds of nightly end-to-end tests, then collecting evidence from several logging systems and repositories before debugging could begin. Because these tests gate weekly production releases, unresolved failures could pull several teams into hours of coordination before a deployment could proceed.
How they did it
- Run a five-day in-person workshop in which AWS architects guide Rescale engineers building hands-on, using the AI-DLC methodology.
- In mob sessions, answer the 20+ clarifying questions the AI generates, producing a scoped requirements document that separates Phase 1 from Phase 2 and four units of work.
- Build the agent with the Strands SDK and containerize it on Amazon EKS. It runs as a step in the existing Argo Workflows CI/CD pipeline when the nightly Playwright suite has failures.
- The agent calls Claude Opus 4.8 on Amazon Bedrock to classify each failure as infrastructure issue, code bug, flaky test or environment problem, and pulls historical failure patterns from a knowledge base.
- Query pod logs through MCP backed by Amazon CloudWatch, and generate a structured remediation report with file and function references for engineers to review.
- Add a PyTest-based evaluation framework with live and stubbed modes to set a classification-agreement baseline.
Results
- As reported by AWS, classification agreement with human triage rose from 44% at initial deployment to 68% today.
- As reported by AWS, the team measured a 20% decrease in the time that test-failure analysis blocks production deployment.
- As reported by AWS, the agent went from problem definition to staging in five days and to production within weeks, and it was demoed correctly classifying an infrastructure failure and a code bug.
As reported by the source (AWS Machine Learning Blog customer story); AgentGid did not measure these figures.
Best for a platform or QA team that already runs Kubernetes, Argo Workflows and nightly Playwright tests, and has engineers comfortable with Bedrock and MCP. Strands is free (OSS), but the model and EKS costs are separate. Note the 68% agreement with human triage and the 20% time saving are AWS-reported, so engineers still review each report. LangGraph (Free, OSS) is a similar option if you're not on AWS.
The agent used here
Similar use cases
How Cornerstone OnDemand cut database diagnosis time with Strands Agents on Amazon Bedrock
Cornerstone OnDemand's three-person Enterprise DataOps team built Orion AI, a multi-agent system on Amazon Bedrock and Strands Agents, in six months. It speeds up database inciden…
As reported by AWS, database diagnosis time dropped from 45 minutes to 10 minutes, a 78% reduction.
Stream low-latency agent speech with Kokoro TTS on Amazon SageMaker AI
An AWS blog post shows how to send each sentence from an AI voice agent to a hosted Kokoro-82M text-to-speech model as soon as it is generated. Audio playback can then begin befor…
As reported by AWS, sending only the first sentence cut mean time to first audio byte from 329.3 ms to 156.7 ms, a 52.4 percent relative improvement …
Route agent requests between local SLMs and Amazon Bedrock with Strands Agents
An AWS blog walkthrough shows how to build an Adaptive Agentic Intelligence Framework with Strands Agents. A small local model handles routine requests, and complex analysis goes …
Routine requests such as device status checks and image analysis run locally with low latency and no cloud dependency.