Stream low-latency agent speech with Kokoro TTS on Amazon SageMaker AI
An AWS blog post shows how to send each sentence from an AI voice agent to a hosted Kokoro-82M text-to-speech model as soon as it is generated. Audio playback can then begin before the full response is finished, and the authors report lower time to first audio.
The problem
Voice agents often wait for the whole response to be generated before text-to-speech starts. A reply that takes about five seconds to finish leaves the customer in silence even when the first sentence was ready quickly. Returning the audio to a browser adds a second problem, because the common Server-Sent Events pipeline is built for text rather than binary audio.
Steps
- Deploy the Kokoro-82M container to a SageMaker AI real-time endpoint on an ml.g5.xlarge GPU instance, using the CDK-based speech-infra CLI, then deploy the Strands agent to Bedrock AgentCore runtime with agent-infra.
- Make the WebSocket handler accept both text and binary frames, since SageMaker AI delivers client messages as binary.
- Keep the bidirectional session open after each synthesis_complete frame and close it only when the client sends an explicit close message.
- Use a sentence chunker that is fed text fragments as they arrive and emits complete sentences, with a final flush for any leftover text.
- Pick a transport for returning audio: base64 audio inside SSE events if you already run SSE, or raw binary audio mixed with JSON control frames on the bidirectional session.
- Compare bidi and batch modes with the agent-infra demo command against your own endpoint.
From the official docs
uv run speech-infra deploy kokoro-82m
uv run agent-infra deploy --bedrock-model-id <your-bedrock-model-id>
uv run agent-infra demo --endpoint-url <your-endpoint-url> --prompt "<your prompt>"Results
- As reported by AWS, sending only the first sentence cut mean time to first audio byte from 329.3 ms to 156.7 ms, a 52.4 percent relative improvement at the TTS transport level (five trials each).
- As reported by AWS, end-to-end time to first audio in bidi mode was 2.4 times faster than batch mode against a local container and 3.5 times faster on the deployed AgentCore runtime.
- As reported by AWS, base64 encoding inflates audio payloads by close to 33%, and SSE chunk boundaries may not align with MP3 frames, so a consumer needs to buffer and reassemble.
As reported by the source (AWS Machine Learning Blog post); AgentGid did not measure these figures.
This suits a team already on AWS with engineers comfortable with CDK, WebSockets and audio streaming. It needs a GPU endpoint (ml.g5.xlarge) plus AgentCore deployment, and the timing gains are AWS-reported, so measure against your own endpoint. Strands Agents is free (OSS), but the hosting isn't; if you'd rather not be tied to AWS, LangGraph (Free, OSS) is a more popular option.
The agent used here
Similar use cases
Route agent requests between local SLMs and Amazon Bedrock with Strands Agents
An AWS blog walkthrough shows how to build an Adaptive Agentic Intelligence Framework with Strands Agents. A small local model handles routine requests, and complex analysis goes …
Routine requests such as device status checks and image analysis run locally with low latency and no cloud dependency.
How Cornerstone OnDemand cut database diagnosis time with Strands Agents on Amazon Bedrock
Cornerstone OnDemand's three-person Enterprise DataOps team built Orion AI, a multi-agent system on Amazon Bedrock and Strands Agents, in six months. It speeds up database inciden…
As reported by AWS, database diagnosis time dropped from 45 minutes to 10 minutes, a 78% reduction.
How Rescale built a test-failure triage agent with Strands Agents and AI-DLC
Rescale, a digital engineering platform company, built an agent that classifies and routes failures from its nightly end-to-end tests, using the AI-DLC methodology and the Strands…
As reported by AWS, classification agreement with human triage rose from 44% at initial deployment to 68% today.