How Datadog tested Codex code review by replaying past incidents
Datadog ran Codex against old pull requests that had contributed to incidents, to check whether AI review would have caught the risk. After the test, it rolled Codex review out widely.
The problem
Senior engineers could not review every change for system-wide risk across interconnected services. Earlier AI review tools gave shallow, noisy comments that engineers tended to ignore.
How they did it
- Piloted Codex review on every pull request in one of Datadog's largest repositories.
- Gathered engineer feedback through thumbs-up and thumbs-down reactions.
- Built an incident replay harness that rebuilt historical PRs linked to incidents and ran Codex on them.
- Asked the engineers who owned each incident whether the Codex feedback would have helped.
- Deployed Codex across the engineering organization after the evaluation.
Results as reported by OpenAI
- As reported by OpenAI: Codex 'found more than 10 cases, or roughly 22% of the incidents' examined, where engineers said its feedback would have made a difference.
- 'More than 1,000 engineers' now use it regularly.
These figures are as reported by OpenAI in OpenAI customer story ↗ (Jan 9, 2026). OpenAI sells the product used here. AgentGid did not measure or independently verify them.
This suits teams with a sizable codebase and a documented incident history, plus someone able to build a replay harness and ask incident owners for feedback. Watch the figures: the 22% catch rate and the 1,000+ engineers come from OpenAI's report, not independent data, and your own results may differ. For a cheaper trial, Pi is Free (OSS), though you pay your model provider.
The agent used here
Similar use cases
How Eon unified multi-cloud incident tooling behind one MCP Gateway on AgentCore
Eon, a cloud data protection vendor, replaced per-tool credentials and local MCP servers with a single authenticated MCP gateway on Amazon Bedrock AgentCore. Developers and an aut…
As reported by AWS (guest post co-written with Eon): developer onboarding dropped from 2 days to 2 minutes.
Organize agent skills as installable company departments with headcount
An open-source GitHub repository called headcount arranges agent skills like a company, with a chief executive over departments such as security, finance, product and marketing. E…
As reported by the repository README: a chief executive over 16 departments, 172 skills in total.
Run Claude Code and Codex agents in parallel on Mac with Offrun
Offrun is a free Mac app that runs coding agent CLIs such as Claude Code and Codex side by side, each in its own git worktree, using the user's own logins. The page is the product…
As reported by the Offrun site: each agent in a project works in its own git worktree, so two agents on the same repo do not touch the same files.
Guides
JEV for Claude Code: A Second Opinion on Every Command
A viral post says jev-axi screens every command your Claude Code agent runs for a fraction of a cent. We read the repo, TypeSafe's docs and privacy policy: what runs locally, what is sent, how it fails, and how it compares with auto mode.
VoiceStudio: The Free, Local ElevenLabs Alternative
A viral post calls VoiceStudio a 100% free ElevenLabs replacement with 19.4K stars. We checked it against GitHub and the project's own pages: what it does, what is really free, the license catch in the default voice model, and how to plug it into Claude Code or Codex.
Best Claude Code Alternatives in 2026: Open-Source, Cheaper and Multi-Model
The strongest alternatives to Claude Code, grouped by why you are switching: price, model choice, open source or a different workflow. Live usage and prices for each.
AI Coding Agent Pricing Explained: Claude Code, Cursor, Codex and Copilot (2026)
What Claude Code, Cursor, OpenAI Codex, GitHub Copilot and other coding agents really cost: plan prices, usage limits, subscription vs API billing, and how to estimate your monthly bill.