See how Claude Code and Codex do on the whole QA job, not just writing tests
QA Wolf ran Claude Code and Codex through the 1,567 evals it uses for its own QA agent, across web, iOS and Android apps, to test whether they can do the rest of the QA job beyond writing a passing Playwright test.
Source
@QAWolfHQ on X · 2026-10-20
Written in our own words. Credit to the original author. Link checked 2026-10-23.
What they did
QA Wolf ran Claude Code and Codex through the 1,567 evals it uses for its own QA agent, across web, iOS and Android apps, to test whether they can do the rest of the QA job beyond writing a passing Playwright test.
Result (as claimed)
A large public comparison of coding agents on real QA tasks to read before trusting one with testing.
Tools they used
Cost: Claude Code: from $17/mo; OpenAI Codex: free plan, paid from $8/mo.
Set it up
Pick a contained repo or task
Start on a branch or side project where mistakes are cheap.
Install and configure
Set up Claude Code, OpenAI Codex, Playwright. Read what it can run on your machine and limit permissions where possible.
Run it on a real task
Give it a well-scoped issue and let it work. Review the diff like any pull request.
Add guardrails
Keep tests and CI in the loop, and never let an agent push to main or touch production secrets unattended.
Copy the full setup prompt
Paste into your agent and it will walk you through the setup, asking before anything that sends, posts or spends.
You are helping me set up this AI use case end to end in Claude Code. Goal: See how Claude Code and Codex do on the whole QA job, not just writing tests What the original author did: QA Wolf ran Claude Code and Codex through the 1,567 evals it uses for its own QA agent, across web, iOS and Android apps, to test whether they can do the rest of the QA job beyond writing a passing Playwright test. Reported result (as claimed by the author): A large public comparison of coding agents on real QA tasks to read before trusting one with testing. Source: https://x.com/QAWolfHQ/status/2112575375524143500 Tools they used: Claude Code, OpenAI Codex, Playwright Category: Dev & ops Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Codex. Goal: See how Claude Code and Codex do on the whole QA job, not just writing tests What the original author did: QA Wolf ran Claude Code and Codex through the 1,567 evals it uses for its own QA agent, across web, iOS and Android apps, to test whether they can do the rest of the QA job beyond writing a passing Playwright test. Reported result (as claimed by the author): A large public comparison of coding agents on real QA tasks to read before trusting one with testing. Source: https://x.com/QAWolfHQ/status/2112575375524143500 Tools they used: Claude Code, OpenAI Codex, Playwright Category: Dev & ops Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Grok Bot. Goal: See how Claude Code and Codex do on the whole QA job, not just writing tests What the original author did: QA Wolf ran Claude Code and Codex through the 1,567 evals it uses for its own QA agent, across web, iOS and Android apps, to test whether they can do the rest of the QA job beyond writing a passing Playwright test. Reported result (as claimed by the author): A large public comparison of coding agents on real QA tasks to read before trusting one with testing. Source: https://x.com/QAWolfHQ/status/2112575375524143500 Tools they used: Claude Code, OpenAI Codex, Playwright Category: Dev & ops Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
The Daily 5
Get five new use cases tomorrow morning
Free. Real sources. Unsubscribe any time.
Guides for this use case
How to test your app with AI agents before you ship · How to vibe code an app with Claude Code · How to use Claude Code and Codex together · How to build an iPhone or Android app with AI
Related use cases
See your Claude Code and Codex limits in your Mac menu bar
Tired of hitting limits mid-task, he built Pikori, a small Mac menu bar app showing Claude Code and Codex 5-hour and weekly limits and reset times, with a pixel buddy that gets nervous as you run low.
Put Claude Code, Codex and Gemini in one terminal list with live status
A developer built agent manager, a local terminal app that connects to the subscriptions and tools you already have. Instead of separate tabs where one agent sits on an unseen permission prompt, every session appears in one list with live status.
Give Claude Code, Codex and Cursor one shared, self-improving memory
An AI educator connected Claude Code, Codex and Cursor to one memory layer, so a correction made in Cursor carries over to Claude Code and skills built in Codex are available everywhere. The memory updates itself as the agents learn.
Use Claude Code, Codex and Gemini from one self-hosted assistant app
Conch is an open-source (MIT) self-hosted assistant that connects the plans and keys you already have, such as Claude Code, Codex, Copilot, Gemini and Ollama, and switches models without losing the thread. It works on web, desktop, phone and Telegram and stores everything in ~/.conch.