Test whether code comments actually help your coding agent
A team deleted all 168,000 lines of comments from a copy of their codebase and had Claude Code with Opus 5.5 build the same features on both copies, three runs each, with a hidden test and a blind Claude judge. The code was just as good without comments and run-to-run cost swung by about 30% either way.
Source
epeicher on Hacker News · 2026-10-14 · Project link
Written in our own words. Credit to the original author. Link checked 2026-10-18.
What they did
A team deleted all 168,000 lines of comments from a copy of their codebase and had Claude Code with Opus 5.5 build the same features on both copies, three runs each, with a hidden test and a blind Claude judge. The code was just as good without comments and run-to-run cost swung by about 30% either way.
Result (as claimed)
Published method and results: no quality difference, and single runs are too noisy to compare.
Tools they used
Cost: Claude Code: from $17/mo; Claude Opus 5.5: from $4 in / $20 out per 1M tokens.
Set it up
Gather your sources
Collect the papers, notes, links or files you want to work from in one place.
Load them into the tool
Open Claude Code, Claude Opus 5.5 and add your sources. Keep the original files so you can check claims later.
Ask focused questions
Ask for summaries, comparisons or connections with citations to the source passages.
Verify and save
Spot check claims against the sources, then save the output as notes you can reuse.
Copy the full setup prompt
Paste into your agent and it will walk you through the setup, asking before anything that sends, posts or spends.
You are helping me set up this AI use case end to end in Claude Code. Goal: Test whether code comments actually help your coding agent What the original author did: A team deleted all 168,000 lines of comments from a copy of their codebase and had Claude Code with Opus 5.5 build the same features on both copies, three runs each, with a hidden test and a blind Claude judge. The code was just as good without comments and run-to-run cost swung by about 30% either way. Reported result (as claimed by the author): Published method and results: no quality difference, and single runs are too noisy to compare. Source: https://news.ycombinator.com/item?id=50090484 Tools they used: Claude Code, Claude Opus 5.5 Category: Research & learning Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Codex. Goal: Test whether code comments actually help your coding agent What the original author did: A team deleted all 168,000 lines of comments from a copy of their codebase and had Claude Code with Opus 5.5 build the same features on both copies, three runs each, with a hidden test and a blind Claude judge. The code was just as good without comments and run-to-run cost swung by about 30% either way. Reported result (as claimed by the author): Published method and results: no quality difference, and single runs are too noisy to compare. Source: https://news.ycombinator.com/item?id=50090484 Tools they used: Claude Code, Claude Opus 5.5 Category: Research & learning Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Grok Bot. Goal: Test whether code comments actually help your coding agent What the original author did: A team deleted all 168,000 lines of comments from a copy of their codebase and had Claude Code with Opus 5.5 build the same features on both copies, three runs each, with a hidden test and a blind Claude judge. The code was just as good without comments and run-to-run cost swung by about 30% either way. Reported result (as claimed by the author): Published method and results: no quality difference, and single runs are too noisy to compare. Source: https://news.ycombinator.com/item?id=50090484 Tools they used: Claude Code, Claude Opus 5.5 Category: Research & learning Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
The Daily 5
Get five new use cases tomorrow morning
Free. Real sources. Unsubscribe any time.
Guides for this use case
How to vibe code an app with Claude Code ยท How to compare AI models on the same prompt
Related use cases
Measure how much time Claude Code actually saves from your logs
After five weeks of using Claude Code for most work, the author went through the logs: 159 sessions, 754 requests and about 70 hours of Claude working, compared against a by-hand estimate.
Give your vibe-coded site a design checkup with a taste skill
Install TasteSkill v2 in Claude Code or Codex and have Opus use it to open your pages at desktop, tablet and phone widths and list every design problem it finds.
Build a VS Code extension that previews referenced equations on hover
A researcher asked Claude Opus for a VS Code extension that shows the referenced equation, figure, section or table on hover, and it worked.
Turn a child's essay into an interactive animated storybook
Given a Chinese composition written for Singapore primary school students, Claude Code with Opus 5.5 built an interactive animated storybook: 12 cinematic shots, watercolor-style 3D scenes, character animation, sound effects and story choices.