Benchmark AI agents by having them play an MMO
RuneScape Bench drops AI agents into an open-source RuneScape server to test long-horizon play like questing, skilling and trading.
Source
evakhoury on Hacker News · 2026-10-17 · Project link
Written in our own words. Credit to the original author. Link checked 2026-10-22.
What they did
RuneScape Bench drops AI agents into an open-source RuneScape server to test long-horizon play like questing, skilling and trading.
Result (as claimed)
A game-based benchmark for agents; 248 points on HN.
Tools they used
Cost: Pricing depends on the tools you pick; check each official pricing page.
Set it up
Gather your sources
Collect the papers, notes, links or files you want to work from in one place.
Load them into the tool
Open AI agents, RuneScape and add your sources. Keep the original files so you can check claims later.
Ask focused questions
Ask for summaries, comparisons or connections with citations to the source passages.
Verify and save
Spot check claims against the sources, then save the output as notes you can reuse.
Copy the full setup prompt
Paste into your agent and it will walk you through the setup, asking before anything that sends, posts or spends.
You are helping me set up this AI use case end to end in Claude Code. Goal: Benchmark AI agents by having them play an MMO What the original author did: RuneScape Bench drops AI agents into an open-source RuneScape server to test long-horizon play like questing, skilling and trading. Reported result (as claimed by the author): A game-based benchmark for agents; 248 points on HN. Source: https://news.ycombinator.com/item?id=50136126 Tools they used: AI agents, RuneScape Category: Research & learning Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Codex. Goal: Benchmark AI agents by having them play an MMO What the original author did: RuneScape Bench drops AI agents into an open-source RuneScape server to test long-horizon play like questing, skilling and trading. Reported result (as claimed by the author): A game-based benchmark for agents; 248 points on HN. Source: https://news.ycombinator.com/item?id=50136126 Tools they used: AI agents, RuneScape Category: Research & learning Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Grok Bot. Goal: Benchmark AI agents by having them play an MMO What the original author did: RuneScape Bench drops AI agents into an open-source RuneScape server to test long-horizon play like questing, skilling and trading. Reported result (as claimed by the author): A game-based benchmark for agents; 248 points on HN. Source: https://news.ycombinator.com/item?id=50136126 Tools they used: AI agents, RuneScape Category: Research & learning Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
The Daily 5
Get five new use cases tomorrow morning
Free. Real sources. Unsubscribe any time.
Guides for this use case
How to build AI trading and investing agents · How to compare AI models on the same prompt · How to use Claude Code skills and plugins · How to run multiple coding agents together
Related use cases
Use thousands of AI agents to find new leads in biological data
OpenAI describes running thousands of AI agents in parallel across complex biological datasets to surface new research leads for scientists to test.
Benchmark AI models by having them play StarCraft through code
StarSkirmish Bench has LLMs play StarCraft: Brood War by writing code, now refreshed with results from the flagship OpenAI, Anthropic, Google and Muse models including Gemini 4 Argon.
Benchmark AI decision models by making them play Pac-Man
Built Jevman so decision-model APIs compete at Pac-Man as a fast, simple decision benchmark.
Make LLM agents team up in Pokemon Showdown battles
A researcher built a 2v2 Pokemon Showdown environment where two LLM agents must coordinate through team chat. You can watch battles with the chat shown, and the code is on GitHub.