Vibe code a mixed-reality Quest 3 app with hand tracking and AI-made 3D assets
An artist built a Quest 3 mixed-reality app with Grok Bot in Unity and Meta's MR Utility Kit, with every camera asset generated in Tripo, passthrough and hand tracking, a cloth-tear simulation and in-app sounds.
Source
@xbh_artist on X · 2026-10-21
Written in our own words. Credit to the original author. Link checked 2026-10-23.
What they did
An artist built a Quest 3 mixed-reality app with Grok Bot in Unity and Meta's MR Utility Kit, with every camera asset generated in Tripo, passthrough and hand tracking, a cloth-tear simulation and in-app sounds.
Result (as claimed)
A working hand-tracked MR experience running live on the headset.
Tools they used
Cost: Grok Bot: from $30/mo.
Set it up
Write a one-page brief
Describe the app in plain words: who it is for, the 3 core screens or actions, and what "done" looks like.
Start the project with your agent
Open Grok Bot and paste the brief. Ask it to propose a stack and a file plan before writing code.
Build one slice at a time
Ship the smallest working flow first, run it, then add features one by one. Commit after each working step.
Test it like a user
Click through on desktop and phone. Ask the agent to add basic tests and fix anything that breaks.
Check security and ship
Keep keys in environment variables, check auth and database rules, then deploy and share the link.
Copy the full setup prompt
Paste into your agent and it will walk you through the setup, asking before anything that sends, posts or spends.
You are helping me set up this AI use case end to end in Claude Code. Goal: Vibe code a mixed-reality Quest 3 app with hand tracking and AI-made 3D assets What the original author did: An artist built a Quest 3 mixed-reality app with Grok Bot in Unity and Meta's MR Utility Kit, with every camera asset generated in Tripo, passthrough and hand tracking, a cloth-tear simulation and in-app sounds. Reported result (as claimed by the author): A working hand-tracked MR experience running live on the headset. Source: https://x.com/xbh_artist/status/2112839879520358624 Tools they used: Grok Bot, Tripo, Unity, Meta Quest 3 Category: Vibe coding & apps Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Codex. Goal: Vibe code a mixed-reality Quest 3 app with hand tracking and AI-made 3D assets What the original author did: An artist built a Quest 3 mixed-reality app with Grok Bot in Unity and Meta's MR Utility Kit, with every camera asset generated in Tripo, passthrough and hand tracking, a cloth-tear simulation and in-app sounds. Reported result (as claimed by the author): A working hand-tracked MR experience running live on the headset. Source: https://x.com/xbh_artist/status/2112839879520358624 Tools they used: Grok Bot, Tripo, Unity, Meta Quest 3 Category: Vibe coding & apps Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Grok Bot. Goal: Vibe code a mixed-reality Quest 3 app with hand tracking and AI-made 3D assets What the original author did: An artist built a Quest 3 mixed-reality app with Grok Bot in Unity and Meta's MR Utility Kit, with every camera asset generated in Tripo, passthrough and hand tracking, a cloth-tear simulation and in-app sounds. Reported result (as claimed by the author): A working hand-tracked MR experience running live on the headset. Source: https://x.com/xbh_artist/status/2112839879520358624 Tools they used: Grok Bot, Tripo, Unity, Meta Quest 3 Category: Vibe coding & apps Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
The Daily 5
Get five new use cases tomorrow morning
Free. Real sources. Unsubscribe any time.
Guides for this use case
How to set up Grok Bot agents for real work · How to make a hand-tracking or gesture-controlled game with AI · How to make 3D scenes and models with AI and Blender · How to vibe code an app with Claude Code
Related use cases
Let Claude Code install Unity and build a VR app from scratch
On a new MacBook, Claude Code installed Unity 6, the Unity CLI and the Pico headset CLI, then built the first version of a "freeze time" VR app using headless versions of every tool.
Have Codex check the work Claude Code did, both run from your Grok Bot
One model builds, a different one reviews, because an AI checking its own work shares the blind spots that caused the mistake. Kacper Rutkowski tried it on his booking site: Codex found 3 problems that would have stopped people booking and fixed them in about a minute.
Have Grok Bot drive Claude Code to build a web app while you sleep
A builder sends Grok Bot an idea, screenshots, a reference site and a screen recording; Grok researches it, writes the plan and has Claude Code build the app overnight.
Build a security scanner for vibe-coded sites in 48 hours
AppVitals flags exposed keys, open databases and API leaks in AI-built sites. Cursor agents wrote it and Grok Bot orchestrated, in under 48 hours.