Let Hermes Agent use your real phone for apps that have no API
Hermes Agent's plugin catalog has a one-install phone-harness plugin. The agent sees the phone screen, taps, swipes and types, so it can work with apps that have no API or plugin of their own.
Source
@witcheer on X · 2026-10-15
Written in our own words. Credit to the original author. Link checked 2026-10-23.
What they did
Hermes Agent's plugin catalog has a one-install phone-harness plugin. The agent sees the phone screen, taps, swipes and types, so it can work with apps that have no API or plugin of their own.
Result (as claimed)
Hermes can check and act inside ordinary phone apps the way a person would.
Tools they used
Cost: Hermes Agent: free plan, paid from $20/mo. Checked 2026-10-14.
Set it up
Map the task
Write the trigger, the inputs, the decisions and the output. Note which steps need a human yes.
Connect the tools
Set up Hermes Agent, phone-harness, Android, iPhone with the narrowest permissions that work. Use a test account or sandbox first.
Build the workflow
Create the steps in order. Add an approval step before anything sends, pays or deletes.
Run it on real but low-stakes data
Run a few times, read every output, and fix prompts or rules that misfire.
Schedule and monitor
Turn on the schedule or trigger, and add a daily summary or alert so you notice failures.
Copy the full setup prompt
Paste into your agent and it will walk you through the setup, asking before anything that sends, posts or spends.
You are helping me set up this AI use case end to end in Claude Code. Goal: Let Hermes Agent use your real phone for apps that have no API What the original author did: Hermes Agent's plugin catalog has a one-install phone-harness plugin. The agent sees the phone screen, taps, swipes and types, so it can work with apps that have no API or plugin of their own. Reported result (as claimed by the author): Hermes can check and act inside ordinary phone apps the way a person would. Source: https://x.com/witcheer/status/2110664477301743817 Tools they used: Hermes Agent, phone-harness, Android, iPhone Category: Automation & business Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Codex. Goal: Let Hermes Agent use your real phone for apps that have no API What the original author did: Hermes Agent's plugin catalog has a one-install phone-harness plugin. The agent sees the phone screen, taps, swipes and types, so it can work with apps that have no API or plugin of their own. Reported result (as claimed by the author): Hermes can check and act inside ordinary phone apps the way a person would. Source: https://x.com/witcheer/status/2110664477301743817 Tools they used: Hermes Agent, phone-harness, Android, iPhone Category: Automation & business Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
You are helping me set up this AI use case end to end in Grok Bot. Goal: Let Hermes Agent use your real phone for apps that have no API What the original author did: Hermes Agent's plugin catalog has a one-install phone-harness plugin. The agent sees the phone screen, taps, swipes and types, so it can work with apps that have no API or plugin of their own. Reported result (as claimed by the author): Hermes can check and act inside ordinary phone apps the way a person would. Source: https://x.com/witcheer/status/2110664477301743817 Tools they used: Hermes Agent, phone-harness, Android, iPhone Category: Automation & business Please: 1. Ask me 3 short questions about my situation (accounts I have, budget, what I want the output to look like). 2. Propose a step-by-step plan using these tools or cheaper alternatives, with the time and cost of each step. Use official pricing pages; say "check pricing" if unsure. 3. Then do the setup with me one step at a time, waiting for my OK before anything that sends messages, posts publicly, spends money or deletes data. 4. Finish with a checklist to run it again next time.
The Daily 5
Get five new use cases tomorrow morning
Free. Real sources. Unsubscribe any time.
Guides for this use case
How to let an AI agent use your phone (iPhone and Android) · How to control coding agents from your phone · How to use Claude Code skills and plugins · How to set up Hermes Agent workflows
Related use cases
Let Grok Bot use the apps on your phone to handle everyday tasks
Grok Bot's iOS and Android apps gained local phone tools: it can read and send messages, check your calendar and create events, set reminders, look up contacts and more, straight from a chat.
Let an agent operate Android apps on the phone itself
Opposable is an Android app that reads the screen through the accessibility service and taps and types in other apps. Claude, ChatGPT or Codex can drive it from a computer, and with none connected a 2B screen model runs the same loop on the phone with no server.
Have Claude Code order a Waymo by tapping through your actual iPhone
A demo shows Claude Code using a real iPhone: it reads the mirrored screen on a Mac with OCR and taps around like a person. It also works on Android over adb, with nothing to install on the phone.
Plan your whole day on your commute with an agent that reads your phone calendar
He uses Grok Bot's new local tools on iPhone and Android to work through his calendar, reminders and contacts on the way to work, asking the bot to lay out the day.