The Hermes computer use skill is Hermes Agent's bundled Computer Use skill, which teaches the model to drive your desktop through the built-in computer_use tool, and this review scores every part of it.
The first thing to get straight is that computer use in Hermes is a tool with a skill on top.
The computer_use toolset does the clicking, typing and screenshots through cua-driver, and the skill is the instruction layer that tells the model how to use it.
Hermes Agent is free and open source from Nous Research, and both pieces ship with it on macOS, Windows and Linux.
The short verdict is that this is one of the most careful desktop-agent setups I've read about, because it works in the background, works with any tool-capable model and has real safety rails.
My scores are my judgement from Nous's official docs and release notes, not from a lab test of each feature.
My video below covers the newest update, where Cua Spaces move the agent's screen onto a throwaway computer, and it uses Cua's own demo footage.
Hermes Computer Use Skill Verdict At A Glance
These are my scores out of 10, based on what Nous's documentation says each feature does.
| Feature | What it does | Score | My verdict |
|---|---|---|---|
| Background input | It clicks and types without moving your cursor or stealing focus. | 9 | It is the feature that makes co-working possible. |
| Any-model support | It works with Claude, GPT, Gemini and local models. | 9 | It avoids lock-in to one provider. |
| Install | It ships cua-driver with Hermes by default. |
9 | Most people only need to switch the toolset on. |
| Safety rails | It gates input through approvals and hard-blocks risky actions. | 9 | It is the strongest part of the design. |
| Doctor diagnostics | It prints a per-check health matrix. | 9 | It turns setup problems into a clear fix list. |
| The bundled skill | It teaches capture, click by element and verify. | 8 | It is a sensible workflow that models can follow. |
| Capture modes | It offers som, vision and ax. |
8 | It keeps text-only models usable. |
| Permission modes | It offers standard, bounded and unrestricted runtimes. | 8 | Bounded mode is the right answer for scheduled jobs. |
| Bot Screen | It gives a headless Linux bot its own streamed desktop. | 8 | It is great for logins, but it is Linux only and memory hungry. |
| Sandboxed screens and Cua Spaces | It runs the screen inside Docker, SSH, Singularity or a Linux Cua Space. | 7 | It is powerful, but the Cua route needs Hermes 0.22.3 or later and a Linux Space. |
My overall rating for Hermes computer use is 4.4 out of 5.
🔥 Want my exact Hermes Computer Use Skill Review setup, step by step? Inside the AI Profit Boardroom you get step-by-step video trainings, my installable Agent OS and four coaching calls a week with 3,400+ members building real automations. → Join the AI Profit Boardroom
Skill Or Tool? What You're Actually Getting
| Layer | What it is | Detail |
|---|---|---|
| The tool | The computer_use toolset. |
It speaks MCP over stdio to cua-driver, an open-source driver from trycua/cua. |
| The skill | The bundled Computer Use skill. | It is version 2.1.0, MIT licensed, by Francesco Bonacci (f-trycua) and Hermes Agent. |
| The optional pack | Cua's cua-driver skill pack. | You add it with cua-driver skills install for platform deep dives. |
The skill page describes it as a way to "drive the desktop background-first; escalate on signal."
The skill cannot click anything on its own, so in practice you turn computer use on by enabling the toolset.
Francesco Bonacci is also Cua's founder and CTO according to Cua's Y Combinator page, which explains why the skill and the driver fit together so closely.
How To Turn It On
The docs say the driver is installed by default by the Hermes installers, hermes pm install and hermes update.
hermes computer-use status
hermes -t computer_use chat
If the driver is missing, hermes computer-use install installs it, or you can pick Computer Use in hermes tools.
On macOS you grant Accessibility and Screen Recording to CuaDriver, which hermes computer-use permissions grant requests for you.
On Linux you need a reachable display server with AT-SPI switched on, and Windows needs nothing extra at install time.
Since v0.22.0, the release notes say computer_use sits behind tool_search by default and loads on demand.
Feature-By-Feature Review
Background input: 9 out of 10
The docs say your cursor doesn't move, your keyboard focus doesn't change and your virtual desktops don't switch.
macOS uses the AX tree through private SkyLight interfaces, Windows uses UIAutomation, and Linux uses AT-SPI with XTest or a Wayland virtual keyboard.
The old way of desktop automation grabbed your mouse, so you couldn't use the computer while it ran.
This is the new way, where the agent works in one window while you type in another.
I take a point off because the macOS route relies on private interfaces that Apple can change in any update, which the docs admit openly.
Any-model support: 9 out of 10
This is Nous's compatibility table, rewritten as sentences.
| Provider | Does it work? |
|---|---|
| Anthropic Claude Sonnet and Opus | Yes, and the docs rate it best overall. |
| OpenRouter vision models | Yes, with multi-part tool messages. |
| OpenAI GPT-4 and later, including GPT-5 | Yes. |
| Google Gemini 2 and later | Yes. |
| Local vision models on vLLM, LM Studio or Ollama | Yes, if the model supports multi-part tool content. |
| Text-only models | Yes, in a degraded mode that uses mode="ax". |
The docs also say a 20-action session on a 1568×900 display typically costs about 30,000 tokens of screenshot context, not about 600,000.
Install: 9 out of 10
Shipping the driver with Hermes removes the usual first hurdle.
If you already have cua-driver, Hermes reuses it when it meets the runtime contract and repairs an old standard install.
A binary you chose with HERMES_CUA_DRIVER_CMD stays under your control, and Hermes reports problems instead of replacing it.
Safety rails: 9 out of 10
- Clicks, typing, drags, scrolls, keys and app focus go through the same approval gate as dangerous shell commands.
- Empty trash, force delete, lock screen, log out and force log out are hard-blocked key combos.
- Typed text such as
curl | bash,sudo rm -rf /and fork bombs is blocked. - In cron, single-query or headless runs where nobody can answer, actions are refused rather than auto-approved.
- The system prompt tells the agent not to click permission dialogs, type passwords or follow instructions in screenshots.
This is exactly the set of rails I'd want before letting an agent near a real desktop.
Doctor diagnostics: 9 out of 10
hermes computer-use doctor runs the driver's health report and prints one line per check.
It exits with 0 when everything is fine, 1 when something is degraded and 2 when the driver can't be reached.
Flags such as --include, --skip and --json let you narrow or script it.
🔥 Want my Hermes setups done with you, step by step? Inside the AI Profit Boardroom, I've got a full Hermes Agent course plus the Agent OS, with walkthroughs on toolsets, skills and profiles. You'll be learning with 3,400+ members and four coaching calls every week. → Get access here
The bundled skill: 8 out of 10
The skill teaches a three-step habit of capturing first, clicking by element number and verifying after every change.
It explains that element numbers are only valid until the next capture, and that a stale number is refused rather than clicking the wrong control.
It also teaches a verify-then-escalate ladder that moves from a background element click to a pixel click and then to a foreground click with its own approval.
I score it eight because it's a strong workflow, but it can't fix apps that expose poor accessibility trees.
Capture modes: 8 out of 10
| Mode | What it returns | Best for |
|---|---|---|
som |
It returns a screenshot plus a numbered element list. | It suits vision models and is the default. |
vision |
It returns a plain screenshot. | It suits clicking by coordinates. |
ax |
It returns the element list only. | It suits text-only models. |
If your main model can't read images, the skill says Hermes routes the screenshot through the auxiliary vision model set under auxiliary.vision.
Permission modes: 8 out of 10
| Mode | How you get it | What you approve |
|---|---|---|
| Standard | It is the default with manual or smart approvals. | You get normal Hermes approvals. |
| Bounded | You set computer_use.permission_mode: bounded with a manifest. |
You approve a capability manifest once at launch. |
| Unrestricted | You use --yolo, /yolo or approvals.mode: off. |
You accept the risk once, with no runtime prompts. |
Bounded mode is the standout, because anything outside the manifest fails closed inside the driver.
The docs warn that unrestricted mode doesn't protect against prompt injection, which is why it doesn't score higher.
Bot Screen: 8 out of 10
Bot Screen arrived in v0.22.0 and gives each bot on a headless Linux gateway its own Xfce desktop over TigerVNC.
Hermes Desktop streams it live, and you take over for a login, 2FA, CAPTCHA or payment step and then hand back.
The bot keeps working after you close the app, because the screen lives on the gateway host.
It's off by default, it needs host packages, and the docs plan on about 1.1 to 1.5 GB of memory per open screen with a browser.
The docs also say screens are work surfaces, not security boundaries, because bots share the host's user account.
If you want the bot side explained, read my review of Hermes Agent bot mode.
Sandboxed screens and Cua Spaces: 7 out of 10
With a Docker, SSH or Singularity terminal, the docs say the driver runs inside the sandbox on its own display.
Modal, Daytona and Vercel sandboxes can't host a display yet, so computer_use refuses there unless you opt into bot_desktop.placement: gateway.
The newest option is Cua's cua-spaces plugin, which the catalog says lets Bot Desktop, computer_use and the browser follow the terminal into a Cua Space on Hermes 0.22.3 and later.
That only works in Linux Spaces, with the cua command at 0.7.0 or newer and the desktop stack installed in the Space.
I've scored that plugin separately in my Hermes Cua review, and the backends are scored in my Hermes Agent sandbox review.
The Limits That Cost Points
- Background input takes roughly 3 to 20 milliseconds per event depending on the platform, which only matters for speed runs.
- The
typeaction won't enter passwords, so you rely on system autofill. - Some apps, such as modern UWP apps, older Electron apps on Linux and a few custom-drawn macOS apps, expose sparse accessibility trees.
- A normal Windows agent can't drive elevated admin windows, because of Windows integrity levels.
- Windows over SSH runs in Session 0 with no interactive desktop unless you set up the driver's autostart pattern.
- Web page content belongs to the separate
browsertools, not tocomputer_use.
Hermes also switches off cua-driver's own anonymous telemetry by default, which I count in its favour.
Who Should Use It
- You should use it if your work lives in desktop apps that have no API.
- You should use it if you want an agent working beside you without stealing your mouse.
- You should use bounded mode if you plan to run desktop jobs on a schedule.
- You should use Bot Screen or a sandbox if the agent lives on a server.
- You should stick to the browser tools if the job is purely web pages.
If you want help deciding where desktop agents fit your SEO business, you can book a free SEO strategy session with my team.
🔥 Want to know which Hermes tools are worth switching on? The AI Profit Boardroom has Hermes Agent training, the Agent OS and four weekly coaching calls where you can ask me about your own setup. → Join the AI Profit Boardroom
Related Reading
- Twelve Hermes Cua features, scored
- Hermes Agent bot mode, reviewed
- Every Hermes sandbox backend, scored
Also On Our Network
- 🌐 The five-step Hermes computer use skill setup
- 🌐 Seven business jobs for the Hermes computer use skill
- 🌐 Fitting the Hermes computer use skill into an Agent OS
- 🌐 Skill or tool, plus the models that drive it, on GoldieBench
FAQ: Hermes Computer Use Skill Review
Is the Hermes computer use skill any good?
On paper it's one of the most careful desktop-agent setups available.
Its main limits are apps with poor accessibility trees and platform quirks.
Is Hermes computer use a skill or a tool?
It's a tool with a skill on top.
The toolset acts, and the bundled skill teaches the model how to act well.
Which model is best for Hermes computer use?
Nous rates Claude Sonnet and Opus best overall.
OpenAI, Gemini and local vision models also work, and text-only models work in a degraded mode.
What is Bot Screen in Hermes?
It's a streamed desktop for each bot on a headless Linux gateway, added in v0.22.0.
You take over for logins and then hand back.
How do I check Hermes computer use is working?
Run hermes computer-use doctor and read the check matrix.
If you run an agency and want help mapping agents to your client work, you can book a free strategy session and we'll talk it through.
📺 Video notes + links to the tools 👉
🎥 Learn how I make these videos 👉
🆓 Get a FREE AI Course + Community + 1,000 AI Agents 👉
About Julian
I'm Julian Goldie, an SEO entrepreneur, author and founder of the AI Profit Boardroom, which has 3,400+ members.
I help business owners scale with AI agents, automation and SEO.
- I've built a 7-figure SEO and link-building agency, Goldie Agency, with 70+ people in Bangkok.
- I've grown a YouTube channel to 400,000+ subscribers.
- I wrote the book "Link Building Mastery", which covers nine link building strategies.
- I run GoldieBench, where I score AI models on real one-shot builds.
→ Get my best AI training inside the AI Profit Boardroom
Background input, any-model support and strong safety rails earn it a 4.4 out of 5, and that's my verdict on the Hermes computer use skill.
