Hermes agent video analysis comes in three forms you can set up today: the built-in video_analyze tool for questions about a clip, the bundled youtube-content skill for turning YouTube transcripts into summaries, and the community video-xray plugin for frame-by-frame reports.

This guide shows how to switch on each one, then compares them side by side so you can pick the right one for the job in front of you.

Each step traces back to official Hermes pages, the project's Python source, or video-xray's catalog file and README.

Want a second opinion on which route fits your stack? Book a free SEO strategy session with my team.

The Three Ways Side By Side

RouteWhat it doesWhat it needsBest for
video_analyze toolSends a clip to a video-capable model and returns a written answer to your questionThe opt-in video toolset plus an auxiliary vision providerQuick questions about short clips, on any platform
youtube-content skillFetches a YouTube transcript and turns it into a summary, chapters, quotes, a thread or a blog postThe bundled skill and its youtube extra prepared onceTalks, tutorials and webinars already on YouTube
video-xray pluginSamples frames in windows, transcribes speech and writes a timestamped Markdown reportHermes Desktop 0.22.1+, Windows, ffmpeg, a vision-capable chat modelLong local recordings you want broken into scenes and quotes

The quick rule: question about a file, use the tool; spoken content on YouTube, use the skill; full breakdown of a long file, use the plugin.

How To Switch On The video_analyze Tool

The built-in tools reference lists video_analyze under the video toolset, which is opt-in and absent from a default CLI session.

  1. Start a session with --toolsets video, or add video to the toolsets: list in config.yaml.
  2. Configure an auxiliary vision provider under auxiliary.vision, as the configuration docs describe.
  3. Set a model that accepts video input; auxiliary.video.model is read first, then auxiliary.vision.model.
  4. Ask about a short local file, such as "list the on-screen text in clip.mp4 with rough timestamps".

The Hermes source describes the tool as sending video to a video-capable model, with Gemini as its example, and accepts mp4, webm, mov, avi, mkv and mpeg.

Video still borrows its provider from the image-vision settings; issue 27313, which asks to split them, remains open.

How To Use The youtube-content Skill

This route skips video files entirely.

The youtube-content docs page sums it up as YouTube transcripts to summaries, threads and blogs, and the skill ships bundled with Hermes.

  1. Prepare the skill's youtube extra once, following the setup section of the skill file.
  2. Type /youtube-content followed by what you want and the link, for example a chapter list for a webinar URL.
  3. Name the format you want: chapters, summary, chapter summaries, thread, blog post or quotes.

Leave the format out and it defaults to a summary.

It takes standard links, short links, Shorts, embeds, live links or a bare video ID, and it chunks transcripts longer than about 50K characters before merging the summaries.

🔥 Get all three video routes set up inside a working agent stack Inside the AI Profit Boardroom you get setup trainings for Hermes, the Agent OS ready to install, live coaching calls four times a week and 3,400+ members swapping the skills and plugins they use. → Join the AI Profit Boardroom

How To Install The video-xray Plugin

The 15 November 2026 canary notes list video-xray as a new catalog entry from @LuuMoss, and its catalog file sets it in the community tier at version 0.2.2.

  1. Get ffmpeg onto the PATH of a Windows PC that runs Hermes Desktop 0.22.1 or later.
  2. Work through the README's three-part install: place the plugin's backend in Hermes' plugins directory, enable it by name with hermes plugins enable, restart Desktop, and finally drop the panel script into the desktop-plugins directory.
  3. Open the workbench with /video, untick "Analyze attached videos on send", and pick a speech engine.
  4. Attach a file, set a time window and a granularity of 8 to 64 frames per segment, and run the job.

The plugin also adds two chat tools, video_xray and video_xray_jobs, so a job can be started by asking in plain words.

What Each Route Can Actually Do

  • video_analyze: describes visual content, motion, sound cues, overlays and scene changes, then answers your question; works from a URL or a local path.
  • youtube-content: produces chapter lists with timestamps, five-to-ten sentence summaries, per-chapter summaries, X threads, blog posts and timestamped quotes from the spoken words.
  • video-xray: builds a report covering plot flow, camera language, characters and settings, quotes and an opening-versus-ending contrast; exports SRT and JSON cues; shows a frame gallery; and writes a shot-by-shot generation prompt.

Only the plugin creates files you can drop into an editor, and only the skill works without any vision model at all.

The Limits That Decide Most Choices

  • Size: video_analyze warns above 20 MB and stops at roughly 50 MB.
  • Hosting: per the source, a hosted agent cannot process video at all.
  • Captions: the YouTube skill returns nothing useful when a video has transcripts disabled, and it never sees on-screen text.
  • Platform: video-xray is verified on Windows only, and its install paths are Windows paths.
  • Tier: video-xray is a community plugin with a single third-party maintainer, not a Nous Research plugin.

Common Mistakes To Avoid

  • Expecting the tool after only adding the toolset. Without a resolvable auxiliary vision client, video_analyze never registers.
  • Assuming a strong main model is enough. The source keeps video on the auxiliary path, so the main model's vision support does not help here.
  • Feeding a full recording to the tool. Trim it under the cap, or use the plugin's time windows instead.
  • Using the YouTube skill for visual checks. A logo, a price tag or a slide only shown on screen is missing from the transcript.
  • Leaving auto-analysis on. The plugin's security note says each attached video starts a job on send by default.
  • Running video-xray with a text-only chat model. The plugin looks through the chat model you already use, so it needs one that can see images.
  • Confusing reading with making. Video creation is a separate toolset; see the Hermes Agent AI video generation guide.

📞 Not sure which route your footage needs? Book a free SEO strategy session and my team will match each kind of video you have to the right route. → Book the free session

Pick A Route By The Job

JobRouteReason
Summarize a conference talk on YouTubeyoutube-contentTranscript only, no download
Check the text on a short ad filevideo_analyzeSees overlays the transcript misses
Subtitle a long training recordingvideo-xraySRT export and windowed runs
Answer a question about a clip posted in a team chatvideo_analyzeGateway caches the clip and opens it on request
Turn a webinar into a blog postyoutube-contentBlog post is a built-in output format
Break a commercial into shotsvideo-xrayAdjustable shot-cut threshold from 0.05 to 0.95

Combining The Routes On One Project

The routes overlap less than they seem, so a single project can use all three without wasted work.

Take a product launch with a public webinar, a set of short ad files and a long internal training recording.

  1. Run /youtube-content on the published webinar and ask for chapter summaries plus timestamped quotes for the recap post.
  2. Run video_analyze on each short ad and ask what text appears on screen and when the product first shows.
  3. On the Windows Desktop machine, run video-xray over the internal training file in windows, with auto-analysis off, and export the SRT.

Each step uses the lightest input that still answers the question: text for the talk, a small file for the ads, sampled frames for the long recording.

That order also limits what leaves your machine, since only the ads go to the auxiliary provider and the training file can stay on a local model.

Who Each Route Suits

  • The tool suits anyone on Linux, macOS or Windows who wants quick, visual answers and already runs Hermes with an auxiliary model.
  • The skill suits content teams and marketers who live on YouTube and care about what was said more than what was shown.
  • The plugin suits Windows Desktop users who process long recordings every week and want files, not just answers.

Plenty of setups end up using two of them: the skill for public videos and the tool for private clips.

Privacy Checks Before You Start

Each route sends something somewhere, so check it against your rules.

  • The tool sends the clip to your auxiliary provider; the website blocklist in the security docs also applies to its URLs.
  • The skill fetches public transcripts from YouTube and hands the text to your chat model.
  • The plugin sends frames to your chat model, sends audio to a cloud speech endpoint only if you set one, and keeps a saved cloud key as readable text behind owner-only permissions.

For staff or customer footage, a local vision model with the plugin keeps frames on the machine.

Model Fit For The Visual Routes

The vision docs list Claude with vision, Gemini and Qwen-VL among vision-capable families, and DeepSeek V4 Pro among text-only models.

The tool needs an auxiliary model that accepts video input, and the Hermes error hint points to a Gemini model.

The plugin needs a chat model that can see images, local or cloud.

The skill needs neither, since it only reads text.

Which Route To Start With

Start with the route that matches your most common video job, not the one with the most features.

For most people that is the YouTube skill for public talks and the built-in tool for private clips, both of which take one sitting to set up.

Add video-xray when long local recordings become a weekly job and a Windows Desktop machine is available.

Test each route on two videos you already know well, so you can tell at a glance whether the output is accurate.

Keep the route that gets your real videos right, and write down which kinds of footage go to which route so the whole team follows the same rule.

Members of the AI Profit Boardroom can watch each route set up step by step inside the Agent OS.

Also On Our Network

FAQ: How To Make Hermes Agent Video Analysis Work

What are the ways to analyze video with Hermes Agent?

Three: the built-in video_analyze tool, the bundled youtube-content skill for YouTube transcripts, and the community video-xray plugin for full frame-by-frame reports.

Which route works without a vision model?

The youtube-content skill. It reads the video's transcript, so any chat model can summarize it, but it cannot see anything shown only on screen.

Why does video_analyze not appear after I add the toolset?

Hermes only registers it when an auxiliary vision client resolves. Configure a provider under auxiliary.vision and a video-capable model, then start a new session.

Can I run video-xray on a Mac?

Not as a verified setup; the catalog names Windows alone. Mac users can rely on video_analyze and the YouTube skill instead.

Which route gives me a subtitle file?

video-xray. It exports SRT and JSON cues that import into CapCut, Premiere or any player.

Can these tools create a new video?

No. All three read video. Generation is a separate Hermes toolset called video_gen.

📺 Video notes + links to the tools 👉

🎥 Learn how I make these videos 👉

🆓 Get a FREE AI Course + Community + 1,000 AI Agents 👉

About Julian

I'm Julian Goldie, an SEO entrepreneur, author and founder of the AI Profit Boardroom, which has 3,400+ members.

I help business owners scale with AI agents, automation and SEO.

  • I've built a 7-figure SEO and link building agency, Goldie Agency, with a team of 70+ SEO experts.
  • I've grown a YouTube channel to 400,000+ subscribers.
  • I wrote "Link Building Mastery".
  • My Udemy courses have taught over 29,000 students.

→ Get my best AI training inside the AI Profit Boardroom

Pick the route that fits your next video, set it up today, and talk it through on a free SEO strategy session or inside the AI Profit Boardroom if you want help with hermes agent video analysis.