Vision based touch-less macOS control via hands (FOSS)

heresalexandria1 pts1 comments

Pawvis: touch-free hand & voice control for your Mac

Skip to content

Download

Touch-free gesture & voice control for macOS

A mouse made of thin air.

A Mac that answers to its name.

Pawvis reads your hand through the webcam. Raise it in the air and the<br>cursor follows, dip a finger to click, or say<br>&ldquo;Pawvis, open Safari&rdquo; out loud.<br>No mouse under your palm, no trackpad, nothing to touch. Your camera<br>never sends a frame anywhere.

Download for macOS

View on GitHub

Signed & notarized · macOS 14+ · Free & open source (MIT)

Gestures

Your webcam is the sensor.

The camera already built into your Mac watches your hand and tracks its<br>21 joints many times a second, using Apple's Vision framework. Nothing to<br>wear, nothing to hold, no depth sensor, no hardware to buy. Sit down, hold<br>up a hand, and your Mac can already see it.

Tracking watches whenever it's on, but the cursor only follows once you<br>show an open hand : all four fingers up, thumb free. Close<br>a fist for a moment, or leave the frame, and it parks. A hand that's merely<br>visible, resting or typing or gesturing, never drags the cursor around.

Move

Hold your hand open, fingers up, and move it. The cursor rides your palm.

Click

Dip your index finger, like tapping a mouse button. A quick release is always a clean click.

measured against your middle finger, so tilting your hand can't click

Right-click

Dip a second finger. Hold it down to right-drag.

pinky by default, configurable

Drag & hold

Keep the finger down and move. Deliberate movement starts a drag immediately; a short window protects quick clicks from becoming accidental drags.

the window length is a slider

Double & triple click

Tap again quickly in the same spot. Chained clicks land where the first one did.

Scroll

Fold your middle and ring fingers in, index and pinky up, then move your hand up and down. The cursor parks while the pose is held.

toggle it, or invert the direction, in Settings

Reach adapts to distance

Auto mode sizes the tracking area from how big your hand looks, so the whole screen stays reachable up close and far away, with your fingers still inside the camera frame. Manual mode gives you a fixed area and a slider.

&ldquo;Show an open hand. The cursor is yours.&rdquo;

On screen, a claw rides your palm: open while pointing, purple while the<br>left button is held, blue for the right, ringed in light blue while<br>scrolling, faded while control is parked. A ring tightens as your click<br>forms, a pulse confirms every click, and small dots mark each detected<br>fingertip. The menu bar icon opens a live status panel (hands seen,<br>control armed, voice state) plus a Gesture Guide that walks through every<br>move. Everything above is tunable in<br>Settings → Gestures .

In motion

Eight seconds, no hardware.

One hand, one built-in camera, and a cursor that goes where she points.

Pawvis moving the cursor by hand, live.

Voice beta

Talk to your Mac<br>by name

Switch it on in Settings → Voice (Beta) , press<br>Start in the menu bar, and say the wake word. Every command<br>begins with it, so anything you say that isn't meant for Pawvis is ignored:<br>never acted on, never typed, never put on screen.

out loud

Pawvis, go to github.com

Pawvis, type good morning

Pawvis, press command shift T

Pawvis, open Notes

Pawvis, switch to Chrome

Pawvis, scroll down a page

Pawvis, click sign in

Pawvis, stop listening

The wake word is configurable, and mishearings are tolerated. Speech<br>recognition is Apple's on-device engine: private, free, no API key, no<br>cloud. Anything that isn't a set phrase goes to on-device<br>Apple Intelligence, which maps your words to an intent and grounds<br>screen commands like &ldquo;click sign in&rdquo; against the area around<br>your pointer, widening to the whole screen only if the target isn't<br>nearby. That part needs macOS 26; everything else works without it.

optional · off by default

Or hand the whole command to an agent

Point Pawvis at an agent CLI you already have installed,<br>Claude Code or Codex , and it stops<br>parsing your speech and starts delegating it. Say<br>&ldquo;Pawvis, clean up my downloads folder&rdquo;<br>and everything after the wake word is piped to the agent to carry out<br>with computer use, running headless in the background.

A panel streams the agent's output live while it works, with a Cancel button.

Running sessions stay listed, and cancellable, under Settings → Voice.

The outcome, success or failure, always flashes in the on-screen capsule.

&ldquo;Pawvis, stop listening&rdquo; stays local, so you can always shut it off instantly.

Read this part twice: the agent does not ask.<br>Claude Code and Codex are launched with their own permission<br>prompts turned off, so nothing pauses to confirm anything. Whatever the<br>agent was handed, it carries out, with your account, your files and<br>your logged-in sessions: deleting things, installing things, running<br>shell commands, sending things on your behalf. Speech recognition is<br>not perfect, and a misheard command is...

pawvis hand click cursor open voice

Related Articles