Gstrl — Control Your Mac With Hand Gestures

Gesture, voice, and AI control for macOS.

View on GitHub<br>See gestures

↓ Scroll to explore

Gestures.

Just your Mac's webcam. No special hardware.

cursor<br>👌 Pinch to move

Right hand pinch — move your hand to drag the cursor anywhere on screen.

click<br>👌 Pinch to click

Quick left pinch for click. Hold for 1 second for right-click.

screenshot<br>⭕ Circle to screenshot

Draw a circle while pinching — captures that region to your clipboard instantly.

↑ ↓ ← →<br>🖐 Swipe for arrows

Open hand flick in any direction — up, down, left, right arrow keys.

dictate<br>✊ Fist to speak

Hold a fist to activate speech-to-text. Say "press enter" or just dictate.

AI agent<br>✊✊ Both fists for AI

Hold both fists — ask Claude anything. Gets spoken back to you.

scroll ↕<br>👌✊ Pinch + fist to scroll

Left pinch, right fist. Move left hand up/down to scroll.

⌫ delete<br>🤙 Six to delete

Hold the shaka — deletes characters. Both hands 🤙🤙 = delete lines.

drag / select<br>👌👌 Both pinch to drag

Left pinch holds click, right hand pinch moves — drag and drop files, select text.

Voice commands.

Say commands to trigger actions instead of typing. Dictation supports multiple languages.

👆<br>Click

click &middot; right click &middot; command click

⌨️<br>Press + key

press enter &middot; press delete &middot; press tab &middot; press escape &middot; press up/down/left/right

Modifiers

command z &middot; control c &middot; shift left &middot; option delete &middot; command shift z

Get started.

Clone, build, run. No accounts needed.

Terminal

git clone https://github.com/TomYang-TZ/Gstrl.git<br>cd Gstrl<br>make install<br>make run<br>Requires macOS 14+, a webcam, and Swift 5.9+. Permissions auto-prompt on first launch. Claude Code CLI optional for AI agent.

How it works.

All processing on-device. Zero latency. Zero cloud.

Webcam captures frames

AVCaptureSession at 30fps (configurable to 120fps) feeds frames to Apple Vision.

Vision detects hand poses

VNDetectHumanHandPoseRequest identifies 21 joints per hand, every frame.

Classifier maps to actions

Pinch detection, velocity-based swipes, and combo tracking turn poses into CGEvents.

Speech + voice commands

Hold a fist to activate speech. Dictate text or say "press enter", "command z", "click" to trigger actions. Dictation supports multiple languages.