GitHub - sophiamyang/finger-frame-effect-ai · GitHub
/" data-turbo-transient="true" />
Skip to content
Search or jump to...
Search code, repositories, users, issues, pull requests...
-->
Search
Clear
Search syntax tips
Provide feedback
--><br>We read every piece of feedback, and take your input very seriously.
Include my email address so I can be contacted
Cancel
Submit feedback
Saved searches
Use saved searches to filter your results more quickly
-->
Name
Query
To see all available qualifiers, see our documentation.
Cancel
Create saved search
Sign in
/;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up
Appearance settings
Resetting focus
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
sophiamyang
finger-frame-effect-ai
Public
Notifications<br>You must be signed in to change notification settings
Fork<br>28
Star<br>312
main
BranchesTags
Go to file
CodeOpen more actions menu
Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit
History<br>15 Commits<br>15 Commits
examples
examples
.gitignore
.gitignore
README.md
README.md
app.js
app.js
composite.py
composite.py
index.html
index.html
requirements.txt
requirements.txt
stylize.py
stylize.py
View all files
Repository files navigation
Finger Frame AI 🎬✨
Try it: https://sophiamyang.github.io/finger-frame-effect-ai/
Upload a video of the two-hand finger-frame gesture — get it back with an<br>AI-generated world inside the frame . The whole video is restyled by a<br>video-to-video model (motion, blinks, and all), then composited so the<br>finger frame acts as a window into the animated version.
The finger-frame family
App<br>Generation<br>Latency
finger-frame-effect (repo) — live camera, local effects<br>Canvas 2D (Van Gogh, toon, glitch, …)<br>none
this app — recorded video, AI restyle<br>Gemini Omni Flash (offline video edit)<br>minutes
finger-frame-effect-lucy (repo) — live camera, live AI<br>Decart Lucy 2.5 (realtime video-to-video)<br>~real time
Real hands, AI world — generated with the default "3D animated movie"<br>style (full-quality mp4).
How it works
Restyle — the uploaded video is sent to<br>Gemini Omni Flash video editing<br>with your chosen style (3D animated movie, anime, claymation,<br>watercolor, or a custom prompt). This is a true video model: the whole<br>clip is regenerated, so the animated version moves exactly like you.<br>Every prompt gets a strict-alignment suffix appended — same framing, no<br>zoom/crop/recentering, facial features at the same screen coordinates,<br>expression preserved frame by frame (mouth openness, blinks, gaze) — so<br>the result lines up behind the finger-frame window.
Track — MediaPipe Hand Landmarker finds both hands per frame, and the<br>finger-frame quad is tracked with the same audited pipeline as the live<br>app (anatomical corner ordering — crossing your fingers renders the<br>bowtie — spread/area gates with hysteresis, teleport rejection,<br>velocity-adaptive smoothing, dropout hold, presence fade).
Composite — the AI video is revealed through the tracked quad with<br>the dashed marching-ants outline and pulsing corner dots.
Export — the result records to a downloadable video — MP4 where the<br>browser supports recording it (Safari, newer Chrome), otherwise .webm<br>(convert with ffmpeg -i finger-frame-ai.webm -c:v libx264 out.mp4).
Bring your own key
The AI step uses your own Gemini API key,<br>entered in the app. It stays in your browser (localStorage only if you check<br>"remember") and is sent only to Google's API. Generation is billed per<br>video and takes a few minutes. Keep clips under ~15MB (a few seconds of<br>720p — any common format: mp4, mov, webm); larger files exceed the inline<br>upload limit. No key? The placeholder style button runs the full<br>track-composite-export pipeline with a hue-shifted stand-in so you can try<br>everything for free.
Run locally
Any static server works:
python3 -m http.server 8124
Then open http://localhost:8124. A ?src= query param loads a video<br>from the server directory (dev convenience).
CLI alternative (Python)
The same pipeline as offline scripts — useful for batch work or<br>frame-accurate H.264 output:
python3 -m venv .venv<br>.venv/bin/pip install -r requirements.txt
export GEMINI_API_KEY=...<br>.venv/bin/python stylize.py input.mp4 -o stylized.mp4 # AI restyle<br>.venv/bin/python composite.py input.mp4 stylized.mp4 -o final.mp4
Any input format ffmpeg/OpenCV can read works (mp4, mov, webm, …).<br>composite.py needs ffmpeg on PATH, outputs H.264 MP4, and carries over<br>the original audio track when present.
Notes
Input/output media are gitignored — personal footage stays local. The<br>only committed media is the sample under examples/.
About<br>No description, website, or topics provided.<br>Resources<br>Readme<br>Activity<br>Stars<br>312 stars<br>Watchers<br>0 watching<br>Forks<br>28 forks<br>Report...