I set out to build a teleportation mirror for iOS. I left with a state-of-the-art AI model running on the iPhone. | by Nadav Cohen | Jun, 2026 | MediumSitemapOpen in appSign up<br>Sign in

Medium Logo

Get app<br>Write

Search

Sign up<br>Sign in

I set out to build a teleportation mirror for iOS. I left with a state-of-the-art AI model running on the iPhone.

Nadav Cohen

10 min read·<br>3 hours ago

Listen

Share

I tried to build the impossible thing, hit a wall made of physics, and walked out with a new understanding of how powerful — and underutilized — the phone in your pocket really is.

What I was chasing: walk through the living room, blink to Paris, blink back. This is a pre-visualization of the goal — the rest of this piece is the real story of how close I got, and the wall I hit.

The idea<br>I have a weakness for problems everyone else has already filed under “too hard.” This one had been circling in my head for months: what if the selfie camera was a portal ? Not a flat Zoom background pasted behind your head — actually you, standing somewhere else. Step sideways and the Paris street behind you shifts with real depth. Tilt the phone and the light of that place slides across your face. It shouldn’t feel like a filter; it should feel like teleportation. You hit record and walk away with a clip people can’t quite explain.<br>“Put me anywhere” isn’t new — but it’s always the same recipe: upload a photo to a cloud model, wait, get a video back. Never live. Never truly yours. Never running on the device in your hand. The instant, embodied, on-device version simply didn’t exist — and the second I understood why (because it’s legitimately hard) I knew I’d have to be the one to try it. That’s the fun of it, isn’t it: walking straight at the thing that’s supposed to be impossible.<br>So I broke “teleportation mirror” into the three problems it actually is:<br>Separate me from my real room — believably. Not a rough cutout fringed with my messy bedroom. A clean, film-grade matte, every single frame, on a phone.<br>Drop me into a beautiful place I can move through. A real environment with depth, rendered from wherever I move the phone, so leaning and stepping produce true parallax — not a sticker pinned at infinity.<br>Relight me so I belong there. A face lit by a bedroom bulb and pasted onto a sunset looks exactly like that — pasted. Presence is lighting. The place has to land on my skin.<br>Nail all three, simultaneously, at ~30 fps, on an iPhone, and you have magic. Here’s how far I got on each — and where physics finally drew the line.

Part 1 — Separating me from the world<br>Everything hangs on the cutout. Get the edge around my hair wrong and the illusion dies in the first frame — the eye is merciless about edges. I didn’t just need a fast matte; I needed a genuinely beautiful one, hair and fingertips and all, running live.<br>I reached for the free option first: Apple’s built-in person segmentation (`VNGeneratePersonSegmentationRequest` for the swift folks). On `.fast` it runs real-time, but the edges are mush. Crank it to `.accurate` and it’s not real-time anymore — and still smears the fine hair detail that sells the whole effect. I worked through the other usual suspects too, and every real-time matting model I tried made the same bargain I refused to take: fast and ugly, or pretty and slow.<br>The one thing that actually looked right was MatAnyone2 (CVPR 2026) — studio-grade edges, rock-steady frame to frame. Naturally, it’s the least convenient option imaginable: a research PyTorch model with no mobile export, and it’s recurrent (it carries a memory of the subject across frames), which is exactly the shape phone accelerators choke on.<br>So I ported it to MLX , Apple’s on-device ML framework, to run it on the iPhone’s GPU. It ran — and it wasn’t fast enough. MLX drives a single Metal queue that would end up wrestling my renderer for the same GPU, and the throughput just wasn’t there. I’ll be honest: this is where I nearly walked away from the entire project. The only matte good enough to sell the magic was too heavy to run live, and I’d already burned through the alternatives.<br>Then I tried the option that sounded the least promising: the Apple Neural Engine — a separate chip from the GPU, purpose-built for exactly this kind of work, and infamous for refusing to run anything that isn’t shaped just so. Coercing a recurrent research model to actually stay resident on the ANE was real surgery (the full gory details are at the end). But it worked: a studio-grade matte at a stable 30 fps, on a different piece of silicon than my renderer — which meant the GPU stayed completely free for the world behind me. That one result is what resurrected the project, and it’s what I ended up shipping. With the cutout finally solved, I went after problem two.

Left, the raw selfie frame; right, the same frame after MatAnyone2 on the Neural Engine. Same instant, background gone — down to individual strands of hair. This is the piece everything else depended...