Automatic1111 for Apple metal, 40% speed up sd1.5

From 8–10 Seconds to 3–7: Teaching Automatic1111 to Speak Metal on an M3 Pro

I use Draw Things a lot on Apple hardware, and one thing has always bothered me about Automatic1111: it feels slower than it should. Not unusably slow. Just slow enough that you notice it. On my M3 Pro, a short five-step DPM++ SDE generation in Automatic1111 was typically landing somewhere around 8–10 seconds. Draw Things had already shown me that Stable Diffusion on Apple Silicon could feel much more immediate than that. So I wanted to see how much of that gap was actually necessary. GitHub - dmikey/stable-diffusion-webui-metal: a fine tuned automatic1111 for Apple Silicon. a fine tuned automatic1111 for Apple Silicon. Contribute to dmikey/stable-diffusion-webui-metal development by creating an account on GitHub. GitHubdmikey

There was one important constraint: I did not want to replace Automatic1111. I wanted the same WebUI, checkpoints, LoRAs, samplers, extensions, API, prompt syntax, and general workflow. I wasn't interested in converting everything to Core ML and building another inference engine around it. The goal was much narrower: How fast can Automatic1111 get if we make the parts that matter behave more like native Apple software? The answer, at least for the workloads I'm running, is quite a bit faster. The same class of generation that was taking roughly 8–10 seconds on my M3 Pro is now generally landing between 3 and 7 seconds. Those are observed ranges across my current workloads, not a controlled benchmark claiming a universal 2x improvement. There is also an important distinction between the runtime improvements and NGMS, which actually reduces the amount of guidance work being performed. Still, the difference in actual use is substantial. More interesting than the final number, though, was what it took to get there. It wasn't one optimization. Start with the workload, not the benchmark The workload I cared about was pretty specific: Stable Diffusion 1.x DPM++ SDE Karras 5 steps CFG around 1.15 384×640 and 512×512 FP16 UNet on MPS FP32 VAE by default That specificity matters. Early on, DPM++ 2M looked like an easy way to shave off time. It was faster, but it didn't produce the result I wanted from the short schedule. That isn't an optimization. It's a different workload. This became the rule for basically everything that followed: if an optimization looks great in isolation but doesn't make the actual generation faster while preserving the result I'm trying to produce, it doesn't count. Metal Flash Attention, selectively Attention was the obvious place to start. PyTorch's MPS backend has gotten substantially better, but there are still Stable Diffusion attention shapes where going directly to Metal makes sense. The mistake would have been treating a custom Metal implementation as universally faster. It isn't. Instead, I added a Metal Flash Attention path specifically for the SD 1.x shapes where it actually won in testing. The router looks conceptually like this: if inference and fp16_mps and query_tokens >= 192 and head_dim in (40, 80, 160): return metal_flash_attention(q, k, v)

return pytorch_sdpa(q, k, v) There are additional checks around masks, training, dropout, tensor layout, grouped-query attention, and supported types, but that's the basic idea. Metal is not the default because Metal sounds faster. It gets the operation when we've measured that shape and it deserves it. Everything else goes back through PyTorch. That fallback is important. Automatic1111 supports far more configurations than my five-step SD 1.x workflow. I didn't want a faster fork that only worked if nobody touched anything. The kernel wasn't the whole problem Getting attention into Metal helped, but it exposed something more interesting. The native extension was committing the MPS command buffer after every attention call. Stable Diffusion calls attention over and over inside every UNet evaluation. With a short five-step generation, repeatedly submitting tiny chunks of work starts becoming a meaningful part of the total runtime. So instead of treating the Metal kernel like its own little application, I integrated it into PyTorch's current MPS stream. The extension ends PyTorch's current kernel coalescing, encodes the Metal Flash Attention operation into the current command buffer, and then lets the rest of the PyTorch MPS work continue from there. The explicit commit after every attention call went away. This ended up being one of the more important lessons from the entire project. The fastest kernel still loses if you submit the command buffer after every call. At these generation times, overhead matters. You're no longer just optimizing how quickly the GPU can multiply matrices. You're optimizing how often Python, PyTorch, MPSGraph, and Metal have to coordinate with each other. There was also a wonderfully obvious reminder not to trust the timer: one of the early versions produced a green image. It was fast. It was also green. The...

Automatic1111 for Apple metal, 40% speed up sd1.5

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

NHS apologises and admits Palantir have access to identifiable patient data

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills