Video editing with p5.js - Dave Pagurek
Video editing with p5.js
August 1, 2026
I recently finished an animation called Your Body Is A Machine:
This one is notable for me because all the visuals are p5.js sketches! But it's not that I made them all separately, screen recorded them, and then edited them together separetely. That's more or less what I did in You Probably Can't Draw A Horse Without Reference, which also includes a few p5.js scenes, such as around 2:21 where there's a physics simulation. No: that would have been to difficult here, where the sketches are synchronized to specific parts of the music, and also where there was not as clear of a deliverable per scene, and I needed to work more experimentally and iteratively.<br>I mostly write about open source work for p5.js here, but in terms of hours I spend per day, the majority go to my day job at Butter, a modular video editor. The idea there is that all the components you can use are code, so it's incredibly extensible, and no asset is ever fully "dead" or locked in, as the code can easily expose controls for you. It's still very related to p5.js: right now, those programmable blocks are all written as p5.js sketches! Mostly we've been using that to easily create new customizable blocks for users of Butter, but for a while, it's been clear to me that p5.js artists could be using this to create something new. There have been a lot of great tools popping up using creative code as a performance medium, but I see Butter doing something similar for more planned productions that need nonlinear editing.<br>So I finally got the opportunity to make something like that recently thanks to Butter doing a bigger push for features for coders, the addition of keyframing of sketch inputs, us needing to test slightly longer projects, and just having a compatible video idea. So here we are, testing this all out with some mild health/body horror!<br>I wanted to break down the process a bit, in case anyone else sees something in here that they'd like to try themselves!<br>Structuring a project<br>Butter should look familiar to anyone who's used any other video editor: there's a timeline, with pills on it representing each thing in the project. You can add new things from some menus on the left. The first menu item is the Build tab, which lets you add a new code component. This pops open a code editor!<br>The Butter timeline and code IDE
So each thing in the timeline for my project is an editable p5.js sketch. If you createCanvas(windowWidth, windowHeight), then Butter will let you resize it on canvas freely. (Fixed-size canvases can still be scaled uniformly.)
You can also stack multiple sketches! If each sketch draws an object, you can position the different objects into a scene. But you can also stack full-screen sketches as filters. In this project, most of my sketches are black and white, and then I reuse a sketch that goes on top of all of those that applies a slight blur and applies a gradient map to recolor the content.
The code for the above is a pretty simple p5.strands shader:<br>let remap
let blur<br>let c1, c2, c3, c4
async function setup() {<br>createCanvas(windowWidth, windowHeight, WEBGL);
blur = createSlider(0, 40, 10, 0.1)<br>c1 = createColorPicker('#000')<br>c2 = createColorPicker('#B11')<br>c3 = createColorPicker('#FB1')<br>c4 = createColorPicker('#fff')
remap = buildFilterShader(() => {<br>let c1v = uniformVec4(color(c1.value()))<br>let c2v = uniformVec4(color(c2.value()))<br>let c3v = uniformVec4(color(c3.value()))<br>let c4v = uniformVec4(color(c4.value()))<br>filterColor.begin()<br>let lum = (rgb) => dot(rgb, [0.2126, 0.7152, 0.0722])<br>let v = getTexture(filterColor.canvasContent, filterColor.texCoord)<br>let progress = lum(v.rgb)<br>let out = c1v<br>if (progress The way that it can apply this filter to the contents of another sketch rather than completely drawing its visuals from scratch is through the use of the Butter-specific function drawParentToCanvas(), which draws everything below the sketch on the timeline onto the sketch's canvas where it can be processed further.<br>Note also that regular p5.js sliders and inputs get turned into editable fields when you select the sketch! You can also add keyframes to most fields if you want a slider value to change over time. If you use orbitControl(), that's keyframeable too, and I used that a bit to animate camera movements in a few scenes.<br>If your sketch draws fully from scratch every frame based only on time variables like frameCount or millis(), you can seek anywhere instantly. But even for sketches that update incrementally from frame to frame, such as a physics simulation, you can still scrub back through time, and it will load for a moment and then show you the right frame.
Scene breakdowns<br>That's what you can do in Butter generally, but now let's look at a few scenes, which are still just standalone p5.js sketches.<br>Folds<br>This one is a single filter shader, using some domain warping on noise to produce the folds. A bulge is applied by pushing...