Making KIO copy many files fast - KDE BlogsMaking KIO copy many files fast<br>Tuesday, 28 July 2026<br> | <br>Méven Car<br>Tags:<br>KIO<br>performance<br>Dolphin<br>#Frameworks
There is a bug in KDE's bug tracker that is almost as old as some of our<br>contributors: bug 342056,<br>"Ridiculously slow file copy (multiple small files)", reported by Alexander<br>Nestorov back in 2014. The report is blunt: copying a 15 GB folder of roughly 3<br>million small files took 5 to 10 hours in KDE, versus about 20 minutes with<br>rsync. One commenter summed up the mood:<br>I generally use cp and rsync instead of dolphin for anything more than a few files.
That bug has been stuck in my head for a while, so this post is about finally<br>fixing it. I have been working on KIO's copy path, and I want to walk<br>through where the time was going, some history that explains why, and show<br>numbers across KIO versions including a plain cp for reference.<br>At with any optimization, let's measure first:
Where a KIO 6.28 copy of 5000 small files spends its blocking time, and there<br>is no single hotspot. Each file blocks in a burst of short syscalls: reading the<br>mount table (/proc/self/mountinfo), opening the source and destination, and<br>round-tripping a command to the worker over an internal socket. The socket<br>round-trips, the send and receive plus the waits for each reply, are the frames<br>highlighted in red, about 15% of the blocking here, spread as thin slivers<br>across both threads because every one of the thousands of files pays them. The<br>frames in green are the real filesystem work the copy cannot avoid: the statx<br>and openat calls, copy_file_range moving the bytes, and the ext4 metadata<br>updates beneath them. That per-file storm, not any single call, is what the rest<br>of this post discusses.<br>(Off-CPU flame graph from sched:sched_switch, weighted by number of blocking<br>switches; click to open the full SVG.)<br>A little history<br>KIO is the layer behind almost every file operation in KDE software, from Dolphin's copy<br>and paste to the network transparency that lets you open sftp:// or smb://<br>URLs as if they were local. Its design goes back to the KDE 2 days, around the<br>year 2000. Back then the answer to "how do I do I/O without freezing the UI" was<br>not a thread but a separate process: a worker (we used to call them kioslaves)<br>is launched for a protocol and talks to the application over a socket. This<br>predates usable threading on Linux (Native POSIX Thread<br>Library only landed<br>in Linux 2.6 in 2003), so processes plus socket IPC was the portable, robust<br>choice, and it still is for network protocols today: if an smb:// worker<br>crashes, your file manager does not go down with it.<br>The flip side is cost. Every request and its reply are serialized and sent over<br>that socket, with an event-loop hop on each side. For file:// that overhead<br>buys you nothing, because there is no untrusted network on the other end, just<br>the local disk.<br>In 2022 David Faure changed that: Implement running KIO workers in-process<br>using a thread landed in Frameworks 5.95. Since then the file worker runs on a thread inside<br>the application instead of as a separate process (other protocols stay<br>out-of-process for robustness). That removed process launch and context-switch<br>cost.<br>No socket between the thread and the app (6.29)<br>There was still a catch that had been hiding in plain sight: even in-process,<br>the worker thread and the application talked to each other over a socketpair,<br>serializing every command as if they were separate processes. Once the thread<br>was used, that socket was pure overhead.<br>That's the red highlight in the first flamegraph.<br>So in-process workers now use a real in-memory transport instead of a loopback<br>socket. A new ThreadConnectionBackend handles commands, and for reads the actual<br>data buffer, straight to the peer thread, with no serialization and no syscall,<br>which makes in-process file reads zero-copy. This is the biggest single jump in<br>the numbers below, and it has landed on master for 6.29<br>(kio!2279).<br>Next: batching the copy and the stat (under review)<br>The changes above make each command cheap. The remaining cost is that there are<br>still so many of them: copying N files is N separate file_copy jobs, each one<br>going through the job scheduler and taking its own round-trip to the worker. For<br>many small files that fixed per-file cost, not the data, is what dominates.<br>CopyJob can dispatch a whole batch of plain local-to-local files to the worker<br>in one command, and the worker copies them (using copy_file_range,<br>and reflink on filesystems like Btrfs or XFS that support it). Each file still<br>gets its progress signal, its byte accounting, its undo entry, and anything<br>the worker cannot do blindly — an existing destination, an unreadable source, a<br>rename or skip decision — is handed back to the normal per-file path. The gate is<br>conservative on purpose so a hung mount can never freeze the batch<br>(kio!2282).
The same copy on 6.29 with batch-copy: the red socket round-trips are gone. Two<br>things removed them: the...