My Accessibility Stack and the future on Wayland – Insane Rambles About Technology

Skip to content

My Accessibility Stack and the future on Wayland

This is an article quite some time in the making. I’ve written 3 or 4 drafts of it over the last 4 months, looking for just the right thing to say. After I wrote the initial draft of this one, it sat in the drafts for another 2 months before I really finished it. In the end, I’ve decided that being straightforward is the best way to go, so here it is:

As the Linux Desktop transitions to a Wayland-only future, I will be locked out of my computer, as the accessibility software I rely on is left behind.

The desktop environment I use, KDE Plasma, has announced that in early 2027, X11 support will be removed from the system. That means in about roughly 9 months, I will no longer be welcome on that desktop environment, being forced to cling to an older version or switch to a more niche environment that still supports it.

Why?

The Wayland desktop has been making great strides in accessibility recently, right? Well, there’s a subset of accessibility that absolutely no one is talking about, and that’s what I’m here to fix today:

Input devices.

Most of the discussion about accessibility refers to output, for users who have limited vision, or are blind. The series of articles written by Fireborn last year discuss the myriad problems that exist while trying to use the Linux desktop while blind, and likewise GNOME has been devoting their attention to, for instance, supporting AccessKit in their applications, which helps screen readers such as Orca render its contents via text to speech.

But accessibility cuts both ways, and it’s equally valid for people to have trouble conveying input to their systems. Such is the case for me, as last year, after a gradual but steady period of decline I was diagnosed with Ehlers-Danlos Syndrome, a musculoskeletal genetic defect which wreaks all sorts of havoc on your body. An earlier draft of this article included my whole, nasty journey of getting diagnosed and treated for such a rare (and often misdiagnosed) condition, but it’s really not that important– What matters is that basically what it did to me in particular was destroy all those important little muscles in the wrist that let you flex your fingers, say, in order to use a keyboard or work a mouse.

Thanks to months of intensive physical therapy with a specialist in hypermobility disorders, I’ve regained partial use of my hands; depending on the day, I can get through maybe a few hours of typing with a specialized keyboard.

(As an aside, the fact that I’ve regained ANY use is something of a miracle, largely owing to good location relative to specialists, liberal medical leave policy in Massachusetts, and my undeniable position of privilege within the horrifically messed up U.S. healthcare system. There are many worlds in which I wind up permanently disabled and never even find out why.)

But regrowing a large set of small muscles that all withered away is terribly slow, very painful, and probably imperfect; I may never truly regain full use. The progress I’ve made doesn’t get me through a full workday, let alone weeks on end; I need another way to get through my life and career.

Enter Talon Voice.

Talon Voice

With possibly one of the most understated landing pages in existence, Talon doesn’t do much at first glance to convey that it’s probably the most powerful hands free input system ever created. Talon is a deeply, thoughtfully crafted core of a hyper-fast and accurate Speech-To-Text ML model, a bespoke scripting language, and Python, all working together in concert to enable nearly infinite extensibility for users to craft their own hands-free means of communication to their applications, either working with the application… Or against it. (That notion of "adversarial accessibility" is something that comes up quite frequently!)

The community series of scripts is the first thing to install to make Talon useful, and boy, is it a whopper. There’s tens of thousands of lines of code in here, all carefully hand-written by individuals to meet their specific needs and conglomerated into a whole so that others can benefit from their work.

With Talon, I can do things like:

Focus applications, saving me the mouse movement and clicks necessary to select them from the taskbar;

Write text, using Dictation Mode (most of this article was written with talon);

Interact with my browser, using the Rango extension for browsers, completely hands free (which happens to be faster than traditionally moving around with a mouse anyhow)

Write my own script to call out over D-Bus to an external speech to text program for times when I’m writing longer prose (Whisper-v3-large is a truly remarkable model, understanding things like proper nouns it’s never heard of before, though it isn’t quite fast)

Make a hissing noise to scroll, which is a motion that is persistently painful for me...