Webhooks Suck: Fixing the Problems with Consuming Webhooks | Svix Blog

AuthorsNameTom HacohenTwitter@TomHacohen<br>Svix is the enterprise ready webhooks sending service. With Svix, you can build a secure, reliable, and scalable webhook platform in minutes. Looking to send webhooks? Give it a try!

Webhooks are simple and ubiquitous, and that's both a weakness and a strength. It means that they are everyone's go-to, even for use-cases that they don't solve very well.<br>They have also traditionally been very difficult to consume due to ad-hoc and complex signature schemes, bad observability, and difficult setup.<br>This post is about many of the challenges with consuming webhooks, and what we are doing to solve them. I've been meaning to write this post for a while, and a recent post on HN was the push I needed to finally finish it.<br>Here are some of the common challenges. Some of them we've already solved, but some are still work in progress.<br>Webhook signature verification<br>One of the most annoying parts of consuming webhooks has traditionally been signature verification. Webhooks are unauthenticated HTTP calls and can therefore be faked and forged by attackers. That's why webhooks are most commonly signed in order to ensure authenticity and their timestamps verified in order to protect against replay attacks.<br>This used to be a hot mess (still is to an extent), where every provider would reinvent their own signature scheme. Some were good, most were bad, and many were very bad. Though even for the good ones, that often meant consumers had to understand, implement, and test a new signature scheme for each provider.<br>To solve this, we created Standard Webhooks, which includes a consistent signature scheme that can be used by everyone. Standard Webhooks includes a set of SDKs for signature verifications which senders and receivers can both use, as well as built-in support in Svix, ngrok, Kong, and many others making it mostly an issue of the past.<br>Webhook observability and diagnosing failures<br>When an API call fails the caller knows it fails. Their call fails, and they get a timeout error, an HTTP 500, or some other type of error. With webhooks that's not the case, as you often don't know you were supposed to receive an event when you receive one. Even if you monitor your infrastructure well, the failure may happen in an area you don't control, e.g. a networking issue or a DNS issue.<br>To solve this Svix provides full visibility into every webhook attempt, including status, failure reason, request content, response content, and other metadata. This gives people the ability to fully diagnose issues when they happen.<br>Svix also supports automatically sending emails and Slack messages to consumers when failures happen consistently. So they get an out-of-band message in addition to the failed webhooks themselves.<br>Managing webhook subscriptions without click ops<br>One of the things I find most annoying about webhooks is click ops. If you want to receive webhooks from a provider an admin almost always has to go to their website, login, navigate to the UI, and update the webhook subscription.<br>This happens completely disjointly from code and infrastructure deployment, and is especially annoying for teams that employ infrastructure-as-code.<br>Svix has always supported controlling webhooks subscriptions via an API. Though while the API works, it always felt a bit cumbersome, as it required writing custom code.<br>That's why we introduced Webhooks AutoConfig earlier this year. It makes updating webhooks subscriptions trivial, which makes consuming webhooks much easier.<br>Local development with webhooks<br>Local development is a common pain with webhooks. The problem is that webhooks are often sent from 3rd party SaaS services which can't access your local development machine.<br>To solve it, most people use some sort of an HTTP forwarding service such as ngrok or Svix Play to forward requests to a local endpoint. This works, but it suffers from a variety of issues.<br>The main problem is that setup is a pain. You need to run this additional CLI app, point it at the local server, and update the sender with the right credentials. This works until you turn your computer off and come back the day after to many failing events that are constantly retrying or a disabled endpoint you need to turn back on.<br>It's also a problem when you have multiple people sharing the same development SaaS service, making changes in parallel, and due to different times of having their laptops on, have completely different states. They also need to contact that service's admin every time they need to modify each developer's webhook subscription. This should probably also be addressed by more SaaS offering separate testing accounts, but that's a problem for another day.<br>Our solution to this is a combination of Webhooks AutoConfig and Polling Endpoints. Webhooks AutoConfig lets people automatically update their subscriptions from their code (never having to access the UI). Polling...