Streaming Consumption API - satd Operator Manual

Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Auto

Light

Rust

Coal

Navy

Ayu

satd Operator Manual

Streaming Consumption API

The Streaming Consumption API is satd's push-based surface for downstream<br>consumers — wallets, Lightning nodes, exchanges, watchtowers, explorers, and<br>other L2 projects. It pairs a real-time event firehose (blocks and mempool<br>transitions) with live, cursor-resumable watch subscriptions keyed on<br>outpoints, scripts, descriptors, and transaction ids.

It exists because the incumbent ways to consume a node each leave the same three<br>gaps — descriptor lifecycle , outpoint-level subscriptions , and<br>cursor-based event replay — that every serious consumer ends up reinventing.<br>satd serves all three natively, in-process, as consensus ground truth rather<br>than reconstructed from a ZMQ side-channel.

This chapter is the integrator guide. The authoritative, wire-level protocol<br>specification — the satd.events.v1 protobuf, frame formats, and cursor<br>semantics — is<br>docs/api/streaming.md.<br>For a step-by-step onramp before this reference, start with<br>Getting Started: Consuming Events.

The base primitive: outpoint subscription

The key generalization is that outpoint subscription is the base primitive.<br>Lightning channel-close detection, watchtower triggers, exchange deposit<br>confirmation, and theft monitoring all reduce to "tell me when this outpoint is<br>spent." Address-watching is outpoint-watching with a derivation rule layered on<br>top. The API builds down to outpoints and layers scripts, descriptors, and<br>transaction-id watches on as conveniences over the same matcher.

Transports

One schema (the satd.events.v1 protobuf definition is the source of truth),<br>three transports:

gRPC native (satd.events.v1, tonic) — the primary transport for<br>programmatic consumers. A server-streaming Subscribe (firehose) and a<br>bidirectional Watch (firehose + managed watch-set).

JSON-over-WebSocket (GET /ws) — a hand-mapped JSON rendering of the<br>same tagged-unions, with a client→server control channel mirroring Watch.

Server-Sent Events (GET /sse) — a read-only JSON firehose (no control<br>channel) for browser / curl consumers.

A Core-compatible ZMQ PUB sink remains for legacy parity; it carries the<br>firehose bodies only (not per-subscriber watch matches) and uses Core's<br>per-topic sequence numbers.

WebSocket and SSE bind a dedicated --streamws port rather than upgrading on<br>the Core-compat JSON-RPC port — keeping the differentiated stream a distinct<br>service on a distinct port. Every streaming listener (--streamws and the gRPC<br>NodeEventStream) runs on the isolated API tokio runtime (--api-threads),<br>never the core block-connecting runtime: a flood of streaming clients can never<br>contend with the threads that connect blocks and accept mempool transactions. See<br>API Scaling & Runtimes for the runtime split, the admission<br>caps, and how to scale beyond one node.

Subscriptions and watch-sets

The gRPC service offers the server-streaming Subscribe (the firehose, with<br>cursor replay) plus a bidirectional Watch whose client→server messages are<br>a tagged union — SetCursor, SetCategories, Add/Remove for scripts,<br>outpoints, transactions, script-prefixes, and descriptors, and SetWatchSet (an<br>atomic whole-set replace: send the complete desired watch-set in one message and<br>the server reconciles it under its lock by effective coverage, replying with a<br>deterministic WatchSetResult). New subscription kinds slot in additively<br>without protocol breakage.

Match events delivered on the per-subscriber Watch channel include:

OutpointSpent — an outpoint was spent (in mempool or a connected block).

ScriptMatched — a script was funded or spent (both sides). For a<br>descriptor-derived script it carries descriptor_matches: which descriptor +<br>the exact (branch, derivation_index) it was derived at (empty for a<br>directly-watched script), so a multi-descriptor consumer can route a hit<br>without its own reverse index. It also carries the matched value<br>(amount/has_amount, at parity with SpentPrevout) so an exact-script<br>consumer can skip the per-match getrawtransaction enrichment call, and —<br>opt-in per connection via SetWatchOptions{include_raw_tx}, off by default —<br>the full consensus-serialized matching transaction inline (raw_tx).

TxidMatched / TxidReplaced / TxidEvicted / TxidUnconfirmed /<br>TxidDepthReached / TxidFinalized — transaction lifecycle and<br>confirmation-depth alarms.

PrefixMatched — a privacy-preserving script-prefix match.

The matcher is decoupled from the consensus path : a dedicated task subscribes<br>to the existing chain/mempool broadcasts and re-reads blocks and accepted<br>transactions the node already holds, then scans the watch-set and delivers<br>matches. A node with no subscribers pays nothing (gated behind a lock-free<br>has_watchers() check), and a slow client's matches are...