Stream Plumbing in Embedded Systems · GitHub

/" data-turbo-transient="true" />

Skip to content

-->

Search Gists

Search Gists

Sign in

Sign up

You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

{{ message }}

Instantly share code, notes, and snippets.

mickjc750/embedded_stream_plumbing.md

Created<br>May 28, 2026 11:08

Show Gist options

Download ZIP

Star

(0)

You must be signed in to star a gist

Fork

(0)

You must be signed in to fork a gist

Embed

Select an option

Embed<br>Embed this gist in your website.

Share<br>Copy sharable link for this gist.

Clone via HTTPS<br>Clone using the web URL.

No results found

Learn more about clone URLs

Clone this repository at &lt;script src=&quot;https://gist.github.com/mickjc750/556fa3b6bb62f383ff2b4b6b99d84faa.js&quot;&gt;&lt;/script&gt;

" readonly="readonly" data-autoselect="true" data-target="primer-text-field.inputElement " aria-describedby="validation-92b2c05c-09c8-4936-ad15-b2eb08272984" class="form-control FormControl-monospace FormControl-input FormControl-small rounded-left-0 rounded-right-0 border-right-0" type="text" name="gist-share-url-sized-down" />

Save mickjc750/556fa3b6bb62f383ff2b4b6b99d84faa to your computer and use it in GitHub Desktop.

Embed

Select an option

Embed<br>Embed this gist in your website.

Share<br>Copy sharable link for this gist.

Clone via HTTPS<br>Clone using the web URL.

No results found

Learn more about clone URLs

Clone this repository at &lt;script src=&quot;https://gist.github.com/mickjc750/556fa3b6bb62f383ff2b4b6b99d84faa.js&quot;&gt;&lt;/script&gt;

" readonly="readonly" data-autoselect="true" data-target="primer-text-field.inputElement " aria-describedby="validation-3af49e08-de50-4c26-9b42-c7f54830734b" class="form-control FormControl-monospace FormControl-input FormControl-small rounded-left-0 rounded-right-0 border-right-0" type="text" name="gist-share-url-original" />

Save mickjc750/556fa3b6bb62f383ff2b4b6b99d84faa to your computer and use it in GitHub Desktop.

Download ZIP

Stream Plumbing in Embedded Systems

Raw

embedded_stream_plumbing.md

Stream Plumbing in Embedded Systems

Introduction

Desktop operating systems provide powerful abstractions for moving data between software components. In POSIX systems, this is commonly achieved using file descriptors together with read() and write() operations.

For example:

a generator writes data to a file descriptor,

while a consumer reads data from it.

These operations typically execute concurrently on different threads, while the kernel buffers and coordinates the transfer.

Embedded systems are often very different:

there may be no operating system,

no file descriptor abstraction,

limited RAM,

and insufficient resources for multithreaded buffering.

As a result, stream plumbing in embedded systems becomes a more explicit architectural problem.

This document explores a practical approach to stream interfaces for embedded platforms.

Basic Stream Interfaces

POSIX-style interfaces typically use a file descriptor as context:

ssize_t read(int fd, void *buf, size_t count);<br>ssize_t write(int fd, const void *buf, size_t count);

In embedded systems, file descriptors may not exist. A common alternative is to replace the file descriptor with an opaque context pointer:

ssize_t read(void *ctx, void *buf, size_t count);<br>ssize_t write(void *ctx, const void *buf, size_t count);

This approach preserves much of the simplicity and familiarity of POSIX APIs.

It also allows easy desktop shimming during testing. For example, a POSIX file descriptor can simply be stored in an integer and passed via ctx.

One drawback is that both ctx and buf are pointer types. Accidentally swapping them may not generate a compiler warning:

read(buf, ctx, count);

This requires careful adherence to argument ordering conventions.

Alternative approaches include opaque typed handles or wrapping the operations in a stream object structure.

Push vs Pull Transfers

Without threads or kernel-managed buffering, embedded transfers generally execute from one side or the other.

Two models emerge:

Generator-Write (Push)

The generator executes the transfer by writing data into the consumer.

ssize_t write(void *ctx, const void *buf, size_t count);

The consumer accepts some amount of data up to count.

This model suits unsolicited or asynchronous data sources, such as:

UART receive streams,

ADC sampling,

interrupt-driven events.

Consumer-Read (Pull)

The consumer executes the transfer by requesting data from the generator.

ssize_t read(void *ctx, void *buf, size_t count);

The generator provides some amount of data up to count.

This model suits solicited data sources, such as:

sensor polling,

flash memory reads,

protocol responses,

Stream Direction Mismatch

When plumbing modules together, it is common...