Plugins case study: Pluggy - Eli Bendersky's website

Toggle navigation

Eli Bendersky's website

About

Projects

Archives

Recently I came upon Pluggy,<br>a Python library for developing plugin systems. It was originally developed<br>as part of the pytest project - known for its rich plugin ecosystem - and<br>later extracted into a standalone library. You're supposed to reach out for<br>Pluggy if you want to add a plugin system to your tool or library and want<br>to use something proven rather than rolling your own.

In this post I will share some notes on how Pluggy works, and will<br>then review how it aligns with the<br>fundamental concepts of plugin infrastructures.

Using Pluggy

Pluggy is built around the concept of hooks: functions that host<br>applications or tools (from here on, just "hosts") expose and plugins implement.<br>A host exposes hooks by using<br>a decorator returned from pluggy.HookspecMarker and a plugin implements this<br>hook using a decorator returned from pluggy.HookimplMarker.

Pluggy's documentation explains<br>this fairly well; in this post, I'll show how to implement the htmlize tool<br>with some plugins, introduced in the original article in my plugin series.

As a reminder, htmlize is a toy tool that takes markup notation similar to<br>reStructuredText, and converts it to to HTML. It supports plugins to handle<br>custom "roles" like:

some text :role:`customized text` and more text

As well as plugins that do arbitrary processing on the entire text.

Defining hooks

Out host defines two hooks:

import pluggy

hookspec = pluggy.HookspecMarker("htmlize")

@hookspec(firstresult=True)<br>def htmlize_role_handler(role_name):<br>"""Return a function accepting role contents.

The function will be called with a single argument - the role contents, and<br>should return what the role gets replaced with.<br>"""<br>pass

@hookspec<br>def htmlize_contents(post, db):<br>"""Return a function accepting full document contents.

The function will be called with a single argument - the document contents<br>(after paragraph splitting and role processing), and should return the<br>transformed contents.<br>"""<br>pass

A hook is created by calling HookspecMarker with the project's name. This<br>project name has to match between the host and its plugins. Pluggy is permissive<br>about what hooks accept as parameters and what they return; for maximal<br>flexibility and to stay true to the original htmlize example, our hooks<br>return functions.

To accompany this HookspecMarker, the host also defines a HookimplMarker with<br>the same name:

hookimpl = pluggy.HookimplMarker("htmlize")

This is used by plugins to attach to hooks when they're loaded.

Loading plugins in the host

The host's main function loads plugins at startup as follows:

pm = pluggy.PluginManager("htmlize")<br>pm.add_hookspecs(hookspecs)<br>pm.load_setuptools_entrypoints("htmlize")

hookspecs is our Python module containing the hooks shown above.<br>load_setuptools_entrypoints is Pluggy's helper for loading plugins that<br>were pip-installed into the same environment and registered as<br>setuptools entry points.<br>It's a way to signal - in one's setup.py or pyproject.toml file - some<br>metadata that projects can review at runtime. In our project, the plugins<br>register themselves with this section in the pyproject.toml file:

[project.entry-points.htmlize]<br>tt = "tt"

This says "for entry point htmlize, define a new entry named tt".<br>Pluggy's load_setuptools_entrypoints then uses importlib.metadata<br>to access this information.

Note that Pluggy doesn't require using this mechanism. Hosts can implement any<br>plugin discovery method they want, and add plugins directly to their<br>PluginManager with the register method. But this is the mechanism used<br>for pytest and many other projects; it makes it very easy to<br>automatically discover and register plugins that are installed with pip and<br>equivalent tools.

Invoking plugins

Once PluginManager loads the plugins, invoking them is straightforward;<br>here's how htmlize invokes the contents hooks [1]:

# Build full contents back again, and ask plugins to act on<br># contents.<br>contents = ''.join(parts)<br>for handler in plugin_manager.hook.htmlize_contents(post=post, db=db):<br>contents = handler(contents)<br>return contents

Generally, hook invocations return a list of all the hooks attached to by<br>different plugins (a single host application can have multiple plugins installed<br>and attaching to the same hook). When the host invokes the hook as shown above,<br>the default order is LIFO, but plugins can affect this with<br>hook options<br>like tryfirst and trylast.

Implementing hooks in plugins

Here's our entire narcissist plugin that's attaching to the contents hook:

import htmlize

@htmlize.hookimpl<br>def htmlize_contents(post, db):<br>repl = f'I ({post.author})'

def hook(contents):<br>return re.sub(r'\bI\b', repl, contents)

return hook

Some notes:

It expects htmlize to be installed; as discussed previously, we rely on<br>Pluggy's default install-based approach where both the host and plugins are<br>installed into the same Python...