[2605.18697] PopPy: Opportunistically Exploiting Parallelism in Python Compound AI Applications

-->

Computer Science > Distributed, Parallel, and Cluster Computing

arXiv:2605.18697 (cs)

[Submitted on 18 May 2026]

Title:PopPy: Opportunistically Exploiting Parallelism in Python Compound AI Applications

Authors:Stephen Mell, David Mell, Konstantinos Kallas, Steve Zdancewic, Osbert Bastani<br>View a PDF of the paper titled PopPy: Opportunistically Exploiting Parallelism in Python Compound AI Applications, by Stephen Mell and 4 other authors

View PDF<br>HTML (experimental)

Abstract:Compound AI applications, which compose calls to ML models using a general-purpose programming language like Python, are widely used for a variety of user-facing tasks, from software engineering to enterprise automation, making their end-to-end latency a critical bottleneck. In contrast to traditional applications, execution time is dominated by the external components, which cannot be handled by traditional language optimization systems, like optimizing compilers.

To address this problem, we develop PopPy, a system that can uncover parallelization opportunities in Python applications that invoke these heavy external components, including those used in compound AI applications. PopPy supports a very expressive fragment of Python and requires minimal developer input to uncover parallelism. It combines an ahead-of-time compiler with a runtime, addressing three key challenges in extracting parallelism from Python applications: language complexity, dynamic dispatch, and variable mutation. On a set of real-world compound AI applications, PopPy achieves up to $6.4\times$ speedups in end-to-end execution time compared to standard Python execution while preserving the sequential program semantics.

Subjects:

Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Programming Languages (cs.PL)

Cite as:<br>arXiv:2605.18697 [cs.DC]

(or<br>arXiv:2605.18697v1 [cs.DC] for this version)

https://doi.org/10.48550/arXiv.2605.18697

Focus to learn more

arXiv-issued DOI via DataCite

Submission history<br>From: Stephen Mell [view email]<br>[v1]<br>Mon, 18 May 2026 17:33:50 UTC (103 KB)

Full-text links:<br>Access Paper:

View a PDF of the paper titled PopPy: Opportunistically Exploiting Parallelism in Python Compound AI Applications, by Stephen Mell and 4 other authors<br>View PDF<br>HTML (experimental)<br>TeX Source

view license

Current browse context:

cs.DC

next >

new<br>recent<br>| 2026-05

Change to browse by:

cs<br>cs.AI<br>cs.PL

References & Citations

NASA ADS<br>Google Scholar

Semantic Scholar

export BibTeX citation<br>Loading...

BibTeX formatted citation

&times;

loading...

Data provided by:

Bookmark

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Which authors of this paper are endorsers? |<br>Disable MathJax (What is MathJax?)