GNU Parallel<br>- GNU Project - Free Software Foundation

Skip to main text

Free Software Supporter:

JOIN THE FSF

GNU Operating System

Supported by the<br>Free Software Foundation

Site navigation<br>Skip

ABOUT GNU

PHILOSOPHY

LICENSES

EDUCATION

SOFTWARE

DISTROS

DOCS

MALWARE

HELP GNU

AUDIO & VIDEO

GNU ART

FUN

GNU'S WHO?

SOFTWARE DIRECTORY

HARDWARE

SITEMAP

GNU Parallel

GNU parallel is a shell tool for executing jobs in parallel using one<br>or more computers. A job can be a single command or a small script<br>that has to be run for each of the lines in the input. The typical<br>input is a list of files, a list of hosts, a list of users, a list of<br>URLs, or a list of tables. A job can also be a command that reads from<br>a pipe. GNU parallel can then split the input and pipe it into<br>commands in parallel.

For people who live life in the parallel lane

If you use xargs and tee today you will find GNU parallel very easy to<br>use as GNU parallel is written to have the same options as xargs. If<br>you write loops in shell, you will find GNU parallel may be able to<br>replace most of the loops and make them run faster by running several<br>jobs in parallel.

GNU parallel makes sure output from the commands is the same output as<br>you would get had you run the commands sequentially. This makes it<br>possible to use output from GNU parallel as input for other programs.

For each line of input GNU parallel will execute command with<br>the line as arguments. If no command is given, the line of input is<br>executed. Several lines will be run in parallel. GNU parallel can<br>often be used as a substitute for xargs or cat | bash .

Download

Stable source releases:

They are available on the main GNU server<br>(HTTPS,<br>HTTP,<br>FTP) and its<br>mirrors; please<br>use a mirror if possible.

Latest release

Official packages, for:

Android Termux

GNU Guix

Gentoo

FreeBSD

Fedora

CERN Centos 7

RHEL 9

Ubuntu Quantal (https://launchpad.net/ubuntu/quantal/i386/parallel/20120422-1).

Debian Jessie (https://packages.debian.org/jessie/parallel), and Sid (http://packages.debian.org/sid/parallel).

Arch Linux (http://www.archlinux.org/packages/community/any/parallel/).

OpenSUSE (https://software.opensuse.org/package/gnu_parallel).

--><br>Alpine Linux

OpenBSD

NetBSD

DragonFlyBSD

OpenMandriva

Alt Linux

Mageia Linux

Amazon Linux

MacOS FINK

Zero install

Solaris 11.2

Void Linux

Slackware

PCLinuxOS

Solus

various systems, at Anaconda Cloud

MSYS2 (Microsoft Windows)

Community maintained packages:

untested .deb/.rpm packages for Debian, CentOS, Arch, Fedora, Mandriva, RedHat,<br>OpenSUSE, and xUbuntu at build.opensuse.org,

packages for Solaris at Sun OpenCSW,

.rpm packages for AIX at Perzl.org.

Unsupported platforms:

Users have reported these working. Unsupported means that errors<br>will have a low priority of getting fixed.

Cygwin: Installation from source works.

Documentation

A lot of work has been put into making documentation for<br>GNU parallel . GNU parallel<br>includes the 4 types of documentation: Tutorial, How-To, Reference<br>and Design Discussion.

Tutorial

If you prefer reading a book buy GNU Parallel 2018 at<br>https://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html<br>or download it<br>at: https://doi.org/10.5281/zenodo.1146014 (source)<br>Read at least chapter 1+2. It should take you less than 20<br>minutes.

Otherwise start by watching the intro videos for a quick<br>introduction:<br>http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

If you want to dive deeper: spend a couple of hours walking through<br>the tutorial (man parallel_tutorial,<br>HTML, PDF). Your<br>command line will love you for it.

How-To

You can find a lot of EXAMPLE s of use in man<br>parallel_examples (HTML,<br>PDF). That will give you an idea<br>of what GNU parallel is capable of, and you may find a solution<br>you can simply adapt to your situation.

Reference

If you need a one page printable cheat sheet you can find it on:<br>https://www.gnu.org/software/parallel/parallel_cheat.pdf

parallel --help shows you the most commonly used options.

The man page is the reference for all options (man<br>parallel, HTML, PDF).<br>The options map is a map showing which options other options<br>relate to (/usr/share/.../parallel_options_map.pdf,<br>PDF).

There are separate man pages for:

env_parallel (man env_parallel,<br>HTML, PDF),

sem (man sem,<br>HTML, PDF),

parcat (man parcat,<br>HTML, PDF),

parset (man parset,<br>HTML, PDF),

parsort (man parsort,<br>HTML, PDF),

GNU sql (man sql,<br>HTML, PDF), and

GNU niceload (man niceload,<br>HTML, PDF).

For alternatives to GNU parallel , see<br>man parallel_alternatives<br>(HTML, PDF).

Design discussion

If you want to know the design decisions behind<br>GNU parallel , see: man parallel_design<br>(HTML, PDF). This<br>is also a good intro if you intend to change GNU parallel .

You may also want to learn about<br>the history of<br>GNU parallel .

Over the years GNU parallel has gotten more safety<br>features (e.g. no silent data loss if the disk runs full in the<br>middle of a job). These features cost...