CheatGNU Parallel Parallel shell jobs
Light · Postcard
102 × 152 mm · PDF
Dark · Postcard
102 × 152 mm · PDF
Ink saver · Postcard
102 × 152 mm · PDF
Light · A4 landscape
297 × 210 mm · PDF
Dark · A4 landscape
297 × 210 mm · PDF
Ink saver · A4 landscape
297 × 210 mm · PDF
Light · A3 landscape
420 × 297 mm · PDF
Dark · A3 landscape
420 × 297 mm · PDF
Ink saver · A3 landscape
420 × 297 mm · PDF
Start here
parallel <cmd> {} ::: <a> <b>
Run per argument
ls | parallel <cmd> {}
Args from stdin
parallel --dry-run <cmd> {} ::: <a>
Preview commands
parallel -j 4 ...
Limit to 4 jobs
parallel -k ...Keep input order
parallel --bar ...
Show progress bar
parallel --joblog <file> ...
Log every job
parallel {.} {/} {#}
Name placeholders
Concepts
- Input ::: / :::: / stdin
- Arguments one per job
- Job slots -j N at a time
- Output per job, in order with -k
Inputs
parallel <cmd> ::: <a> <b>
<c>
Arguments on the command line
<producer> | parallel <cmd>
One argument per stdin line
parallel <cmd> :::: <file>
One argument per line of a file
parallel -0 <cmd>
NUL-separated stdin (find -print0)
parallel -d <delim> <cmd>
Custom input delimiter
parallel <cmd> ::: <a> <b> ::: <x>
<y>
All combinations of two sources
parallel <cmd> ::: <a> <b> :::+
<x> <y>
Pair sources (a-x, b-y)
Replacement strings
{}The argument itself
{.}Argument without extension
{/}Basename (no directory)
{/.}Basename without extension
{#}Job sequence number
{%}Job slot number, 1 to -j
{1} {2} {-1}
Argument from source n; negative counts from end
Running jobs
parallel -j <N> ...
Run N jobs at once
parallel -n <N> ...
Up to N arguments per command
parallel -X ...Pack many args into each command line
Safety and preview
parallel --dry-run <cmd> ::: <a>
<b>
Print commands, run nothing
parallel --halt now,fail=1 ...
Stop all jobs at first failure
parallel -q <cmd> '<arg with $>'
Quote the command (simple commands only)
Output and progress
parallel -k ...Print output in input order
parallel --bar ...
Progress bar on stderr
parallel --tag ...
Prefix each line with its argument
Joblog and resume
parallel --joblog <file> ...
Record seq, time, exit code, command
parallel --joblog <file> --resume ...
Skip jobs already logged
parallel --joblog <file> --resume-failed ...
Rerun failed jobs, then the rest
Splitting stdin
<producer> | parallel --pipe <cmd>
Give each job a block of stdin
parallel --pipe -N <N> <cmd>
Exactly N records per job
Remote hosts
parallel -S <host1>,<host2> ...
Run jobs on these ssh hosts
parallel --transferfile {} -S <host> ...
Copy the input file to the host first
Recipes
ls *.png | parallel convert {} {.}.jpg# or safe for odd names:find . -name '*.png' -print0 | parallel -0 convert {} {.}.jpgparallel --dry-run gzip {} ::: *.logparallel -j 4 --bar --joblog jobs.log gzip {} ::: *.log# interrupted? pick up where it stopped:parallel -j 4 --joblog jobs.log --resume gzip {} ::: *.logparallel --tag -k ./run.sh --lr {1} --bs {2} \ ::: 0.1 0.01 ::: 32 64cat big.txt | parallel --pipe --block 20M -k \ grep -c '<pattern>' | paste -sd+ | bc
