CheatPress

sysstat Performance statistics

Version 12.x · source

sysstat, Postcard, light preview Light · Postcard 102 × 152 mm · PDF sysstat, Postcard, dark preview Dark · Postcard 102 × 152 mm · PDF sysstat, Postcard, ink saver preview Ink saver · Postcard 102 × 152 mm · PDF
sysstat, A5, light preview Light · A5 148 × 210 mm · PDF sysstat, A5, dark preview Dark · A5 148 × 210 mm · PDF sysstat, A5, ink saver preview Ink saver · A5 148 × 210 mm · PDF
sysstat, A4, light preview Light · A4 210 × 297 mm · PDF sysstat, A4, dark preview Dark · A4 210 × 297 mm · PDF sysstat, A4, ink saver preview Ink saver · A4 210 × 297 mm · PDF
sysstat, A4 landscape, light preview Light · A4 landscape 297 × 210 mm · PDF sysstat, A4 landscape, dark preview Dark · A4 landscape 297 × 210 mm · PDF sysstat, A4 landscape, ink saver preview Ink saver · A4 landscape 297 × 210 mm · PDF
sysstat, A3, light preview Light · A3 297 × 420 mm · PDF sysstat, A3, dark preview Dark · A3 297 × 420 mm · PDF sysstat, A3, ink saver preview Ink saver · A3 297 × 420 mm · PDF
sysstat, A3 landscape, light preview Light · A3 landscape 420 × 297 mm · PDF sysstat, A3 landscape, dark preview Dark · A3 landscape 420 × 297 mm · PDF sysstat, A3 landscape, ink saver preview Ink saver · A3 landscape 420 × 297 mm · PDF
sysstat, Letter, light preview Light · Letter 8.5 × 11 in · PDF sysstat, Letter, dark preview Dark · Letter 8.5 × 11 in · PDF sysstat, Letter, ink saver preview Ink saver · Letter 8.5 × 11 in · PDF
sysstat, Tabloid, light preview Light · Tabloid 11 × 17 in · PDF sysstat, Tabloid, dark preview Dark · Tabloid 11 × 17 in · PDF sysstat, Tabloid, ink saver preview Ink saver · Tabloid 11 × 17 in · PDF

Start here

iostat -xz 2
Disk load, busy devices only
iostat -c 2
CPU split incl. iowait
sar -u 2 5
CPU, 5 samples
sar -r 2 5
Memory use
sar -n DEV 2 5
Network throughput
sar -q 2 5
Load and run queue
pidstat -u 2
CPU per process
pidstat -d 2
Disk I/O per process

Concepts

  1. Kernel counters
  2. sadc collector
  3. /var/log/sa/saDD daily file
  4. sar -f report

Disks (iostat)

iostat -x 2
Extended disk stats every 2 s
First report is the average since boot; add -y to skip it.
iostat -xz 2
Only devices with activity
iostat -c 2
CPU report only
iostat -xy 1 5
Five samples, no since-boot report
iostat -xm 2
Rates in MB per second
iostat -d -h
Human-readable device report
iostat -p <device> 2
Device with its partitions
iostat -N -x 2
Show LVM / device-mapper names
iostat -t -x 5
Timestamp every report
iostat -o JSON -x 1 1
One report as JSON

System (sar live)

sar -u 2 5
CPU utilisation, 5 samples
Add -P ALL for every core separately.
sar -r 2 5
Memory utilisation
sar -n DEV 2 5
Network throughput per interface
sar -d -p 2 5
Per-disk activity, readable names
sar -q 2 5
Load average and run queue
sar -P ALL 2 5
Each CPU core
sar -S 2 5
Swap usage
sar -n EDEV 2 5
Interface errors and drops
sar -b 2 5
Overall I/O transfer rates
sar -B 2 5
Paging and page faults
sar -W 2 5
Swap in and out per second
sar -w 2 5
Context switches and task creation
sar -n TCP,ETCP 2 5
TCP connections and errors
sar -q PSI 2 5
Pressure stall (CPU, I/O, memory)
Needs a recent sysstat and a kernel with PSI.
sar -F MOUNT 2 5
Filesystem usage by mount point
Only if the collector gathers filesystem data.

History (sar -f)

sar -u
Today's CPU history
Reads the daily file; needs the collector running.
sar -r -f /var/log/sa/sa<DD>
Memory on day DD of the month
sar -u -f /var/log/sa/sa<DD> -s 09:00 -e 11:00
CPU in a time window
sar -q -1
Load history from yesterday
sar -u -i 600
One record per 10 minutes
sar -o <file> 5 12 >/dev/null
Record samples to your own file
sar -u -r -f <file>
Several reports from one file
sar -u -f <file> -C
Show sadc comments in the file

Processes (pidstat)

pidstat -u 2 5
CPU per process
pidstat -r 2 5
Memory and page faults per process
pidstat -d 2 5
Disk I/O per process
pidstat -p <pid> 1
Watch one process
pidstat -t -p <pid> 1
Per-thread numbers
pidstat -w 2 5
Context switches per process
pidstat -l 2 5
Full command line
pidstat -C <name> 2
Only commands matching name
pidstat -U 2
User names instead of UIDs
pidstat -u -I 2
CPU divided by core count
pidstat -h -u -r -d 1
CPU, memory, I/O on one line

Reading the numbers

iostat -x columns
Column Meaning
%util time with I/O in flight
await avg ms per request, incl. queue
r_await / w_await same, reads / writes
aqu-sz average queue length
rareq-sz / wareq-sz avg request size, KiB
%iowait CPU idle, I/O pending
%steal CPU lost to hypervisor
pidstat counters
Column Meaning
cswch/s voluntary switches: blocked, waiting
nvcswch/s involuntary: preempted by others
majflt/s faults that read from disk
minflt/s faults served from memory
kB_ccwr/s writes cancelled before disk

Recipes

Is the disk the bottleneck?
iostat -xz 2pidstat -d 2   # who is doing the I/O
What happened at 3 am?
sar -u -f /var/log/sa/sa<DD> -s 02:30 -e 03:30sar -q -f /var/log/sa/sa<DD> -s 02:30 -e 03:30
Capture, then analyse later
sar -o /tmp/cap.sa 5 60 >/dev/nullsar -u -r -f /tmp/cap.sa
Find the CPU and memory hog
pidstat -u -r -l 2 5

Help

man sar
Manual page with all report fields
sar -V
Show the version