mpstat (multiple processor statistics) is an utility that report processors related statistics. Below is the sample default output generated by mpstat with no options:
$ mpstat Linux 2.6.18-92.1.22.el5 (dblx131.lab.perumal.org) 05/19/2009 01:02:40 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s 01:02:40 PM all 1.22 0.68 0.46 0.62 0.03 0.00 0.00 96.99 179.81
The report generated by the mpstat command has the following format:
- CPU: Processor number, starts with 0. The keyword all indicates that statistics are calculated as averages among all processors.
- %user: Percentage of CPU utilization that occurred while executing at the user level (application).
- %nice: Percentage of CPU utilization that occurred while executing at the user level with nice priority.
- %sys: Percentage of CPU utilization that occurred while executing at the system level (kernel). Note that this does not include time spent servicing interrupts or softirqs.
- %iowait: Percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.
- %irq: Percentage of time spent by the CPU or CPUs to service interrupts.
- %soft: Percentage of time spent by the CPU or CPUs to service softirqs. A softirq (software interrupt) is one of up to 32 enumerated software interrupts which can run on multiple CPUs at once.
- %steal: Percentage of time spent in involuntary wait by the virtual CPU or CPUs while the hypervisor was servicing another virtual processor.
- %idle: Percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request.
- intr/s: Total number of interrupts received per second by the CPU or CPUs.
On SMP machines a processor that does not have any activity at all is a disabled (offline) processor.
Below are some general tips, which you can use while interpreting the output –
- If %user is very high then your application is consuming the CPUs and it is being overburdened.
- If %sys is high then your server is burnened by the system (kernel) calls.
- If %iowait constantly a non-zero number, then you may have some disk I/O contention. It is recommended to check the “Time spent waiting for IO (wa)” of vmstat to see whether there is any waiting on disk storage subsystem. You should also consult the iostat output.
Few of the most frequently used examples below –
- Display five reports of global statistics among all processors at two second intervals.
$ mpstat 2 5 Linux 2.6.18-92.1.22.el5 (dblx131.lab.perumal.org) 05/19/2009 01:02:50 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s 01:02:52 PM all 83.58 0.00 14.43 0.00 1.99 0.00 0.00 0.00 1066.67 01:02:54 PM all 88.50 0.00 10.50 0.00 1.00 0.00 0.00 0.00 1061.00 01:02:56 PM all 85.07 0.00 13.43 0.00 1.49 0.00 0.00 0.00 1059.20 01:02:58 PM all 88.00 0.00 11.00 0.00 1.00 0.00 0.00 0.00 1061.00 01:03:00 PM all 88.00 0.00 11.00 0.00 1.00 0.00 0.00 0.00 1058.00 Average: all 86.63 0.00 12.08 0.00 1.30 0.00 0.00 0.00 1061.18
- Display five reports of statistics for processor 0 at two second intervals.
$ mpstat -P 0 2 5 Linux 2.6.18-92.1.22.el5 (dblx131.lab.perumal.org) 05/19/2009 01:03:01 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s 01:03:03 PM 0 86.57 0.00 12.44 0.00 1.00 0.00 0.00 0.00 1048.76 01:03:05 PM 0 88.50 0.00 10.50 0.00 1.00 0.00 0.00 0.00 1063.00 01:03:07 PM 0 95.00 0.00 5.00 0.00 0.00 0.00 0.00 0.00 1023.00 01:03:09 PM 0 92.50 0.00 7.00 0.00 0.50 0.00 0.00 0.00 1029.00 01:03:11 PM 0 96.00 0.00 4.00 0.00 0.00 0.00 0.00 0.00 1019.00 Average: 0 91.71 0.00 7.79 0.00 0.50 0.00 0.00 0.00 1036.56
- Display reports of statistics for all processors at current time.
$ mpstat -P ALL Linux 2.6.18-92.1.22.el5 (dblx131.lab.perumal.org) 05/19/2009 01:03:13 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s 01:03:13 PM all 1.22 0.68 0.46 0.62 0.03 0.00 0.00 96.99 179.82 01:03:13 PM 0 1.22 0.68 0.46 0.62 0.03 0.00 0.00 96.99 179.82