Processes: ps, top and jobs

Programs and processes

A program is a file on disk, like /usr/sbin/httpd. A process is a running copy of it, with its own memory. One program can run as many processes: a busy web server may have dozens of httpd processes. The kernel gives each process a number, the PID. The first process started at boot is systemd, with PID 1; every other process has a parent (PPID) that started it.

ps: a snapshot

$ ps aux | head -4
USER   PID %CPU %MEM    VSZ   RSS TTY  STAT START   TIME COMMAND
root     1  0.0  0.4 174744 16884 ?    Ss   09:01   0:02 /usr/lib/systemd/systemd --switched-root --system
root     2  0.0  0.0      0     0 ?    S    09:01   0:00 [kthreadd]
root   842  0.0  0.2  15840  9100 ?    Ss   09:01   0:00 sshd: /usr/sbin/sshd -D [listener] ...
$ ps -ef --forest | grep -A2 sshd    # parent/child tree
$ ps -o pid,ppid,user,%cpu,%mem,cmd --sort=-%mem | head -5   # pick columns, sort by memory
$ pgrep -l httpd
1301 httpd
1302 httpd

A zombie (Z) is a finished process whose parent has not yet collected its exit status. A few zombies are harmless; thousands mean the parent program has a bug. You cannot kill a zombie; you fix or restart its parent.

top: a live view

$ top
top - 10:42:01 up 3 days,  2:10,  2 users,  load average: 0.52, 0.48, 0.40
Tasks: 182 total,   1 running, 181 sleeping,   0 stopped,   0 zombie
%Cpu(s):  4.1 us,  1.2 sy,  0.0 ni, 94.5 id,  0.1 wa,  0.0 hi,  0.1 si,  0.0 st
MiB Mem :   3654.2 total,    512.4 free,   1820.6 used,   1321.2 buff/cache
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 1301 apache    20   0  412844  20480   8192 S   2.3   0.5   0:12.44 httpd

The load average is the average number of processes running or waiting to run over the last 1, 5 and 15 minutes. Compare it with the number of CPUs (lscpu or nproc): a load of 4 on a 4-CPU server is busy but fine; 12 on 4 CPUs means work is queueing. High wa (I/O wait) in the CPU line means processes are waiting for the disk.

Jobs: foreground and background

$ sleep 600 &            # & starts it in the background
[1] 5120
$ jobs
[1]+  Running                 sleep 600 &
$ tar -czf /tmp/etc.tgz /etc     # started in the foreground... press Ctrl+Z
^Z
[2]+  Stopped                 tar -czf /tmp/etc.tgz /etc
$ bg %2                   # continue job 2 in the background
$ fg %1                   # bring job 1 to the foreground (Ctrl+C would stop it)

Jobs belong to your shell. If you log out, they usually get a hang-up signal and stop. For a long task on a remote server, use a terminal multiplexer like tmux, or nohup command &, or better, run it as a systemd service or timer.

Common mistakes

• ps aux | grep httpd also shows the grep line itself. Use pgrep httpd, or grep [h]ttpd.
• Reading %MEM in top as "free memory is low, panic". Linux uses spare RAM as disk cache (buff/cache). Look at the "avail Mem" value in top or "available" in free -h instead.
• Starting a 3-hour backup in an SSH session with & and then closing the laptop. Use tmux or a systemd unit.