Reading text: grep, less, pipes and redirection

Why text matters so much

On Linux almost everything is text: config files, logs, command output. If you can read, filter and save text quickly, you can troubleshoot almost anything. A few small tools do this, and they can be joined together.

Reading files

$ cat /etc/hostname           # print a short file
web01.example.com
$ less /var/log/messages      # scroll a long file: Space = next page, /word = search, q = quit
$ head -n 5 /etc/passwd       # first 5 lines
$ tail -n 20 /var/log/secure  # last 20 lines
$ tail -f /var/log/messages   # follow: keep printing new lines (Ctrl+C to stop)
$ wc -l /etc/passwd           # count lines
38 /etc/passwd

grep: find matching lines

$ grep asha /etc/passwd
asha:x:1000:1000:Asha Rao:/home/asha:/bin/bash
$ grep -i error /var/log/messages | tail -n 3    # -i = ignore case
$ grep -v '^#' /etc/ssh/sshd_config | grep -v '^$'  # hide comments and empty lines
$ grep -r "Listen" /etc/httpd/                   # -r = search all files under a directory
$ grep -c Failed /var/log/secure                 # -c = count matching lines
14
$ grep -n PermitRootLogin /etc/ssh/sshd_config   # -n = show line numbers
40:#PermitRootLogin prohibit-password

grep takes a regular expression, a small pattern language. The most useful parts: ^ means start of line, $ means end of line, . means any one character, and .* means "anything". So ^root matches lines that start with root, and bash$ matches lines that end with bash. Use -E for extended patterns like (error|fail).

Redirection: where output goes

Every command has three standard streams: standard input (stdin, number 0, usually the keyboard), standard output (stdout, 1, normal results) and standard error (stderr, 2, error messages). Redirection sends them somewhere else.

$ find / -name "*.repo" > repos.txt 2> /dev/null
$ date >> /root/checks.log
$ cat repos.txt
/etc/yum.repos.d/redhat.repo

Pipes: joining commands

A pipe | connects the stdout of one command to the stdin of the next. Each tool does one small job, and the pipe chains them. tee is handy in the middle: it writes to a file AND passes the text on.

$ grep "Failed password" /var/log/secure | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head -3
     41 203.0.113.50
     12 198.51.100.7
      3 192.0.2.44
$ ps aux | grep httpd | wc -l
9
$ dnf list installed | tee installed.txt | wc -l
512

Common mistakes

• Running sudo echo text > /etc/file: the redirection is done by YOUR shell, not by sudo, so it fails with "Permission denied". Use echo text | sudo tee -a /etc/file instead.
• Reading and writing the same file in one line: sort data.txt > data.txt empties it before sort reads it. Write to a new file, then mv.
• Forgetting that grep patterns are case-sensitive. Use -i when you are not sure.