Linux networking for DevOps

Why networking matters

Apps talk to databases, caches, other services and the internet. When something breaks, the question is often "where does the connection stop?" You do not need to be a network engineer, but you must be able to check each step: does this machine have an address, can it reach the other machine, does the name resolve, is anything listening on the port, and is a firewall in the way?

Your addresses and routes

ip addr            # addresses on each interface (ip a for short)
ip route           # routing table; "default via" is the gateway
hostname -I        # just the IP addresses

Is the other side reachable?

ping -c 4 10.0.1.20                 # 4 ICMP echo requests
curl -I https://example.com          # only the response headers: status code, server
curl -v http://localhost:3000/healthz   # verbose: DNS, connect, request, response
wget https://example.com/file.tar.gz # download a file

What is listening? ss -tulpn

A port is a number from 0 to 65535 that identifies a service on a machine: 22 for SSH, 80 for HTTP, 443 for HTTPS, 5432 for PostgreSQL, 3306 for MySQL, 6379 for Redis. A program "listens" on a port and waits for connections.

sudo ss -tulpn
# Netid State  Recv-Q Send-Q Local Address:Port  Peer Address:Port Process
# tcp   LISTEN 0      511          0.0.0.0:80         0.0.0.0:*     users:(("nginx",pid=812,fd=6))
# tcp   LISTEN 0      4096       127.0.0.1:5432       0.0.0.0:*     users:(("postgres",pid=640,fd=5))

Read the local address carefully. 0.0.0.0:80 means "all IPv4 addresses", so other machines can connect. 127.0.0.1:5432 means "only this machine" (localhost). A classic problem: the database listens on 127.0.0.1, so the app on another server gets "connection refused". netstat -tulpn shows the same on older systems, but ss is the modern tool.

Names: DNS and /etc/hosts

dig api.example.com +short     # the IP address(es) for a name
dig api.example.com MX         # mail records
nslookup api.example.com       # older tool, same idea
cat /etc/hosts                 # local name overrides

On most Linux systems, the file /etc/nsswitch.conf says the order: usually /etc/hosts first ("files"), then DNS. That is why an old line in /etc/hosts can make one server see a different IP than everyone else, a confusing bug. Note that dig ignores /etc/hosts, while curl and your app use it.

Firewalls

# Ubuntu: ufw
sudo ufw allow 22/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

# RHEL / Rocky / Alma: firewalld
sudo firewall-cmd --add-service=https --permanent
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

In the cloud there is usually a second firewall outside the machine, such as an AWS security group or an Azure network security group. A port must be open in both.

SSH, scp, config and port forwarding

ssh deploy@203.0.113.10                 # log in
ssh -p 2222 deploy@203.0.113.10         # SSH on a non-standard port (lower-case p)
scp app.tar.gz deploy@203.0.113.10:/tmp/    # copy a file to the server
scp -P 2222 -r config/ deploy@203.0.113.10:/opt/app/   # scp uses capital P for the port
ssh -L 5433:localhost:5432 deploy@bastion   # local port 5433 -> port 5432 on bastion

# ~/.ssh/config
Host web1
    HostName 203.0.113.10
    User deploy
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Connection refused versus timeout

A troubleshooting order

1. Does the name resolve? dig the host name.
2. Is the address reachable? ping, or better, curl the real port (nc -zv host port tests any TCP port).
3. Is the service listening on the right port and address? ss -tulpn on the server.
4. Is a firewall or security group in the way? Check ufw or firewalld and the cloud rules.
5. Does the app itself answer correctly? curl -v and the app’s logs.