noted.do
The Tech Guy

How to Read a Server Log Without Panicking

The first time you open the access log on a public web server, it looks like you're under siege. Requests for /wp-login.php on a site that has never run WordPress. Requests for /.env, /phpmyadmin, /admin/config.bak, strings full of ../../../etc/passwd. Thousands of them, from IPs all over the world.

Take a breath. That's just the internet. Every server with a public IP gets scanned constantly by automated bots looking for easy targets. They aren't after you specifically. They're knocking on every door in the city to see which ones are unlocked. A 404 in response means the door wasn't there, which is exactly what you want.

Once you get past that, logs are genuinely useful, and they're much less cryptic than they look.

Anatomy of an access log line

Most web servers (Nginx, Apache, and plenty of others) default to something close to the "combined" log format. A line looks like this:

203.0.113.42 - - [14/Mar/2025:09:21:07 +0000] "GET /blog/hello HTTP/1.1" 200 5123 "https://example.com/" "Mozilla/5.0 (...)"

Left to right, that's:

  • 203.0.113.42 is the client's IP address. If you're behind a reverse proxy or CDN and this is always the same address, your proxy isn't passing the real IP through.
  • The two dashes are leftovers for identity and authenticated user. Almost always empty.
  • The timestamp in brackets, with its time zone offset. Check that offset before you start correlating events with "it broke around 3pm."
  • "GET /blog/hello HTTP/1.1" is the request itself: method, path, protocol.
  • 200 is the status code. This is the most important field.
  • 5123 is the response size in bytes.
  • The referrer, the page that linked here, if the browser sent one.
  • The user agent, which says what browser or bot made the request. It's self-reported, so bots can and do lie.

The status codes that matter

You don't need to memorize all of them. The first digit tells you most of the story. 2xx is success. 3xx is a redirect. 4xx means the client asked for something wrong (404 not found, 403 forbidden, 401 needs login, 429 too many requests). 5xx means your server broke.

That's the big mental shift. A pile of 404s is usually noise. A pile of 5xx errors is your problem, and those are the lines worth chasing. A 502 or 504 behind a reverse proxy usually means the proxy is fine but the app behind it is down or too slow to answer.

The error log is the other half

The access log tells you what happened. The error log tells you why. When you see a 500 in the access log, find the matching timestamp in the error log (or your application's own log, for something like PHP-FPM or a Node app). That's where the actual message lives: a missing file, a permissions problem, a database connection refused, a stack trace pointing at a specific line.

On a typical Linux box these live somewhere under /var/log/, like /var/log/nginx/access.log and /var/log/nginx/error.log. If your stuff runs under systemd, journalctl -u servicename shows that service's output. In Docker, it's docker logs containername.

A few commands that do most of the work

Watch new lines arrive live while you reproduce a problem:

tail -f /var/log/nginx/error.log

Find all the server errors:

grep '" 5[0-9][0-9] ' /var/log/nginx/access.log

Count which IPs are hitting you most:

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

Count status codes, which gives you a quick health check at a glance:

awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

That last one assumes the default combined format. If you've customized your log format, the field number will be different.

Older logs usually get rotated into files like access.log.1 and access.log.2.gz. For the compressed ones, zgrep and zcat work like grep and cat.

When to actually worry

Scanners getting 404s: ignore them. If they bug you, tools like fail2ban or rate limiting at the proxy can block repeat offenders, but it's housekeeping, not an emergency.

What deserves a closer look is anything getting a 200 that shouldn't. A request for /.env or /backup.sql that returns 200 with a real file size means something sensitive is publicly readable, and you should fix that right now (and rotate whatever secrets were in it). Same for a sudden flood of 5xx errors, a login endpoint taking thousands of POST requests from one address, or traffic to a path you don't recognize that keeps succeeding.

My advice is to read your logs on a calm day, when nothing is wrong. Get a feel for what normal looks like on your server. Then when something does break, the weird lines jump out instead of blending into the noise.

Comments

Be kind. See the rules.