SHELL | Logs and troubleshooting • PHiNiT.DE

Command	Description	Platform	Example
journalctl	Displays log entries from systemd . Allows filtering by services, priorities, time periods, etc.	Systemd-based systems (Ubuntu from 15.04, RHEL/CentOS 7+)	journalctl -u ssh.service
DMESG	Displays kernel ring buffer messages, useful for hardware and driver issues.	All	‘dmesg
tail	Displays the last lines of a file, often used to monitor ongoing logs.	All	tail -f /var/log/syslog
grep	Searches files or outputs for patterns, useful for filtering log entries.	All	grep error /var/log/syslog
less	Interactive pager for displaying text files, allows comfortable navigation in logs.	All	less /var/log/syslog
cat	Outputs the contents of files on the console.	All	cat /var/log/syslog
Logger	Writes messages to the system log, useful for testing.	All	logger “test message”
rsyslog	A daemon for processing log messages, configurable via /etc/rsyslog.conf.	All	–
/var/log/syslog	Contains general system messages and errors (main log file).	Ubuntu/Debian	–
/var/log/messages	Contains general system and kernel messages.	RHEL/CentOS	–
/var/log/auth.log	Logs authentication attempts, useful for security checks.	Ubuntu/Debian	–
/var/log/secure	Contains security and authentication messages.	RHEL/CentOS	–
Logrotate	Utility for rotating, compressing, and deleting old log files, configurable via /etc/logrotate.conf.	All	–
strace	Tracks system calls of a process, useful for troubleshooting programs.	All	strace -p
lsof	Lists open files, can be used to see which processes are using log files.	All	lsof /var/log/syslog
netstat/ss	Displays network connections, useful for troubleshooting network services.	All	netstat tulips or ss tulips
ps	Displays running processes, can be used to identify hanging processes.	All	ps aux
top / htop	Real-time system monitoring, displays CPU, memory consumption and running processes.	All	top or htop
systemctl	Controls systemd services, can be used to check service status.	Systemd-based systems	systemctl status sshd
Service	Controls system services in SysVinit systems.	Legacy Systems	Service SSH Status
chkconfig	Managing system services at boot (SysVinit).	RHEL/CentOS (older versions)	chkconfig –list
update-rc.d	Managing startup scripts (SysVinit).	Ubuntu/Debian (older versions)	update-rc.d ssh enable
uptime	Displays how long the system has been running, as well as the average system load.	All	uptime
free	Displays memory and swap usage, helpful for memory issues.	All	free -h
vmstat	Displays system performance, including memory, swap, CPU usage.	All	VMSTAT 5
sar	System Activity Report, provides historical performance data (part of the sysstat package).	All	sar -u 1 3
IOSTAT	Displays CPU and I/O statistics for devices and partitions (sysstat package required).	All	IOSTAT -XZ 5
tcpdump	Intercept and display network packets, helpful for network troubleshooting.	All	tcpdump -i eth0 port 80
ping	Checks the reachability of hosts, basic network diagnostics.	All	ping example.com
Traceroute	Shows the path of the packets to the target host, useful in case of network problems.	All	Traceroute example.com
mtr	Combination of ping and traceroute, provides continuous monitoring.	All	mtr example.com
hostnamectl	Displays system information, including kernel and architecture (systemd).	Systemd-based systems	hostnamectl
who	Displays which users are logged in, useful for monitoring.	All	who
Load	Shows a list of recent logins, can be used for security checks.	All	Load
sudo	Executes commands with administrator privileges, important for commands that require root privileges.	All	sudo tail /var/log/auth.log

Notes on platform differences

Systemd vs. SysVinit:
- Systemd is the default init system in modern distributions such as Ubuntu (from 15.04), RHEL/CentOS 7 and later.
- Older versions use SysVinit, which requires the use of commands such as service and chkconfig.

Log files:
- Ubuntu/Debian:
  - General system messages: /var/log/syslog
  - Authentication protocols: /var/log/auth.log
- RHEL/CentOS:
  - General system messages: /var/log/messages
  - Authentication protocols: /var/log/secure

Additional Tools

Tool	Description	Example
gdB	Debugger for programs, useful for crash analysis.	gdb /path/to/program
valgrind	Tool to uncover memory errors and memory leaks.	Valgrind ./Program
ltrace	Tracks calls to library functions, useful for debugging programs.	LTRACE ./Program
IP	Modern network management tool, replaces older tools such as ifconfig.	IP Addr Show
ethtool	Displays and changes settings for network interfaces, helpful in case of network problems.	ethtool eth0
nslookup / dig	Perform DNS queries, helpful for troubleshooting name resolution.	dig example.com
fdisk / lsblk	Displays disks and partitions, useful in case of storage problems.	LSBLK
LSHW	Lists hardware components of the system.	lshw -short

Example of troubleshooting

Problem: A service does not start correctly.

Troubleshooting steps:

Check service status:
- Systemd:
  - systemctl status
- SysVinit:
  - Service Status

To view logs of the service:
- journalctl -u (Systemd)
- Checking the relevant log files:
  - Ubuntu/Debian: /var/log/syslog, /var/log/auth.log
  - RHEL/CentOS: /var/log/messages, /var/log/secure

Filtering error messages:
- grep -i error /var/log/syslog (Ubuntu/Debian)
- grep -i error /var/log/messages (RHEL/CentOS)

Check configuration files:
- Perform a syntax check (if available):
  - For example, for Nginx: nginx -t
  - For Apache: apachectl configtest

Check ports and network:
- To check if the port is already in use:
  - sudo lsof -i :
- To view network connections:
  - netstat tulips or ss tulips

Review resources:
- Check CPU and memory usage:
  - top or htop
- Check disk space:
  - df -h

To rule out hardware problems:
- Checking kernel messages:
  - dmesg | less