LPIC1 Exam Guide
Search…
108.2. System logging
Weight: 3
Description: Candidates should be able to configure the syslog daemon. This objective also includes configuring the logging daemon to send log output to a central log server or accept log output as a central log server. Use of the systemd journal subsystem is covered. Also, awareness of rsyslog and syslog-ng as alternative logging systems is included.
Key Knowledge Areas:
- Configuration of the syslog daemon
- Understanding of standard facilities, priorities and actions
- Configuration of logrotate
- Awareness of rsyslog and syslog-ng
Terms and Utilities:
- syslog.conf
- syslogd
- klogd
- /var/log/
- logger
- logrotate
- /etc/logrotate.conf
- /etc/logrotate.d/
- journalctl
- /etc/systemd/journald.conf
- /var/log/journal/
Why Logging?
A Linux system has many subsystems and applications running. We use system logging to gather data about our running system from the moment it boots. Sometimes we just need to know that all is well. At other times we use this data for auditing, debugging, knowing when a disk or other resource is running out of capacity, and many other purposes.
syslog
Syslog is a standard for sending and receiving notification messages–in a particular format–from various network devices. The messages include time stamps, event messages, severity, host IP addresses, diagnostics and more.
syslogd
The syslog daemon is a server process that provides a message logging facility for application and system processes.Unfortunately linux logging is one of aspects of linux which is transition fase. The traditional syslog facility and its syslogd daemon has been supplemented by other logging facilities such as rsyslog, syslog-ng, and the systemd journal subsystem.
syslog.conf
The syslog.conf file is the main configuration file for the syslogd. Whenever syslogd receives a log message, it acts based on the message's type (or facility) and its priority (called selector fields).
1
type (facility).priority (severity) destination(where to send the log)
Copied!
Facilities are simply categories. Some facilities in Linux are:
auth, user, kern, cron, daemon, mail, local1, local2, ...authuser: User-level messageskern: Kernel messagescorn: Clock daemondaemon: System daemonsmail: Mail systemlocal0 – local7: Locally used facilities
priorities Unlike facilities, which have no relationship to each other, priorities are hierarchical. Possible priorities in Linux are:
emerg/panic, alert, crit, err/error, warn/warning, notice, info, debug- 1.
emerg :System is unusable - 2.
alert :Action must be taken immediately - 3.
critical :Critical conditions - 4.
err :Error conditions - 5.
warning :Warning conditions - 6.
notice :Normal but significant conditions - 7.
info :Informational messages - 8.
debug :Debug-level messages
if we log some specific priority , all the more important things will be logged too
Action: Each line in this file specifies one or more facility/priority selectors followed by an action . On the action field we can have things like:
action
example
notes
filename
/var/log/messages
Writes logs to specified file
username
user2
Will notify that person on the screen
@ip
@192.168.10.42
Will send logs to specified log server and that server decides how to treat logs based on its configs.
In the following syslog.conf line, mail.notice is the selector and /var/log/mail is the action (i.e., “write messages to /var/log/mail”):
1
mail.notice /var/log/mail
Copied!
Within the selector, “mail” is the facility (message category) and “notice” is the level of priority. You can see part of syslog.conf (CentOS6) :
1
#### RULES ####
2
3
# Log all kernel messages to the console.
4
# Logging much else clutters up the screen.
5
#kern.* /dev/console
6
7
# Log anything (except mail) of level info or higher.
8
# Don't log private authentication messages!
9
*.info;mail.none;authpriv.none;cron.none /var/log/messages
10
11
# The authpriv file has restricted access.
12
authpriv.* /var/log/secure
13
14
# Log all the mail messages in one place.
15
mail.* -/var/log/maillog
16
17
18
# Log cron stuff
19
cron.* /var/log/cron
20
21
# Everybody gets emergency messages
22
*.emerg *
23
24
# Save news errors of level crit and higher in a special file.
25
uucp,news.crit /var/log/spooler
26
27
# Save boot messages also to boot.log
28
local7.* /var/log/boot.log
29
Copied!
*: wildcard . signifying “any facility” or "any priority"dash - : means it can use memory cache (:don't waist time constantly writing to the disk )equal sign = : to log ONLY one specific level of priority.facility.=priority action
There is also /etc/rsyslog.d/ directory and it is better for different softwares and admins to add their specific configs there, instead of editing the main configuration file (See Ubuntu16).
klogd
How do boot-time kernel messages get logged before a file system is even mounted? The kernel stores messages in a ring buffer in memory. The
klogd daemon processes these messages directly to a console, or a file such as /var/log/dmesg, or through the syslog facility./var/log
Almost all logfiles are located under /var/log directory and its sub-directories on Linux(CentOS6).
1
[[email protected] ~]# ls /var/log
2
anaconda.ifcfg.log cron messages-20200217 tallylog
3
anaconda.log cron-20200217 ntpstats vmware-caf
4
anaconda.program.log cups pm-powersave.log vmware-install.log
5
anaconda.storage.log dmesg ppp vmware-tools-upgrader.log
6
anaconda.syslog dmesg.old prelink vmware-vgauthsvc.log.0
7
anaconda.xlog dracut.log sa vmware-vmsvc.log
8
anaconda.yum.log gdm samba wpa_supplicant.log
9
audit httpd secure wtmp
10
boot.log lastlog secure-20200217 Xorg.0.log
11
btmp maillog spice-vdagent.log Xorg.0.log.old
12
btmp-20200217 maillog-20200217 spooler yum.log
13
ConsoleKit messages spooler-20200217 yum.log-20200217
Copied!
You can use your favorite text editor or less or tail commands in conjunction with grep to read these log files.
creating rsyslog listener
We can creating rsyslog listener and catch other systems log messages. That is pretty easy.
1
###CentOS 6
2
vi /etc/rsyslog.conf
3
4
###Change from
5
#$UDPServerRun 514
6
#$ModLoad imtcp
7
#$InputTCPServerRun 514
8
9
### to
10
$UDPServerRun 514
11
$ModLoad imtcp
12
$InputTCPServerRun 514
Copied!
1
###ubuntu 16
2
vim /etc/default/rsyslog
3
4
### Change From
5
RSYSLOGD_OPTIONS=""
6
7
### To
8
RSYSLOGD_OPTIONS="-r"
Copied!
and finally do not forget to restart the service
systemctl restart rsyslog .journalctl
Systemd also has its own journaling program called journald and it stores things in binary files. We can't go and see text files (like what we did in syslog/rsyslog), so we have to use special tool called journalctl to access them(CentOS7):
1
[[email protected] ~]# journalctl
2
-- Logs begin at Mon 2020-02-10 02:51:48 EST, end at Tue 2020-02-18 09:19:13 EST. --
3
Feb 10 02:51:48 localhost.localdomain systemd-journal[106]: Runtime journal is using 8.
4
Feb 10 02:51:48 localhost.localdomain kernel: Initializing cgroup subsys cpuset
5
Feb 10 02:51:48 localhost.localdomain kernel: Initializing cgroup subsys cpu
6
Feb 10 02:51:48 localhost.localdomain kernel: Initializing cgroup subsys cpuacct
7
Feb 10 02:51:48 localhost.localdomain kernel: Linux version 3.10.0-693.el7.x86_64 (buil
8
Feb 10 02:51:48 localhost.localdomain kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-
9
Feb 10 02:51:48 localhost.localdomain kernel: Disabled fast string operations
10
Feb 10 02:51:48 localhost.localdomain kernel: e820: BIOS-provided physical RAM map:
11
Feb 10 02:51:48 localhost.localdomain kernel: BIOS-e820: [mem 0x0000000000000000-0x0000
12
Feb 10 02:51:48 localhost.localdomain kernel: BIOS-e820: [mem 0x000000000009ec00-0x0000
13
Feb 10 02:51:48 localhost.localdomain kernel: BIOS-e820: [mem 0x00000000000dc000-0x0000
Copied!
As we mentioned earlier , linux logging is one of aspects of linux which is in under change. Distributions with systemd has journald, beside that some of them still preserve rsyslog and some other not. Try to find out your linux logging system
/etc/systemd/journald.conf
The config file of journalctl is located at /etc/systemd/journald.conf (CentOS7)
1
[[email protected] ~]# cat /etc/systemd/journald.conf
2
# This file is part of systemd.
3
#
4
# systemd is free software; you can redistribute it and/or modify it
5
# under the terms of the GNU Lesser General Public License as published by
6
# the Free Software Foundation; either version 2.1 of the License, or
7
# (at your option) any later version.
8
#
9
# Entries in this file show the compile time defaults.
10
# You can change settings by editing this file.
11
# Defaults can be restored by simply deleting this file.
12
#
13
# See journald.conf(5) for details.
14
15
[Journal]
16
#Storage=auto
17
#Compress=yes
18
#Seal=yes
19
#SplitMode=uid
20
#SyncIntervalSec=5m
21
#RateLimitInterval=30s
22
#RateLimitBurst=1000
23
#SystemMaxUse=
24
#SystemKeepFree=
25
#SystemMaxFileSize=
26
#RuntimeMaxUse=
27
#RuntimeKeepFree=
28
#RuntimeMaxFileSize=
29
#MaxRetentionSec=
30
#MaxFileSec=1month
31
#ForwardToSyslog=yes
32
#ForwardToKMsg=no
33
#ForwardToConsole=no
34
#ForwardToWall=yes
35
#TTYPath=/dev/console
36
#MaxLevelStore=debug
37
#MaxLevelSyslog=debug
38
#MaxLevelKMsg=notice
39
#MaxLevelConsole=info
40
#MaxLevelWall=emerg
Copied!
logger
The Linux logger command provides an easy way to generate some logs(centOS6)
1
[[email protected] ~]# logger local1.emerg Hello! This is my log!
Copied!
and it will appear at /var/log/syslog (or /var/log/messages):
1
[[email protected] ~]# tail -5 /var/log/messages
2
Feb 18 06:54:42 server1 NetworkManager[2183]: <info> nameserver '172.16.43.2'
3
Feb 18 06:54:42 server1 NetworkManager[2183]: <info> domain name 'localdomain'
4
Feb 18 06:54:42 server1 dhclient[29269]: bound to 172.16.43.137 -- renewal in 693 seconds.
5
Feb 18 06:54:42 server1 nm-dispatcher.action: Script '/etc/NetworkManager/dispatcher.d/13-named' exited with error status 1.
6
Feb 18 06:59:34 server1 payam: Hello! This is my log!
7
Copied!
logrotate
With the amount of logging that is possible, we need to be able to control the size of log files. This is done using the
logrotateutility , which is usually run as a cron job.The important files to pay attention to are:
- /usr/sbin/logrotate -- the logrotate command itself (the executable)
- /etc/cron.daily/logrotate -- the shell script that runs logrotate on a daily basis (note that it might be /etc/cron.daily/logrotate.cron on some systems)
- /etc/logrotate.conf -- the log rotation configuration file
Another important file is /etc/logrotate.d, included in the process through this line in the /etc/logrotate.conf file:
/etc/logrotate.conf
Use the /etc/logrotate.conf configuration file to specify how your log rotating and archiving should happen.
1
[[email protected] ~]# cat /etc/logrotate.conf
2
# see "man logrotate" for details
3
# rotate log files weekly
4
weekly
5
6
# keep 4 weeks worth of backlogs
7
rotate 4
8
9
# create new (empty) log files after rotating old ones
10
create
11
12
# use date as a suffix of the rotated file
13
dateext
14
15
# uncomment this if you want your log files compressed
16
#compress
17
18
# RPM packages drop log rotation information into this directory
19
include /etc/logrotate.d
20
21
# no packages own wtmp and btmp -- we'll rotate them here
22
/var/log/wtmp {
23
monthly
24
create 0664 root utmp
25
minsize 1M
26
rotate 1
27
}
28
29
/var/log/btmp {
30
missingok
31
monthly
32
create 0600 root utmp
33
rotate 1
34
}
35
36
# system-specific logs may be also be configured here.
Copied!
Each log file may be handled daily, weekly, monthly, or when it grows too large.
parameter
meaning
missingok
don’t write an error message if the log file is missing
daily, weekly, monthly
rotate logs daily, weekly, monthly
rotate N
keep the latest N logs and delete the older ones
compress
compress the log (creates gz files)
create mode owner group
Immediately after rotation (before the postrotate script is run) the log file is created with this acces and owner
minsize N
Log files are rotated when they grow bigger than size bytes, but not before the additionally specified time interval(daily,...)
this file contains some default settings and sets up rotation for a few logs that are not owned by any system packages. It also uses an
include statement to pull in configuration from any file in the /etc/logrotate.d directory(CentOS6)./etc/logrotate.d
Any packages we install that need help with log rotation will place their Logrotate configuration here.
1
[[email protected] ~]# ls /etc/logrotate.d/
2
ConsoleKit cups dracut httpd named ppp psacct syslog wpa_supplicant yum
3
4
[[email protected] ~]# cat /etc/logrotate.d/httpd
5
/var/log/httpd/*log {
6
missingok
7
notifempty
8
sharedscripts
9
delaycompress
10
postrotate
11
/sbin/service httpd reload > /dev/null 2>/dev/null || true
12
endscript
13
}
Copied!
These are the meaning of some of these parameters:
parameter
meaning
missingok
don’t write an error message if the log file is missing
notifempty
don’t rotate the log file if it is empty.
shared scripts
Run prerotate and postrotate scripts for every log file which is rotated
delaycompress
Postpone compression of the previous log file to the next rotation cycle
That's all!
.
.
.
.
.
Last modified 2yr ago