107.2. Automate system administration tasks by scheduling jobs

Weight: 4
Description: Candidates should be able to use cron or anacron to run jobs at regular intervals and to use at to run jobs at a specific time.
Key Knowledge Areas:
Manage cron and at jobs
Configure user access to cron and at services
Configure anacron
Terms and Utilities:
/etc/cron.{d,daily,hourly,monthly,weekly}/
/etc/at.deny
/etc/at.allow
/etc/crontab
/etc/cron.allow
/etc/cron.deny
/var/spool/cron/
crontab
at
atq
atrm
anacron
/etc/anacrontab
Many system administration tasks must be done regularly, such as rotating log files, backing up files or databases, preparing reports, or installing system updates. In this lesson we will learn how to automate these kinds of jobs by setting up  scheduling for them
By scheduling :
We can start a job for a time at which system usage is low
Errors will be reduced due to less manual interaction is needed.
We are  sure that  jobs  always run the same way
System administrators can sleep more!
Run jobs at regular intervals
Linux systems have two facilities for scheduling jobs to run at regular intervals:
The original cron facility is best suited to servers and systems that are continuously powered on.
The anacron (or anachronistic cron) facility is suited to systems such as desktops or laptops that can be asleep or running on battery power.
cron
anacron
Good for servers
Good for laptops and desktops
Granularity from one minute to one year
Daily, weekly, and monthly granularity
Job runs only if system is running at scheduled time
Job runs when system is next available
Can be used by normal users
Needs root authority
Schedule periodic jobs with cron
 The cron facility consists of the cron daemon and a set of tables that describe what work is to be done and with what frequency. The cron daemon is usually started by the init, upstart, or systemd process at system startup. 
crontab file syntax and Operators
Crontab (cron table) is a text file that specifies the schedule of cron jobs. The cron daemon wakes up every minute and checks each crontab for jobs that need to run. 
 There are two types of crontab files.  The  individual user crontab files and  system-wide crontab files.
crontab syntax and operators
Each line in the user crontab file contains six fields separated by a space followed by the command to be run.
1
* * * * * command(s)
2
- - - - -
3
| | | | |
4
| | | | ----- Day of week (0 - 7) (Sunday=0 or 7)
5
| | | ------- Month (1 - 12)
6
| | --------- Day of month (1 - 31)
7
| ----------- Hour (0 - 23)
8
------------- Minute (0 - 59)
Copied!
* -The asterisk operator means any value or always.
, -The comma operator allows you to specify a list of values for repetition. 
- -The hyphen operator allows you to specify a range of values. 
/ -The slash operator allows you to specify values that will be repeated over a certain interval between them.
Also if you have @reboot or @daily instead of time fields, the command will be run once after the reboot or daily.
Lets see some examples:
1
@reboot      ### on reboot
2
​
3
50 13 * * *  ### 1:50 PM daily
4
​
5
40 6 4 * *   ### 4th of every month at 6:40AM
6
​
7
05 * * 1 0   ### 5 mintues past everyhour,each Sunday in January
8
​
9
0 15 29 11 5 ### 3:00PM Every November 29th  that lands on a Friday
10
​
11
1,5,10 * * * * # 1,5,10 mintues past every hour
12
​
13
30 20 * * 1-5 ## weekdays at 8:20 PM
14
​
15
*/5 * * * *  ### ever 5 mintues
Copied!
user specific crons
/var/spool/cron/
 Users crontab files are stored by the user’s name, and their location varies by operating systems. In Red Hat based system such as CentOS, crontab files are stored in the /var/spool/cron directory while on Debian and Ubuntu files are stored in the /var/spool/cron/crontabs directory.
Although you can edit the user crontab files manually, it is recommended to use the crontab command.
crontab command
The crontab command allows you to install or open a crontab file for editing.
You can use the crontab command to view, add, remove, or modify cron jobs using the following options:
crontab -e : Edit crontab file, or create one if it doesn’t already exist.
crontab -l : Display crontab file contents.
crontab -r : Remove your current crontab file.
crontab -i :Remove your current crontab file with a prompt before removal.
crontab -u <username> : Edit other user crontab file. Requires system administrator privileges.
1
[email protected]:~$ crontab -e
2
no crontab for user1 - using an empty one
3
​
4
Select an editor.  To change later, run 'select-editor'.
5
  1. /bin/ed
6
  2. /bin/nano        <---- easiest
7
  3. /usr/bin/vim.basic
8
  4. /usr/bin/vim.tiny
9
​
10
Choose 1-4 [2]: 3
Copied!
The crontab  -e command opens the crontab file using the editor specified by the VISUAL or EDITOR environment variables.
1
# Edit this file to introduce tasks to be run by cron.
2
# 
3
# Each task to run has to be defined through a single line
4
# indicating with different fields when the task will be run
5
# and what command to run for the task
6
# 
7
# To define the time you can provide concrete values for
8
# minute (m), hour (h), day of month (dom), month (mon),
9
# and day of week (dow) or use '*' in these fields (for 'any').# 
10
# Notice that tasks will be started based on the cron's system
11
# daemon's notion of time and timezones.
12
# 
13
# Output of the crontab jobs (including errors) is sent through
14
# email to the user the crontab file belongs to (unless redirected).
15
# 
16
# For example, you can run a backup of all your user accounts
17
# at 5 a.m every week with:
18
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
19
# 
20
# For more information see the manual pages of crontab(5) and cron(8)
21
# 
22
# m h  dom mon dow   command
23
~                                                                               
24
"/tmp/crontab.f5VKYq/crontab" 22L, 888C                       1,1           All
Copied!
as an example add bellow line to above and it would send and email every 5 mintues:
1
5 * * * * echo "Hello" | mail -s "Cron Test" [email protected]
Copied!
crontab -e also check the syntax before exiting the file , which is really helpful.
crontab -l would show the above contents. Lets check if user crontab file has been created:
1
[email protected]:/var/spool/cron/crontabs# ls -l
2
total 4
3
-rw------- 1 user1 crontab 1154 Feb 15 05:07 user1
Copied!
system wide cron
 In addition to the user crontab files in /var/spool/cron, the cron daemon also checks /etc/crontab and any crontabs in the /etc/cron.d directory. 
/etc/crontab , /etc/cron.d
 /etc/crontab and the files inside the /etc/cron.d directory are system-wide crontab files that can be edited only by the system administrators.
 /etc/crontab is  updated by direct editing. You cannot use the crontab command to update file files or files in the /etc/cron.d directory.
System-wide Crontab Files
The syntax of system-wide crontab files is slightly different than user crontabs. It contains an additional mandatory user field that specifies which user will run the cron job.
1
* * * * * <username> command(s)
Copied!
This file should be edited with an editor directly and we can mention which user runs command(s).
1
# /etc/crontab: system-wide crontab
2
# Unlike any other crontab you don't have to run the `crontab'
3
# command to install the new version when you edit this file
4
# and files in /etc/cron.d. These files also have username fields,
5
# that none of the other crontabs do.
6
​
7
SHELL=/bin/sh
8
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
9
​
10
# m h dom mon dow user  command
11
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
12
25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
13
47 6    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
14
52 6    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
15
#
16
~                                                                               
17
~                                                                               
18
~                                                                                                                                                                                                                                         
19
"/etc/crontab" 15L, 722C                                      14,27-35      All
Copied!
/etc/cron.{daily,hourly,monthly,weekly}/
In most Linux distributions you can also put scripts inside the /etc/cron.{hourly,daily,weekly,monthly} directories and the scripts will be executed every hour/day/week/month.
1
[email protected]:~# tree /etc/cron*
2
/etc/cron.d
3
├── anacron
4
├── php
5
└── popularity-contest
6
/etc/cron.daily
7
├── 0anacron
8
├── apache2
9
├── apport
10
├── apt-compat
11
├── aptitude
12
├── bsdmainutils
13
├── cracklib-runtime
14
├── dpkg
15
├── logrotate
16
├── man-db
17
├── mlocate
18
├── passwd
19
├── popularity-contest
20
├── quota
21
├── samba
22
├── update-notifier-common
23
└── upstart
24
/etc/cron.hourly
25
/etc/cron.monthly
26
└── 0anacron
27
/etc/crontab [error opening dir]
28
/etc/cron.weekly
29
├── 0anacron
30
├── fstrim
31
├── man-db
32
└── update-notifier-common
33
​
34
0 directories, 25 files
Copied!
as an example lets take look at one of them:
1
[email protected]:~# cat /etc/cron.daily/passwd 
2
#!/bin/sh
3
​
4
cd /var/backups || exit 0
5
​
6
for FILE in passwd group shadow gshadow; do
7
        test -f /etc/$FILE              || continue
8
        cmp -s $FILE.bak /etc/$FILE     && continue
9
        cp -p /etc/$FILE $FILE.bak && chmod 600 $FILE.bak
10
done
Copied!
anacron
The cron facility works well for systems that run continuously.If the system is down when the cron should run a task, that cron job wont run till the next occurrence! But anacron creates the timestamp each time a daily, weekly or monthly job runs. 
Note: anacron checks the timestamps at BOOT TIME and does not handle jobs that must run hourly or every minute.
/etc/anacron
 The table of jobs for anacron is stored in /etc/anacrontab, which has a slightly different format from /etc/crontab.
1
[email protected]:/# cat /etc/anacrontab 
2
# /etc/anacrontab: configuration file for anacron
3
​
4
# See anacron(8) and anacrontab(5) for details.
5
​
6
SHELL=/bin/sh
7
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
8
HOME=/root
9
LOGNAME=root
10
​
11
# These replace cron's entries
12
1	5	cron.daily	run-parts --report /etc/cron.daily
13
7	10	cron.weekly	run-parts --report /etc/cron.weekly
14
@monthly	15	cron.monthly	run-parts --report /etc/cron.monthly
Copied!
 just like/etc/crontab , /etc/anacrontab is updated by direct editing. 
Anacrontab Format
1
period   delay   job-identifier   command
Copied!
 period in days : specifies the frequency of execution of a job in  N days.
 delay in minutes: number of minutes anacron should wait before executing the job after reboot.
 job-identifier :It is the name for the job’s timestamp file. It should be unique for each job. This will be available as a file under the /var/spool/anacron directory.
 command: specifies the command to execute. 
/var/spool/anacron
 anacron keeps a time stamp file in /var/spool/anacron for each job to record when the job runs. When anacron runs, it checks to see if the required number of days has passed since a job last ran and runs the job if necessary.
1
[email protected]:/# ls -l /var/spool/anacron/
2
total 12
3
-rw------- 1 root root 9 Feb 15 07:35 cron.daily
4
-rw------- 1 root root 9 Feb  1 07:45 cron.monthly
5
-rw------- 1 root root 9 Feb 10 00:46 cron.weekly
Copied!
This file will contain a single line that indicates the last time when this job was executed.
1
[email protected]:/# cat /var/spool/anacron/cron.daily 
2
20200215
Copied!
at
 Sometimes you need to run a job at a future time just once, rather than regularly. For this purpose you use the at command. (ubuntu: apt install at)
 A typical at command sequence looks like this
1
at 5:45
Copied!
By running at command  It then places you at a special prompt, where you can type in the command (or series of commands) to be run at the scheduled time. When you're done, press Control-D on a new line, and your command will be placed in the queue.
1
[email protected]:~$ at 5:45
2
warning: commands will be executed using /bin/sh
3
at> touch BlaHBlaH 
4
at> <EOT>
5
job 3 at Sat Feb 15 05:45:00 2020
6
​
7
[email protected]:~$ ls -ltrh | grep -i blah
8
-rw-rw-r-- 1 user1 user1    0 Feb 15 05:45 BlaHBlaH
Copied!
warning: commands will be executed using /bin/sh
Some other examples of at command:
1
  Example                  Schedule Task at
2
-----------                -------------------
3
at 10:00 AM                at coming 10:00 AM
4
at 10:00 AM Sun            at 10:00 AM on coming Sunday
5
at 10:00 AM July 25        at 10:00 AM on coming 25’th July
6
at 10:00 AM 6/22/2019      at 10:00 AM on coming 22’nd June 2019
7
at 10:00 AM next month     at 10:00 AM on the same date at next month
8
at 10:00 AM tomorrow       at 10:00 AM tomorrow
9
at teatime                 to execute on next 4:00 pM
10
at midnight                to execute on next 12:00 AM
11
at now + 1 hour            to execute just after 1 hour
12
at now + 1 week            to execute just after 1 week
13
at now + 1 month           to execute just after 1 month
14
at now + 1 year            to execute just after 1 year
Copied!
at command has other members in its family:
atq
lists the pending jobs of users
1
[email protected]:~$ at 06:20
2
warning: commands will be executed using /bin/sh
3
at> touch file1
4
at> <EOT>
5
job 4 at Sat Feb 15 06:20:00 2020
6
[email protected]:~$ at 06:30
7
warning: commands will be executed using /bin/sh
8
at> touch file2
9
at> <EOT>
10
job 5 at Sat Feb 15 06:30:00 2020
11
​
12
[email protected]:~$ atq
13
4	Sat Feb 15 06:20:00 2020 a user1
14
5	Sat Feb 15 06:30:00 2020 a user1
Copied!
atrm 
 delete jobs by their job number
1
[email protected]:~$ atrm 5
2
[email protected]:~$ atq
3
4	Sat Feb 15 06:20:00 2020 a user1
Copied!
  atq command only shows the list of jobs but if you want to check what script/commands are scheduled with that task use at -c JobNum command and see the last line.
both cron and are system services.
Configure user access to job scheduling
We can control access to the crontab command by using two files in the /etc/cron.d directory: cron.deny and cron.allow. These files permit only specified users to perform crontab command tasks such as creating, editing, displaying, or removing their own crontab files.
The cron.deny and cron.allow files consist of a list of user names, one user name per line.
/etc/cron.allow , /etc/cron.deny
These access control files work together as follows:
If cron.allow exists, only the users who are listed in this file can create, edit, display, or remove crontab files.
If cron.allow does not exist, all users can submit crontab files, except for users who are listed in cron.deny.
If neither cron.allow nor cron.deny exists, superuser privileges are required to run the crontab command.
Superuser privileges are required to edit or create the cron.deny and cron.allow files.
1
[email protected]:~# cat /etc/cron.deny
2
user2
Copied!
 /etc/at.allow , /etc/at.deny
The corresponding /etc/at.allow and /etc/at.deny files have similar effects for the at facility.
.
.
.
Crontab Variables
The cron daemon automatically sets several environment variables.
The default path is set to PATH=/usr/bin:/bin. If the command you are calling is present in the cron specified path, you can either use the absolute path to the command or change the cron $PATH variable. You can’t implicitly append :$PATH as you would do with a regular script.
The default shell is set to /bin/sh. You can set a different shell by changing the SHELL variable.
Cron invokes the command from the user’s home directory. The HOME variable can be overridden by settings in the crontab.
The email notification is sent to the owner of the crontab. To overwrite the default behavior, you can use the MAILTO environment variable with a list (comma separated) of all the email addresses you want to receive the email notifications. If MAILTO is defined but empty (MAILTO=""), no mail is sent.
.
.
​https://developer.ibm.com/tutorials/l-lpic1-107-2/​
​https://linuxize.com/post/scheduling-cron-jobs-with-crontab/​
​https://jadi.gitbooks.io/lpic1/content/1072_automate_system_administration_tasks_by_scheduling_jobs.html​
​https://www.thegeekdiary.com/centos-rhel-anacron-basics-what-is-anacron-and-how-to-configure-it/​
​https://www.thegeekstuff.com/2011/05/anacron-examples/​
​https://www.computerhope.com/unix/uat.htm​
​https://tecadmin.net/one-time-task-scheduling-using-at-commad-in-linux/​
​https://docs.oracle.com/cd/E19253-01/817-0403/sysrescron-23/index.html​
.