103.3. Perform basic file management

103.3 Perform basic file management
Weight: 4
Description: Candidates should be able to use the basic Linux commands to manage files and directories.
Key Knowledge Areas:
Copy, move and remove files and directories individually
Copy multiple files and directories recursively
Remove files and directories recursively
Use simple and advanced wildcard specifications in commands
Using find to locate and act on files based on type, size, or time
Usage of tar, cpio and dd
Terms and Utilities:
cp
find
mkdir
mv
ls
rm
rmdir
touch
tar
cpio
dd
file
gzip
gunzip
bzip2
xz
file globbing
As we said Linux is the world of processes and files, in this section we start talking about file management in linux and we will talk about both files and directories (folders). Tools are pretty basic and their names are abstraction of what they really do.
ls
ls lists files and directories, and their associated metadata, such as file size, ownership, and modification time.
1
ls [options] [file|dir]
Copied!
We can use absolute paths (ex:[email protected]:~$ ls /home/user1/Music) or relative paths (ex: [email protected]:~$ ls Music/) with ls.
With no options, ls lists the files contained in the current directory, sorting them alphabetically.
1
[email protected]:~$ ls
2
Desktop    Downloads         Music     Public     Videos
3
Documents  examples.desktop  Pictures  Templates
Copied!
 -1 prints outs each result in 1 line:
1
[email protected]:~$ ls -1
2
Desktop
3
Documents
4
Downloads
5
examples.desktop
6
Music
7
Pictures
8
Public
9
Templates
10
Videos
Copied!
The default output of the ls command shows only the names of the files, which is not very informative. So lets use ls with -l for long listing format :
1
[email protected]:~$ ls -l
2
total 44
3
drwxr-xr-x 2 user1 user1 4096 Nov  4  2018 Desktop
4
drwxr-xr-x 2 user1 user1 4096 Nov  4  2018 Documents
5
drwxr-xr-x 2 user1 user1 4096 Nov  4  2018 Downloads
6
-rw-r--r-- 1 user1 user1 8980 Apr 20  2016 examples.desktop
7
drwxr-xr-x 2 user1 user1 4096 Nov  4  2018 Music
8
drwxr-xr-x 2 user1 user1 4096 Nov  4  2018 Pictures
9
drwxr-xr-x 2 user1 user1 4096 Nov  4  2018 Public
10
drwxr-xr-x 2 user1 user1 4096 Nov  4  2018 Templates
11
drwxr-xr-x 2 user1 user1 4096 Nov  4  2018 Videos
Copied!
The first character represents the file type: "-" for a regular file, "d" for a directory, "l" for a symbolic link(we will see them).
When the long listing format is used the ls command will display the following file information:
1.
The file type
2.
The file permissions
3.
Number of hard links to the file
4.
File owner
5.
File group
6.
File size
7.
Date and Time
8.
File name
Show Hidden Files
By default, the ls command will not show hidden files. In Linux, a hidden file is any file that begins with a dot (.).To display all files including the hidden files use the -a option:
1
[email protected]:~$ ls -a
2
.              .config           .gconf         Pictures   .Xauthority
3
..             Desktop           .gnupg         .profile   .xsession-errors
4
.bash_history  .dmrc             .ICEauthority  Public     .xsession-errors.old
5
.bash_logout   Documents         .local         .ssh
6
.bashrc        Downloads         .mozilla       Templates
7
.cache         examples.desktop  Music          Videos
Copied!
Some other usefull options:
ls command options
description
-l
list with long format - show permissions
-r
list in reverse order
-t
sort by time & date
-lh
List Files with Human Readable Format
-F
List Files and Directories with ‘/’ Character at the end
-R
Recursively list Sub-Directories
-a
list all files including hidden file starting with '.'
Copying, Moving & Deleting
To make it easier lets classfied them in some groups:
Before going into how to use the touch command, let’s start by reviewing the file timestamps in Linux.
Linux Files Timestamps
In Linux every single file is associated with timestamps, and every file stores the information of last access time, last modification time and last change time. So, whenever we create new file, access or modify an existing file, the timestamps of that file automatically updated.
A file in Linux has three timestamps:
atime (access time) - The last time the file was accessed/opened by some command or application such as cat, vim or grep.
mtime (modify time) - The last time the file’s content was modified.
ctime (change time) - The last time the file’s attribute or content was changed. The attribute includes file permissions, file ownership or file location.
To display the file status including the timestamps you can use the stat command.(we use ubuntu 16.04 here)
1
[email protected]:~# stat /etc/hosts
2
  File: '/etc/hosts'
3
  Size: 247           Blocks: 8          IO Block: 4096   regular file
4
Device: 801h/2049d    Inode: 1200795     Links: 1
5
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
6
Access: 2019-09-08 02:34:44.962514623 -0700
7
Modify: 2018-07-01 23:15:30.140815060 -0700
8
Change: 2018-07-01 23:15:30.144817112 -0700
9
 Birth: -
Copied!
touch
The touch command is a standard command used in UNIX/Linux operating system which is used to create, change and modify timestamps of a file.
touch command (with no option) create an empty file if file does not exist.
if file exists touch command would change its time stamps .
creating an empty file:
touch command is used to create a file without any content. The file created using touch command is empty. This command can be used when the user doesn’t have data to store at the time of file creation.
1
touch file_name
Copied!
example:
1
[email protected]:~/test-space# touch file1
2
[email protected]:~/test-space# ls -l
3
total 0
4
-rw-r--r-- 1 root root 0 Sep  8 02:37 file1
Copied!
creating multiple empty files:
1
touch File1_name File2_name File3_name
Copied!
for example:
1
[email protected]:~/test-space# ls -l
2
total 0
3
-rw-r--r-- 1 root root 0 Sep  8 02:37 file1
4
[email protected]:~/test-space# touch file1 file2 file3
5
[email protected]:~/test-space# ls -l
6
total 0
7
-rw-r--r-- 1 root root 0 Sep  8 02:44 file1
8
-rw-r--r-- 1 root root 0 Sep  8 02:44 file2
9
-rw-r--r-- 1 root root 0 Sep  8 02:44 file3
Copied!
 avoids creating new files:
1
[email protected]:~/test-space# touch -c file4
2
[email protected]:~/test-space# ls -l
3
total 0
4
-rw-r--r-- 1 root root 0 Sep  8 02:44 file1
5
-rw-r--r-- 1 root root 0 Sep  8 02:44 file2
6
-rw-r--r-- 1 root root 0 Sep  8 02:44 file3
Copied!
-coption tells that if the file does not exist, do not create it:
touch command timestamp options:
-a: change the access time only
-d: update the access and modification times
-m: change the modification time only
-r: use the access and modification times of other file as reference.
-t: creates a file using a specified time
Lets do some timestamp modifications.
Create a file using a specified time:touch -t YYMMDDHHMM fileName
1
[email protected]:~/test-space# touch -t 9812010510 myfile
2
[email protected]:~/test-space# ls -l | grep myfile
3
-rw-r--r-- 1 root root 0 Dec  1  1998 myfile
4
[email protected]:~/test-space# stat myfile
5
  File: 'myfile'
6
  Size: 0             Blocks: 0          IO Block: 4096   regular empty file
7
Device: 801h/2049d    Inode: 2228284     Links: 1
8
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
9
Access: 1998-12-01 05:10:00.000000000 -0800
10
Modify: 1998-12-01 05:10:00.000000000 -0800
11
Change: 2019-09-08 03:19:28.410459835 -0700
12
 Birth: -
Copied!
Change the access time:touch -a fileName
1
[email protected]:~/test-space# touch -a  myfile
Copied!
It only updates last access time.
1
[email protected]:~/test-space# stat myfile
2
  File: 'myfile'
3
  Size: 0             Blocks: 0          IO Block: 4096   regular empty file
4
Device: 801h/2049d    Inode: 2228284     Links: 1
5
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
6
Access: 2019-09-08 03:26:42.378185145 -0700
7
Modify: 1998-12-01 05:10:00.000000000 -0800
8
Change: 2019-09-08 03:26:42.378185145 -0700
Copied!
Change the modification time:touch -m fileName
1
[email protected]:~/test-space# touch -m myfile
Copied!
It only updates last modification time.
1
[email protected]:~/test-space# stat myfile 
2
  File: 'myfile'
3
  Size: 0             Blocks: 0          IO Block: 4096   regular empty file
4
Device: 801h/2049d    Inode: 2228284     Links: 1
5
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
6
Access: 2019-09-08 03:26:42.378185145 -0700
7
Modify: 2019-09-08 03:28:06.314537292 -0700
8
Change: 2019-09-08 03:28:06.314537292 -0700
9
 Birth: -
Copied!
Explicitly Set the Access and Modification times: touch -c -t YYDDHHMM filename
Update access and modification time:touch -c -d fileName
We use the timestamp of another file with -r option: touch -r refrence_file_name file_name
1
[email protected]:~/test-space# stat file1
2
  File: 'file1'
3
  Size: 0             Blocks: 0          IO Block: 4096   regular empty file
4
Device: 801h/2049d    Inode: 2228281     Links: 1
5
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
6
Access: 2019-09-08 02:44:27.744736670 -0700
7
Modify: 2019-09-08 02:44:27.744736670 -0700
8
Change: 2019-09-08 02:44:27.744736670 -0700
9
 Birth: -
10
[email protected]:~/test-space# touch -r file1 myfile 
11
[email protected]:~/test-space# stat myfile 
12
  File: 'myfile'
13
  Size: 0             Blocks: 0          IO Block: 4096   regular empty file
14
Device: 801h/2049d    Inode: 2228284     Links: 1
15
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
16
Access: 2019-09-08 02:44:27.744736670 -0700
17
Modify: 2019-09-08 02:44:27.744736670 -0700
18
Change: 2019-09-08 03:34:14.724762374 -0700
19
 Birth: -
Copied!
and many more examples.
mkdir
mkdir command in Linux allows the user to create directories (also referred to as folders in some operating systems )
1
mkdir [options...] [directories ...]
Copied!
1
[email protected]:~/test-space# mkdir dir1
2
[email protected]:~/test-space# ls -l
3
total 4
4
drwxr-xr-x 2 root root 4096 Sep  8 03:44 dir1
Copied!
This command can create multiple directories at once as well as set the permissions for the directories. It is important to note that the user executing this command must have enough permissions to create a directory in the parent directory, or he/she may recieve a ‘permission denied’ error.
1
[email protected]:~/test-space# mkdir --help
2
Usage: mkdir [OPTION]... DIRECTORY...
3
Create the DIRECTORY(ies), if they do not already exist.
4
​
5
Mandatory arguments to long options are mandatory for short options too.
6
  -m, --mode=MODE   set file mode (as in chmod), not a=rwx - umask
7
  -p, --parents     no error if existing, make parent directories as needed
8
  -v, --verbose     print a message for each created directory
9
  -Z                   set SELinux security context of each created directory
10
                         to the default type
11
      --context[=CTX]  like -Z, or if CTX is specified then set the SELinux
12
                         or SMACK security context to CTX
13
      --help     display this help and exit
14
      --version  output version information and exit
15
​
16
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
17
Full documentation at: <http://www.gnu.org/software/coreutils/mkdir>
18
or available locally via: info '(coreutils) mkdir invocation'
Copied!
Lets try some of the most useful switches, -penables the command to create parent directories as necessary. If the directories exist, no error is specified:
1
[email protected]:~/test-space# mkdir -p dir1/floor1/floor2
2
[email protected]:~/test-space# tree
3
.
4
└── dir1
5
    └── floor1
6
        └── floor2
7
​
8
3 directories, 0 files
Copied!
We can create a directory and set the permissions(will be discussed later) for that directory at the same time using -m option:
1
[email protected]:~/test-space# mkdir -m777 dir2 
2
[email protected]:~/test-space# ls -l
3
total 8
4
drwxr-xr-x 3 root root 4096 Sep  8 03:57 dir1
5
drwxrwxrwx 2 root root 4096 Sep  8 04:09 dir2
Copied!
The syntax of the mode is the same as the chmod command (will be discussed later).
1
[email protected]:~/test-space# mkdir -v -m o=--- dir3
2
mkdir: created directory 'dir3'
3
[email protected]:~/test-space# ls -l
4
total 12
5
drwxr-xr-x 3 root root 4096 Sep  8 03:57 dir1
6
drwxrwxrwx 2 root root 4096 Sep  8 04:09 dir2
7
drwxr-x--- 2 root root 4096 Sep  8 04:14 dir3
Copied!
with -voption, it prints a message for each created directory.
cp
The cp command is a command-line utility for copying files and directories.  [Copies of files are independent of the original file( unlike the mv command)  ].The basic syntax of the cp command is:
1
cp [options….] source(s) destination
Copied!
cp can take 1 or more sources(s) but just one destination.
It supports copying one or more files or directories with options for taking backups and preserving attributes.
Do not forget to consider source and destination types (files or directory) when using cp command.
If the target is an existing directory, then all sources are copied into the target. 
If the target is a directory that does not exist, then the (single) source must also be a directory and a copy of the source directory and its contents is made with the target name as the new name. 
If the target is a file, then the (single) source must also be a file and a copy of the source file is made with the target name as the new name, replacing any existing file of the same name.
lets do some examples:
1
[email protected]:~/test-space# tree
2
.
3
├── dir1
4
│   └── floor1
5
│       └── floor2
6
├── dir2
7
├── dir3
8
└── file1
9
​
10
5 directories, 1 file
Copied!
copy a file:
1
[email protected]:~/test-space# cp file1 file2
2
[email protected]:~/test-space# ls
3
dir1  dir2  dir3  file1  file2
Copied!
take a backup when copying a file
If a copy operation will overwrite a file the -b flag may be used to create a back up of the file. This copies the file into place and writes a backup file.
1
[email protected]:~/test-space# ls
2
dir1  dir2  dir3   file1  file2  
3
[email protected]:~/test-space# echo "this is file2 content" > file2
4
[email protected]:~/test-space# echo "ehis is file1 content" > file1
5
[email protected]:~/test-space# cp -b file1 file2
6
[email protected]:~/test-space# ls
7
dir1  dir2  dir3   file1  file2  file2~ 
8
[email protected]:~/test-space# cat file2
9
ehis is file1 content
10
[email protected]:~/test-space# cat file2~
11
this is file2 content
Copied!
specify the name of the backup file use the -S option. try cp -S .file2bak file1 file2
copy multiple files (into a directory)
1
[email protected]:~/test-space# cp file1 file2 dir2/
2
[email protected]:~/test-space# tree
3
.
4
├── dir1
5
│   └── floor1
6
│       └── floor2
7
├── dir2
8
│   ├── file1
9
│   └── file2
10
├── dir3
11
├── file1
12
├── file2
13
└── file2~
14
​
15
​
16
5 directories, 4 files
Copied!
Copying files recursively
We can use cp to copy entire directory structures from one place to another using the -R option to perform a recursive copy.
 Let's say you are the user root and you have a directory, /home/test-space, which contains many files and subdirectories. You want to copy all those files, and all the subdirectories (and the files and subdirectories they contain), to a new location, /root/files-backup. You can copy all of them using the command:
cp -R ~/test-space ~/files-backup
If the directory files-backup already exists, the directory files will be placed inside.
If files-backup does not already exist, it will be created and the contents of the files directory will be placed inside it.
copy a directory
By default the cp command will not copy directories. Attempting to copy a directory results in an error.
1
[email protected]:~/test-space# cp dir2 dir4
2
cp: omitting directory 'dir2'
Copied!
To copy a directory pass the -R flag. This will recursively copy a folder and create a copy.
1
[email protected]:~/test-space# cp -R dir1 NewDirectoryA
2
[email protected]:~/test-space# tree
3
.
4
├── dir1
5
│   └── floor1
6
│       └── floor2
7
├── dir2
8
│   ├── file1
9
│   └── file2
10
├── dir3
11
├── file1
12
├── file2
13
├── file2~
14
└── NewDirectoryA
15
    └── floor1
16
        └── floor2
17
​
18
8 directories, 5 files
Copied!
copy multiple directories
To copy multiple directories pass the path of the directories to be copied followed by the destination directory.
1
[email protected]:~/test-space# cp -R dir1 dir2 NewDirectoryB
2
cp: target 'NewDirectoryB' is not a directory
3
r[email protected]:~/test-space# mkdir NewDirectoryB
4
[email protected]:~/test-space# cp -R dir1 dir2 NewDirectoryB
5
[email protected]:~/test-space# tree
6
.
7
├── dir1
8
│   └── floor1
9
│       └── floor2
10
├── dir2
11
│   ├── file1
12
│   └── file2
13
├── dir3
14
├── file1
15
├── file2
16
├── file2~
17
├── NewDirectoryA
18
│   └── floor1
19
│       └── floor2
20
└── NewDirectoryB
21
    ├── dir1
22
    │   └── floor1
23
    │       └── floor2
24
    └── dir2
25
        ├── file1
26
        └── file2
27
​
28
13 directories, 7 files
Copied!
Now lets take a look at some cp command options:
cp command useful options:
Description
-v,--verbose
Verbose mode; explain what is being done.
--preserve
preserve file attributes (permissions, group and user owernship).By default mode, ownership and timestamps will be preserved.
-i , --interactive
Prompt before overwrite (overrides a previous -n option).
-n , --no-clobber
Do not overwrite an existing file. If -i/--interactive was previously specified, this option overrides it. This option cannot be specified with -b/--backup, because backups are only created when a file would have been overwritten.
-f,--force
If an existing destination file cannot be opened, remove it and try again. This option has no effect if the -n/--no-clobber option is used. However, it applies independently of -i/--interactive; neither option cancels the effect of the other.
cp command has lots of options.Try man cp command for more information.
mv
mv is used to move or rename one or more files or directories. In general, the names we can use follow the same rules as for copying with cp; [If you are moving a file on the same file system, the inode wont change]. 
Rename a file( or directory) name source to destination:
1
mv [options]  source destination
Copied!
Move source file(s) or directory(s) to a directory named destination:
1
mv [options] source [source2 ...] destination
Copied!
Same as the previous syntax, but specifying the directory first, and the source file(s)(or directory(s)) last
1
mv [options] -t destination source [source2 ...]
Copied!
Lets try mv command:
1
[email protected]:~/test-space# tree -F
2
.
3
├── dir1/
4
├── dir2/
5
├── file1
Copied!
Renaming a File or a directory:
1
[email protected]:~/test-space# mv file1 NewFile1
2
[email protected]:~/test-space# mv dir1 dirA
3
[email protected]:~/test-space# tree -F
4
.
5
├── dir2/
6
├── dirA/
7
├── file2
8
└── NewFile1
Copied!
Moving Files into a directory:
1
[email protected]:~/test-space# mv NewFile1 file2 dir3
2
mv: target 'dir3' is not a directory
3
[email protected]:~/test-space# mkdir dir3
4
[email protected]:~/test-space# mv NewFile1 file2 dir3
5
[email protected]:~/test-space# tree -F
6
.
7
├── dir2/
8
├── dir3/
9
│   ├── file2
10
│   └── NewFile1
11
└── dirA/
12
​
13
3 directories, 2 files
Copied!
Moving a directory into another :
1
[email protected]:~/test-space# mv dir3 dirA
2
[email protected]:~/test-space# tree -F
3
.
4
├── dir2/
5
└── dirA/
6
    └── dir3/
7
        ├── file2
8
        └── NewFile1
9
​
10
3 directories, 2 files
Copied!
we could usemv -t dirA dir3 command as well. -t, --target-directoryMove all sources into the directory destination.
usefull mv command options:
mv command options
description
-v
verbose - print source and destination files
-i
interactive prompt before overwrite
-u
update - move when source is newer than destination
-f
force move by overwriting destination file without prompt
cp vs mv :
Normally, the cp command will copy a file over an existing copy, if the existing file is writable. On the other hand, the mv will not move or rename a file if the target exists. We can overcome this using the-fswitch.
rm
rm stands for remove here. rm removes files or directories.
1
rm [OPTION]... FILE...
Copied!
Lets try:
1
[email protected]:~/test-space# tree -F
2
.
3
├── dir1/
4
├── dir2/
5
│   └── MyFile
6
└── file1
7
​
8
2 directories, 2 files
Copied!
Removing Files
1
[email protected]:~/test-space# rm file1
2
[email protected]:~/test-space# tree -F
3
.
4
├── dir1/
5
└── dir2/
6
    └── MyFile
7
​
8
2 directories, 1 file
Copied!
File names starting with a dash, How to remove it?
To remove a file whose name begins with a dash ("-"), you can specify a double dash ("--") separately before the file name. This extra dash is necessary so that rm does not misinterpret the file name as an option: rm -- -file.txtOr, we can delete it by referring to it with a pathname : rm /home/hope/-file.txt
some other options of rm command:
mv options
description
-v
Verbose mode; explain at all times what is being done.
-i
prompt before every removal
-f,--force
Ignore none existant files,and never prompt before removing.
-d,--dir
Remove empty directories.
Removing directories
1
[email protected]:~/test-space# rm dir1
2
rm: cannot remove 'dir1': Is a directory
Copied!
By default, rm does not remove directories.
If the specified directory is empty, it may be removed with the -d/--dir option, instead.
1
[email protected]:~/test-space# rm -d dir1
2
[email protected]:~/test-space# tree -F
3
.
4
└── dir2/
5
    └── MyFile
6
​
7
1 directory, 1 file
Copied!
What if desired directory contains some files? If the -r/-R/--recursive option is specified, we can remove that directory however rm will remove any matching directories and their contents!
1
[email protected]:~/test-space# rm -rf dir2
2
[email protected]:~/test-space# tree -F
3
.
4
​
5
0 directories, 0 files
Copied!
rm -dlets us to remove a directory without specifying -r/-R/--recursive, provided that the directory is empty. In other words, rm -d is equivalent to using rmdir.
rmdir
Removing directories using the rmdir command is the opposite of creating them. We can remove a directory with rmdir only if it is empty as there is no option to force removal.
1
rmdir [options] directory ...
Copied!
Again, there is a-p option to remove parents as well. Let try it:
1
[email protected]:~/test-space# tree -F
2
.
3
├── dir1/
4
│   └── file1
5
├── dir2/
6
└── dir3/
7
    └── dirA/
8
        └── dirB/
9
​
10
5 directories, 1 file
11
[email protected]:~/test-space# rmdir dir1
12
rmdir: failed to remove 'dir1': Directory not empty
13
[email protected]:~/test-space# rmdir dir2
14
[email protected]:~/test-space# tree -F
15
.
16
├── dir1/
17
│   └── file1
18
└── dir3/
19
    └── dirA/
20
        └── dirB/
21
​
22
4 directories, 1 file
23
[email protected]:~/test-space# rmdir -p dir3/dirA/
24
[email protected]:~/test-space# rmdir -p dir3/dirA/dirB/
25
[email protected]:~/test-space# tree -F
26
.
27
└── dir1/
28
    └── file1
29
​
30
1 directory, 1 file
Copied!
Normally, whenrmdir is instructed to remove a non-empty directory, it reports an error. With --ignore-fail-on-non-emptyoption suppresses those error messages.
Handling multiple files and directories
Some time we need to work on more than one files, Now we try to have review over some recursive commands
Recursive manipulation
Recursive listing
The ls command has a -R (note uppercase “R”) option for listing a directory and all its subdirectories. The recursive option applies only to directory names, for example, in a directory tree.
1
[email protected]:~/test-space# tree
2
.
3
├── dir1
4
│   └── text1
5
├── dir2
6
├── file1
7
└── file2
8
​
9
2 directories, 3 files
10
[email protected]:~/test-space# ls -R
11
.:
12
dir1  dir2  file1  file2
13
​
14
./dir1:
15
text1
16
​
17
./dir2:
Copied!
Recursive copy
We can use the -r (or -R or --recursive) option to cause the cp command to descend into source directories and copy the contents recursively. To prevent an infinite recursion, we cannot copy the source directory itself!
1
[email protected]:~# cp -R test-space NewCopy
2
[email protected]:~# tree NewCopy/
3
NewCopy/
4
├── dir1
5
│   └── text1
6
├── dir2
7
├── file1
8
└── file2
9
​
10
2 directories, 3 files
Copied!
Recursive deletion
I mentioned earlier that rmdir only removes empty directories. We can use the -r (or -R or --recursive) option to cause the rm command to remove both files and directories:
1
[email protected]:~# ls
2
client.conf             NewCopy      unzip_6.0-20ubuntu1_amd64.deb
3
files-backup            openvpn.key  zip_3.0-11_amd64.deb
4
jcal_0.4.1-2_amd64.deb  test-space
5
[email protected]:~# rm -rf NewCopy/
6
[email protected]:~# ls
7
client.conf             openvpn.key                    zip_3.0-11_amd64.deb
8
files-backup            test-space
9
jcal_0.4.1-2_amd64.deb  unzip_6.0-20ubuntu1_amd64.deb
Copied!
Wildcards and Globbing
File globbing is a feature provided by the UNIX/Linux shell to represent multiple filenames by using special characters called wildcards with a single file name. A wildcard is essentially a symbol which may be used to substitute for one or more characters. Therefore, we can use wildcards for generating the appropriate combination of file names as per our requirement.
1
[email protected]:~/test-space# ls -l
2
total 0
3
-rw-r--r-- 1 root root 0 Sep 28 00:26 aaa.txt
4
-rw-r--r-- 1 root root 0 Sep 28 00:27 ababa.txt
5
-rw-r--r-- 1 root root 0 Sep 28 00:26 bbbb.txt
6
-rw-r--r-- 1 root root 0 Sep 27 23:33 file1
7
-rw-r--r-- 1 root root 0 Sep 27 23:33 file2
Copied!
? is used to match any single character. We can use ‘?’ for multiple times for matching multiple characters.
1
[email protected]:~/test-space# ls ???.txt
2
aaa.txt
Copied!
* is used to match zero or more characters. If we have less information to search any file or information then we can use ‘*’ in globbing pattern.
1
[email protected]:~/test-space# ls *.txt
2
aaa.txt  ababa.txt  bbbb.txt
Copied!
[ ]is used to match the character from the range. Some of the mostly used range declarations are mentioned below:
[A-Z] : All uppercase alphabets
[a-z] : All lowercase alphabets
[a-zA-Z0-9] : All uppercase alphabets, lowercase alphabet and digits
The-character between two others represents a range that includes the two other characters and all characters between them in the collating sequence.
1
[email protected]:~/test-space# ls [e-z]*
2
file1  file2
Copied!
The! character means NOT so  it matches any character except the remaining characters.
1
[email protected]:~/test-space# ls [!e-z]*
2
aaa.txt  ababa.txt  bbbb.txt
Copied!
{ }can be used to match filenames with more than one globbing patterns. Each pattern is separated by ‘,’ in curly bracket without any space.
1
[email protected]:~/test-space# ls {???.txt,????.txt}
2
aaa.txt  bbbb.txt
Copied!
rm {*.doc,*.docx} : delete all files whose extensions are ‘doc’ or ‘docx’.
and finally \ is used as an "escape" character, we have used it to protect a subsequent special character. example: "\\” searches for a backslash
We can disable globbing using set -f command.
Wildcard patterns vs Regular Expressions
Wildcard patterns and regular expression patterns share some characteristics, but they are not the same. Pay careful attention.
Now that we’ve covered the file and directory topic with the big recursive hammer that hits everything, and the globbing hammer that hits more selectively, let’s look at the find command, which can be more like a surgeon’s knife.
Finding Files
 The find command is used to find files in one or more directory trees, based on criteria such as name, time stamp, or size.
find
The find command searches for files or directories using all or part of the name, or by other search criteria, such as size, type, file owner, creation date, or last access date.
1
find starting/path options  expression
Copied!
1.
The starting/path attribute will define the top level directory where find begins filtering.
2.
The optionsattribute will control the behavior and optimization method of the find process.
3.
The expression attribute controls the tests that search the directory hierarchy to produce output.
The most basic find is a search by name or part of a name:
1
[email protected]:~/test-space# find /etc/ -iname "[y-z]*"
2
/etc/xdg/autostart/zeitgeist-datahub.desktop
3
/etc/zsh_command_not_found
4
/etc/vmware-tools/messages/zh_TW
5
/etc/vmware-tools/messages/zh_CN
6
/etc/dhcp/dhclient-exit-hooks.d/zzz_avahi-autoipd
7
/etc/kernel/postrm.d/zz-update-grub
8
/etc/kernel/postinst.d/zz-update-grub
9
/etc/brltty/Contraction/zu.ctb
10
/etc/brltty/Contraction/zh-tw-ucb.ctb
11
/etc/brltty/Contraction/zh-tw.ctb
Copied!
-name option used for searching for files based on their name. -imakes it case insensitive. In the first example above, we found both files and a directory (/etc). 
finding hidden files : If you want to find a file or directory whose name begins with a dot, such as .bashrc or the current directory (.), then you must specify a leading dot as part of the pattern. Otherwise, name searches ignore these files or directories. find . -name ".*"
note: If you want to chain different results together, you can use the “-and” or “-or” commands. The “-and” is assumed if omitted. find . -name file1 -or -name file9
Finding files by type
We can specify the type of files you want to find with the “-type” parameter. It works like this:
1
find -type type_descriptor query
Copied!
-type fwill search for a regular file
-type d will search for a directory
-type l will search for a symbolic link
1
[email protected]:~/test-space# find /etc/ -type f -iname "[y-z]*"
2
/etc/xdg/autostart/zeitgeist-datahub.desktop
3
/etc/zsh_command_not_found
4
/etc/dhcp/dhclient-exit-hooks.d/zzz_avahi-autoipd
5
/etc/kernel/postrm.d/zz-update-grub
6
/etc/kernel/postinst.d/zz-update-grub
7
/etc/brltty/Contraction/zu.ctb
8
/etc/brltty/Contraction/zh-tw-ucb.ctb
9
/etc/brltty/Contraction/zh-tw.ctb
Copied!
Finding files by size
We can also search by file size, either for a specific size (n) or for files that are either larger (+n) or smaller than a given value (-n). By using both upper and lower size bounds, we can find files whose size is within a given range. By default, the -size option of find assumes a unit of ‘b’ for 512-byte blocks.
-size  +/- [b] [c] [w] [k] [M} [G]
1
b     for 512-byte blocks (this is the default if no suffix is used)
2
c     for bytes
3
w     for two-byte words
4
k     for Kilobytes (units of 1024 bytes)
5
M     for Megabytes (units of 1048576 bytes)
6
G     for Gigabytes (units of 1073741824 bytes)
Copied!
as an example lets find files smaller than 1 kilobytes:
1
[email protected]:~/test-space# find /etc/ -type f -size -1k
2
/etc/dictionaries-common/ispell-default
3
/etc/cups/printers.conf
4
/etc/vmware-caf/pme/config/_previous_/cafenv.config
5
/etc/.pwd.lock
6
/etc/apt/trusted.gpg.d/peek-developers_ubuntu_stable.gpg~
7
/etc/security/opasswd
8
/etc/gconf/gconf.xml.defaults/%gconf-tree.xml
9
/etc/gconf/gconf.xml.mandatory/%gconf-tree.xml
10
/etc/sensors.d/.placeholder
11
/etc/newt/palette.original
Copied!
We can find all empty files using find . -size 0b or find . -empty .
Finding files based on their time:
Linux stores time data about access times, modification times, and change times.
Access Time: Last time a file was read or written to.
Modification Time: Last time the contents of the file were modified.
Change Time: Last time the file’s inode meta-data was changed.
We can use the time stamps described with the touch command to locate files having particular time stamps.
1
-amin n: The file was last accessed n minutes ago
2
-anewer: The file was last accessed more recently than it was modified
3
-atime n: The file was last accessed more n days ago
4
-cmin n: The file was last changed n minutes ago
5
-cnewer: The file was last changed more recently than the file was modified
6
-ctime n: The file was last changed more than n days ago
7
-mmin n: File's data was last modified n minutes ago
8
-mtime n: File's data was last modified n days ago
Copied!
again (+/-) signs can be used to give it a range.
Find Changed Files in Last 2 Hours:
1
[email protected]:~/test-space# find /home -cmin -120
2
/home/user1/.cache/upstart
3
/home/user1/.cache/upstart/indicator-printers.log
4
/home/user1/.cache/upstart/unity-panel-service.log
Copied!
note : Adding the -daystart option to -mtime or -atime means that we want to consider days as calendar days, starting at midnight. So to list the regular files in your home directory that were modified yesterday we can use find ~/ -daystart -type f -mtime 1 .
We can also find files by owner and permissions and use filter result by depth (will be discussed in later sections"104-7")
Acting on files with two other switches:
find command switch
meaning
-ls
list current file in ls -dils format on standard output
-print
print the full file name on the standard output
As you can see find command has tons of options, get into more details using man page files.
Executing and Combining Find Commands (-exec)
We can execute an arbitrary helper command on everything that find matches by using the “-exec” parameter. This is called like this:
1
find find_parameters -exec command_and_params {} \;
Copied!
The {} is used as a placeholder for the files that find matches. The “\;” is used so that find knows where the command ends.
As an instance this will remove all empty files:
1
find . -empty -exec rm {} \;
Copied!
We could remove all empty files in this directory and its subdirectories:
1
find . -empty -exec rm '{}' \;
Copied!
We could then change the directory permissions like this:
1
find . -name "*.deb" -exec cp {} /tmp/packages \;
Copied!
Identifying files
File names often have a suffix such as gif, jpeg, or html that gives a hint of what the file might contain. Linux does not require such suffixes and generally does not use them to identify a file type. Knowing what type of file you are dealing with helps you know what program to use to display or manipulate it.
file
The file command tells us something about the type of data in one or more files.
1
file [option] [filename]
Copied!
1
[email protected]:~/test-space# ls -l
2
total 10880
3
-rwxrw-rw- 1 user1 user1 10819475 Mar 29  2015 01 - Intro.mp4
4
-rw------- 1 user1 user1   221941 Sep  1 00:24 zabbix-cli-manual-1.7.0.pdf
5
-rw------- 1 user1 user1    91087 Aug 31 00:40 zabbix-cli-rpm-2.1.1-1.tar.gz
6
​
7
[email protected]:~/test-space# file 01\ -\ Intro.mp4 
8
01 - Intro.mp4: ISO Media, MP4 Base Media v1 [IS0 14496-12:2003]
Copied!
-b, –brief : This is used to display just file type in brief mode.
1
[email protected]:~/test-space# file  -b zabbix-cli-manual-1.7.0.pdf 
2
PDF document, version 1.4
Copied!
-z: Try to look inside compressed files.
1
[email protected]:~/test-space# file -z zabbix-cli-rpm-2.1.1-1.tar.gz zabbix-cli-rpm-2.1.1-1.tar.gz:
2
POSIX tar archive (gzip compressed data, from Unix)
Copied!
and -ioption  To view mime type of file.
A MIME type is a label used to identify a type of data. It is used so software can know how to handle the data. It serves the same purpose on the Internet that file extensions do on Microsoft Windows.
Archiving and Compressing files
Archiving and compressing are two different things
tar just archive files and does not do any compression by default.
zip does both archiving and compresion.
gzip and bzip2 are used just for compression.
zip
Zip is one of the most popular archive file format out there. With zip, you can compress multiple files into one file. This not only saves disk space, it also saves network bandwidth. This is why you’ll encounter zip files almost all the time.
1
zip options archive inpath inpath ...
Copied!
Lets take a look a look:
1
[email protected]:~/test-space/myfiles# tree -F
2
.
3
├── dir1/
4
│   └── myvideo.mp4*
5
├── dir2/
6
│   ├── zabbix-cli-manual-1.7.0.pdf
7
│   └── zabbix-cli-rpm-2.1.1-1.tar.gz
8
├── myconf.txt
9
└── mylog.txt
10
​
11
2 directories, 5 files
12
​
13
[email protected]:~/test-space/myfiles# zip myconf.zip myconf.txt 
14
  adding: myconf.txt (deflated 60%)
15
​
16
[email protected]:~/test-space/myfiles# tree -F
17
.
18
├── dir1/
19
│   └── myvideo.mp4*
20
├── dir2/
21
│   ├── zabbix-cli-manual-1.7.0.pdf
22
│   └── zabbix-cli-rpm-2.1.1-1.tar.gz
23
├── myconf.txt
24
├── myconf.zip
25
└── mylog.txt
26
​
27
2 directories, 6 files
Copied!
use the -r option with the zip command and it will recursively zips the files in a directory. This option helps you to zip all the files present in the specified directory:
1
[email protected]:~/test-space/myfiles# cd ..
2
[email protected]:~/test-space# zip -r myfiles.zip myfiles/
3
  adding: myfiles/ (stored 0%)
4
  adding: myfiles/dir1/ (stored 0%)
5
  adding: myfiles/dir1/myvideo.mp4 (deflated 71%)
6
  adding: myfiles/etc/ (stored 0%)
7
  adding: myfiles/dir2/ (stored 0%)
8
  adding: myfiles/dir2/zabbix-cli-rpm-2.1.1-1.tar.gz (deflated 0%)
9
  adding: myfiles/dir2/zabbix-cli-manual-1.7.0.pdf (deflated 88%)
10
  adding: myfiles/mylog.txt (deflated 89%)
11
  adding: myfiles/myconf.txt (deflated 60%)
12
  adding: myfiles/myconf.zip (stored 0%)
Copied!
uzip
A separate companion program, unzip, unpacks and uncompresses zip archives.
1
[email protected]:~/test-space# rm -rf myfiles
2
[email protected]:~/test-space# unzip myfiles.zip 
3
Archive:  myfiles.zip
4
   creating: myfiles/
5
   creating: myfiles/dir1/
6
  inflating: myfiles/dir1/myvideo.mp4  
7
   creating: myfiles/etc/
8
   creating: myfiles/dir2/
9
  inflating: myfiles/dir2/zabbix-cli-rpm-2.1.1-1.tar.gz  
10
  inflating: myfiles/dir2/zabbix-cli-manual-1.7.0.pdf  
11
  inflating: myfiles/mylog.txt       
12
  inflating: myfiles/myconf.txt      
13
 extracting: myfiles/myconf.zip   
Copied!
zip command example
Description 
zip –v filename.zip file1.txt
 Verbose mode or print diagnostic version info.
zip –d filename.zip file.txt
 Removes the file from the zip archive.
zip –u filename.zip file.txt
Updates the file in the zip archive.
zip –m filename.zip file.txt
Deletes the original files after zipping.
zip –x filename.zip file_to_be_excluded
Exclude the files in creating the zip.
Compressing files
gzip
Gzip (GNU zip) is a compressing tool, which is used to truncate the file size. 
1
gzip [OPTION]... [FILE]...
Copied!
By default original file will be replaced by the compressed file ending with extension (.gz) and gzip removes the original files after creating the compressed file. gzip keeps the original file mode, ownership, and timestamp.
1
[email protected]:~/test-space/myfiles# tree -F
2
.
3
├── dir1/
4
│   └── myvideo.mp4*
5
├── dir2/
6
│   ├── zabbix-cli-manual-1.7.0.pdf
7
│   └── zabbix-cli-rpm-2.1.1-1.tar.gz
8
├── myconf.txt
9
└── mylog.txt
10
​
11
2 directories, 5 files
12
[email protected]:~/test-space/myfiles# gzip mylog.txt 
13
[email protected]:~/test-space/myfiles# tree -F
14
.
15
├── dir1/
16
│   └── myvideo.mp4*
17
├── dir2/
18
│   ├── zabbix-cli-manual-1.7.0.pdf
19
│   └── zabbix-cli-rpm-2.1.1-1.tar.gz
20
├── myconf.txt
21
└── mylog.txt.gz
22
​
23
2 directories, 5 files
Copied!
gunzip
To decompress a file we can use gunzip command and your original file will be back.
1
[email protected]:~/test-space/myfiles# ls -l
2
total 136
3
drwxr-xr-x 2 root root   4096 Sep 30 05:42 dir1
4
drwxr-xr-x 2 root root   4096 Oct  5 05:25 dir2
5
-rw-r--r-- 1 root root   2084 Sep 30 05:44 myconf.txt
6
-rw-r----- 1 root root 123925 Sep 30 05:41 mylog.txt.gz
7
[email protected]:~/test-space/myfiles# gunzip mylog.txt.gz 
8
[email protected]:~/test-space/myfiles# ls -l
9
total 1144
10
drwxr-xr-x 2 root root    4096 Sep 30 05:42 dir1
11
drwxr-xr-x 2 root root    4096 Oct  5 05:25 dir2
12
-rw-r--r-- 1 root root    2084 Sep 30 05:44 myconf.txt
13
-rw-r----- 1 root root 1156369 Sep 30 05:41 mylog.txt
Copied!
also we can use -d option to decompress a file using the “gzip” command.ex : gzip -d mydoc.gz
gzip command example
Description
gzip -r testfolder
compress every file in a folder and its subfolders
gzip -k mydoc.txt