Description: Candidates should be able to maintain a standard filesystem, as well as the extra data associated with a journaling filesystem.
Key Knowledge Areas:
Verify the integrity of filesystems
Monitor free space and inodes
Repair simple filesystem problems
Terms and Utilities:
XFS tools (such as xfs_metadump and xfs_info)
In cases when your system crashes or loses power, your filesystems may be left in an inconsistent state, with some changes completed and some not.
Operating with a damaged filesystem is not a good idea as you are likely to further compound any existing errors.We’ll take a look at some tools to help us manage such problems.
The main tool for checking and repairing filesystems is
fsck, which, like
mkfs, is really a front end to filesystem-checking routines for the various filesystem types.
[email protected]:~# fsckfsck fsck.ext3 fsck.fat fsck.nfsfsck.cramfs fsck.ext4 fsck.minix fsck.vfatfsck.ext2 fsck.ext4dev fsck.msdos[email protected]:~# ls /sbin/fsck*/sbin/fsck /sbin/fsck.ext3 /sbin/fsck.fat /sbin/fsck.nfs/sbin/fsck.cramfs /sbin/fsck.ext4 /sbin/fsck.minix /sbin/fsck.vfat/sbin/fsck.ext2 /sbin/fsck.ext4dev /sbin/fsck.msdos
Some of these are just links to e2fsck command and they are the same
The fsck command in Linux allows us to manually check for file system inconsistencies, Fsck command needs to be run with superuser privileges or root(ubuntu 16.04 here):
[email protected]:~# fsck /dev/sda1fsck from util-linux 2.27.1e2fsck 1.42.13 (17-May-2015)/dev/sda1 is mounted.e2fsck: Cannot continue, aborting.
Lets simply check file system on an unmounted ext3 partition (sdb1) and try to fix errors :
[email protected]:~# fsck /dev/sdb1fsck from util-linux 2.27.1e2fsck 1.42.13 (17-May-2015)/dev/sdb1: clean, 11/1310720 files, 126322/5242624 blocks
This command will attempt to check /dev/sdb1, and report any errors it finds.The exit code returned by fsck is one of following conditions:
0 No errors
1 Filesystem errors corrected
2 System should be rebooted
4 Filesystem errors left uncorrected
8 Operational error
16 Usage or syntax error
32 Checking canceled by user request
128 Shared-library error
-N option just shows what would be executed but do not attempt to repair them:
[email protected]:~# fsck -N /dev/sdb1fsck from util-linux 2.27.1[/sbin/fsck.ext3 (1) -- /dev/sdb1] fsck.ext3 /dev/sdb1
-n causes these commands not to fix anything and just show what was going to be done:
[email protected]:~# fsck -n /dev/sdb1fsck from util-linux 2.27.1e2fsck 1.42.13 (17-May-2015)/dev/sdb1: clean, 11/1310720 files, 126322/5242624 blocks
Normally, fsck will skip parts of the filesystem marked as "clean" — meaning all pending writes were successfully made. The -f ("force") option specifies that fsck should check parts of the filesystem even if they are not "dirty". The result is a less efficient, but a more thorough check.
[email protected]:~# fsck -f /dev/sdb1fsck from util-linux 2.27.1e2fsck 1.42.13 (17-May-2015)Pass 1: Checking inodes, blocks, and sizesPass 2: Checking directory structurePass 3: Checking directory connectivityPass 4: Checking reference countsPass 5: Checking group summary information/dev/sdb1: 11/1310720 files (0.0% non-contiguous), 126322/5242624 blocks
We can also check file systems using their UUID.(use blkid command ):[email protected]:~# blkid /dev/sdb1/dev/sdb1: UUID="b9ca56aa-4ea3-4f61-92c8-18c867fd991f" SEC_TYPE="ext2" TYPE="ext3" PARTUUID="4d71bc84-01"[email protected]:~# fsck -N UUID="b9ca56aa-4ea3-4f61-92c8-18c867fd991f"fsck from util-linux 2.27.1e2fsck 1.42.13 (17-May-2015)/dev/sdb1: clean, 11/1310720 files, 126322/5242624 blocks
fsck command example
fsck -M /dev/sda1
prevents running fsck on mounted filesystem
fsck -t ext3 /dev/sdb1
Check Only a Specific Filesystem Type
fsck -y -f /dev/sdb1
pass “yes” to all the questions to fix
For checking a XFS filesystem, wehave to use xfs_check command
There are several more advanced tools that we can use to examine or repair a filesystem.
tune2fs:Adjusts parameters on ext2 and ext3 filesystems. Use this to add a journal
dumpe2fs: shows all super blocks info
debugfs: interactive file system editor
The ext family of file systems also has a utility called
tune2fs, which can be used to inspect information about the block count as well as information about whether the filesystem is journaled (ext3 or ext4) or not (ext2).
-l shows contents of the filesystem superblock, including the current values of the parameters:
[email protected]:~# tune2fs -l /dev/sdb1tune2fs 1.42.13 (17-May-2015)Filesystem volume name: <none>Last mounted on: <not available>Filesystem UUID: b9ca56aa-4ea3-4f61-92c8-18c867fd991fFilesystem magic number: 0xEF53Filesystem revision #: 1 (dynamic)Filesystem features: has_journal ext_attr resize_inode dir_index filetype sparse_super large_fileFilesystem flags: signed_directory_hashDefault mount options: user_xattr aclFilesystem state: cleanErrors behavior: ContinueFilesystem OS type: LinuxInode count: 1310720Block count: 5242624Reserved block count: 262131Free blocks: 5116302Free inodes: 1310709First block: 0Block size: 4096Fragment size: 4096Reserved GDT blocks: 1022Blocks per group: 32768Fragments per group: 32768Inodes per group: 8192Inode blocks per group: 512Filesystem created: Wed Jan 22 23:42:22 2020Last mount time: n/aLast write time: Thu Jan 23 02:21:07 2020Mount count: 0Maximum mount count: -1Last checked: Thu Jan 23 02:21:07 2020Check interval: 0 (<none>)Reserved blocks uid: 0 (user root)Reserved blocks gid: 0 (group root)First inode: 11Inode size: 256Required extra isize: 28Desired extra isize: 28Journal inode: 8Default directory hash: half_md4Directory Hash Seed: 73007ad4-213f-44cd-b8f0-4fbd0c1e90b2Journal backup: inode blocks
The command can also be used to set many parameters or convert an ext2 filesystem to ext3 by adding a journal using -j option:
tune2fs -j /dev/sdd1
Also we can use tune2fs for changing or modifying partition label:
[email protected]:~# tune2fs -L myroot /dev/sda1tune2fs 1.42.13 (17-May-2015)
dumpe2fs command is used to print the super block and blocks group information for the filesystem present on device.
[email protected]:~# dumpe2fs /dev/sdb1 | grep superblockdumpe2fs 1.42.13 (17-May-2015)Primary superblock at 0, Group descriptors at 1-2Backup superblock at 32768, Group descriptors at 32769-32770Backup superblock at 98304, Group descriptors at 98305-98306Backup superblock at 163840, Group descriptors at 163841-163842Backup superblock at 229376, Group descriptors at 229377-229378Backup superblock at 294912, Group descriptors at 294913-294914Backup superblock at 819200, Group descriptors at 819201-819202Backup superblock at 884736, Group descriptors at 884737-884738Backup superblock at 1605632, Group descriptors at 1605633-1605634Backup superblock at 2654208, Group descriptors at 2654209-2654210Backup superblock at 4096000, Group descriptors at 4096001-4096002
Is an interactive filesystem debugger. Use it to examine or change the state of an ext2 or ext3 filesystem. It opens the filesystem in read-only mode unless we tell it not to (with
[email protected]:~# debugfs /dev/sda1debugfs 1.42.13 (17-May-2015)debugfs: cd /etc/debugfs: pwd[pwd] INODE: 1179649 PATH: /etc[root] INODE: 2 PATH: /debugfs: stat passwdInode: 1187630 Type: regular Mode: 0644 Flags: 0x80000Generation: 2904697195 Version: 0x00000000:00000001User: 0 Group: 0 Size: 2388File ACL: 0 Directory ACL: 0Links: 1 Blockcount: 8Fragment: Address: 0 Number: 0 Size: 0ctime: 0x5e2064c5:64a1019c -- Thu Jan 16 16:57:33 2020atime: 0x5e2cb8a0:4aa345f8 -- Sun Jan 26 01:22:32 2020mtime: 0x5e2064c5:64a1019c -- Thu Jan 16 16:57:33 2020crtime: 0x5e2064c5:64a1019c -- Thu Jan 16 16:57:33 2020Size of extra inode fields: 32EXTENTS:(0):4265700debugfs: ncheck 1187630Inode Pathname1187630 /etc/passwddebugfs: q
xfs file system has it's own family commands. xfs_info is the same as tune2fs but for xfs.
xfsprogs package must be installed
[email protected]:~# xfs_info /dev/sdb1meta-data=/dev/sdb1 isize=512 agcount=4, agsize=1310656 blks= sectsz=512 attr=2, projid32bit=1= crc=1 finobt=1 spinodes=0data = bsize=4096 blocks=5242624, imaxpct=25= sunit=0 swidth=0 blksnaming =version 2 bsize=4096 ascii-ci=0 ftype=1log =internal bsize=4096 blocks=2560, version=2= sectsz=512 sunit=0 blks, lazy-count=1realtime =none extsz=4096 blocks=0, rtextents=0
complete check of file system
check and fixes problems
same as xfs_check
[email protected]:~# xfs_repair -n /dev/sdb1xfs_repair: /dev/sdb1 contains a mounted and writable filesystemfatal error -- couldn't initialize XFS library
[email protected]:~# xfs_repair /dev/sdb1Phase 1 - find and verify superblock...Phase 2 - using internal log- zero log...- scan filesystem freespace and inode maps...- found root inode chunkPhase 3 - for each AG...- scan and clear agi unlinked lists...- process known inodes and perform inode discovery...- agno = 0- agno = 1- agno = 2- agno = 3- process newly discovered inodes...Phase 4 - check for duplicate blocks...- setting up duplicate extent list...- check for inodes claiming duplicate blocks...- agno = 0- agno = 1- agno = 2- agno = 3Phase 5 - rebuild AG headers and trees...- reset superblock...Phase 6 - check inode connectivity...- resetting contents of realtime bitmap and summary inodes- traversing filesystem ...- traversal finished ...- moving disconnected inodes to lost+found ...Phase 7 - verify and correct link counts...done
In XFS, you can only extend file system and can not reduce it.
On a storage device, a file or directory is contained in a collection of blocks. Information about a file is contained in an inode.
Reminder : inodes keeps information such as who the owner is, when the file was last accessed, how large it is, whether it is a directory, and who can read from or write to it. The inode number is also known as the file serial number and is unique within a particular filesystem.
Data blocks and inodes each take space on a filesystem, so we need to monitor the space usage to ensure that your filesystems have space for growth.
df (DiskFree) command is used to find out about the free and used space of file systems.
If no file name is passed as an argument with df command then it shows the space available on all currently mounted file systems
[email protected]:~# dfFilesystem 1K-blocks Used Available Use% Mounted onudev 462796 0 462796 0% /devtmpfs 98508 6544 91964 7% /run/dev/sda1 50442940 5613840 42243708 12% /tmpfs 492540 284 492256 1% /dev/shmtmpfs 5120 4 5116 1% /run/locktmpfs 492540 0 492540 0% /sys/fs/cgrouptmpfs 98508 48 98460 1% /run/user/1001
-T print file system type, -h, –human-readable print sizes (in power of 1024):
[email protected]:~# df -ThFilesystem Type Size Used Avail Use% Mounted onudev devtmpfs 452M 0 452M 0% /devtmpfs tmpfs 97M 6.4M 90M 7% /run/dev/sda1 ext4 49G 5.4G 41G 12% /tmpfs tmpfs 481M 284K 481M 1% /dev/shmtmpfs tmpfs 5.0M 4.0K 5.0M 1% /run/locktmpfs tmpfs 481M 0 481M 0% /sys/fs/cgrouptmpfs tmpfs 97M 48K 97M 1% /run/user/1001
-Hmake numbers human readable also (in powers of 1000).
-i list inode information instead of block usage:
[email protected]:~# df -iFilesystem Inodes IUsed IFree IUse% Mounted onudev 115699 456 115243 1% /devtmpfs 123135 750 122385 1% /run/dev/sda1 3211264 256212 2955052 8% /tmpfs 123135 9 123126 1% /dev/shmtmpfs 123135 6 123129 1% /run/locktmpfs 123135 17 123118 1% /sys/fs/cgrouptmpfs 123135 27 123108 1% /run/user/1001
Remember? there is no owner or access rights on vfat filesystems. vfat file format has no inodes!
df command example
dislpay all information includes pseudo, duplicate and inaccessible file systems.
df -Th /home
Display Information of /home File System
df -k or -m or -h
displays information in Bytes, MB , GB
df -t ext3
include specific file system type
df -x xfs
exclude specific file system type
df command gives information about a whole filesystem. Sometimes you might want to know how much space is used by a specific file or directory, To answer this kind of question, we use the
du (Disk Usage) command, used to check the information of disk usage of files and directories on a machine.
du [OPTION]... [FILE]...
write count of all files, not just directories
human readable Means we can see sizes in Bytes, KB, MB, GB,...
grand total usage disk space at the last line
go N or fewer sub directories further
display only total for each directory
[email protected]:~# du16 ./.aptitude8 ./.cache/dconf12 ./.cache8 ./.config/htop8 ./.config/gedit8 ./.config/dconf4 ./.config/ibus/bus8 ./.config/ibus36 ./.config4 ./.gnupg/private-keys-v1.d48 ./.gnupg4 ./.gconf12 ./.elinks16 ./.ssh4 ./.nano8 ./.local/share12 ./.local336 ./backup8 ./.dbus/session-bus12 ./.dbus4 ./test-space932 .[email protected]:~# du -sh932K .
--time option is used to display the last modification time in the output of du.
--exclude=PATTERN will exclude files that match PATTERN example:
du -ah --exclude="*.txt" /home/payam
Lets take a look at some other repairing tools beside tools which we have learned in this lesson:
Adjusts parameters on ext2 and ext3 filesystems and can set journaling .
Prints the super block and block group descriptor information for an ext2 or ext3 filesystem.
Is an interactive filesystem debugger. Use it to examine or change the state of an ext2 or ext3 filesystem.
Displays and adjusts parameters on ReiserFS filesystems.
Performs similar functions to dumpe2fs and debugfs for ReiserFS filesystems.
Displays XFS filesystem information.
Expands an XFS filesystem
Changes the parameters of an XFS filesystem.
Repairs an XFS filesystem when the mount checks are not sufficient to repair the system.
Examines or debugs an XFS filesystem.
Displays many aspects of btrfs filesystem information
Check btrfs filesystems
Finds the block that is the root of the btrfs filesystem
Displays btrfs internal metadata
Tune various btrfs filesystem parameters, and enables or disables some extended features
Attempt to restore files from a damaged btrfs filesystem
With the special thanks of shawn powers.