4.2 Filesystem Types & Storage Management

Filesystem Types & Storage Management

Data storage is one of the foundational pillars of computing. Physical media—whether rotating hard disk drives (HDDs), solid-state drives (SSDs), or NVMe storage arrays—are merely vast sequences of raw, unformatted binary sectors. To make this physical hardware usable for applications and end users, an operating system must establish a Filesystem. A filesystem dictates how binary data is named, indexed, structured, secured, and retrieved across physical storage blocks.

---

1. Filesystem Fundamentals: Storage Blocks, Metadata & Inodes

+-----------------------------------------------------------------------------+
|                        FILESYSTEM DATA & METADATA LAYOUT                    |
|                                                                             |
|   PHYSICAL SECTORS (512B / 4096B)  -->  LOGICAL CLUSTERS / BLOCKS (4 KB)    |
|                                                                             |
|   +---------------------------------------------------------------------+   |
|   | SUPERBLOCK / BOOT SECTOR (Volume Geometry, Total Inodes/Blocks)     |   |
|   +---------------------------------------------------------------------+   |
|   | INODE TABLE / MASTER FILE TABLE (MFT)                               |   |
|   | - Inode 1042: File Type, Permissions (0644), UID/GID, Size (12 KB),  |   |
|   |               Timestamps (atime, mtime, ctime), Data Block Pointers |   |
|   +---------------------------------------------------------------------+   |
|   | DIRECTORY DATA BLOCKS (Maps File Names to Inode Numbers)            |   |
|   | - "report.pdf" ---> Inode 1042                                      |   |
|   | - "notes.txt"  ---> Inode 1043                                      |   |
|   +---------------------------------------------------------------------+   |
|   | STORAGE DATA BLOCKS (Actual File Payload Contents)                  |   |
|   | - Block 5021, Block 5022, Block 5023 (Binary contents of report.pdf)|   |
|   +---------------------------------------------------------------------+   |
+-----------------------------------------------------------------------------+

Sectors vs. Clusters (Allocation Units)

• Physical Sector: The smallest physical storage unit written by drive hardware. Historically 512 bytes; modern drives use Advanced Format (4Kn / 512e) with 4,096-byte (4 KB) physical sectors.
• Allocation Unit (Cluster / Block): The smallest logical chunk of disk space an operating system allocates to a file. Typically 4 KB by default.
  • Slack Space: If a 1 KB text file is saved on a filesystem with 4 KB clusters, the entire 4 KB cluster is allocated, leaving 3 KB of unused "slack space" that cannot be used by other files.

Linux Inodes (Index Nodes)

In POSIX filesystems (such as ext4), an Inode is a data structure storing all metadata about a filesystem object except its filename and file content:

• Metadata Stored in Inode: Inode number, file type (regular, directory, symlink), permission mode (e.g., 0755), User ID (UID), Group ID (GID), file size in bytes, hard link count, access/modification/change timestamps (atime, mtime, ctime), and pointers to data blocks.
• Directory Mechanism: A directory in Linux is simply a special file containing a list of (Filename, Inode Number) mapping pairs. This design explains why renaming a file is an instantaneous metadata update that requires zero data copying.
• Viewing Inodes: ls -i filename.txt or stat filename.txt.
• Inode Exhaustion: A filesystem can run out of inodes (df -i shows 100% inode utilization) if millions of tiny zero-byte files are created, preventing any new files from being written even if gigabytes of raw disk space remain free.

Hard Links vs. Symbolic (Soft) Links

• Hard Link (ln source.txt hardlink.txt): Creates an additional directory entry pointing directly to the same inode number on the same filesystem. The data remains accessible until all hard links (link count = 0) are deleted. Hard links cannot cross filesystem boundaries or link to directories.
• Symbolic Link (ln -s /source/path symlink.txt): A distinct file with its own unique inode containing a text string representing the target file's path. If the target file is deleted or moved, the symbolic link breaks ("dangling link"). Symlinks can span across different filesystems and link to directories.

---

2. Journaling Mechanics & Crash Consistency

In early unjournaled filesystems (like FAT32 or ext2), an unexpected power outage during a file write often left the filesystem in an inconsistent state (e.g., directory pointed to a block, but the block allocation bitmap wasn't updated). Recovering required running lengthy disk-wide integrity checks (chkdsk or fsck) scanning every single block on the drive.

+-----------------------------------------------------------------------------+
|                        JOURNALING TRANSACTION WORKFLOW                      |
|                                                                             |
|   Step 1: Write Intent      Step 2: Commit Transaction   Step 3: Write Data |
|   +-------------------+     +------------------------+   +----------------+ |
|   | Write operation   | --> | Kernel commits atomic  | ->| Data & Metadata| |
|   | logged to Journal |     | record to Journal disk |   | written to     | |
|   | circular buffer   |     | (Transaction Valid)    |   | permanent disk | |
|   +-------------------+     +------------------------+   +----------------+ |
|                                                                  |          |
|                                    Step 4: Checkpoint Cleared    v          |
|                                    +------------------------------------+   |
|                                    | Journal entry marked as finished;  |   |
|                                    | zero need for disk-wide fsck sweep |   |
|                                    +------------------------------------+   |
+-----------------------------------------------------------------------------+

How Journaling Works

Modern enterprise filesystems (NTFS, ext4, XFS) maintain a dedicated circular Journal (Write-Ahead Log):

1. When a write occurs, the filesystem logs the intended metadata and data changes into the journal.
2. The journal transaction is marked as "committed".
3. The actual filesystem structures and data blocks are updated on disk.
4. Once complete, the journal checkpoint is cleared.

• Crash Recovery: If power fails during Step 3, the OS reboots, inspects the journal in seconds, sees the committed transaction, and "replays" the journal to complete the write (or discards uncommitted transactions), restoring consistency in milliseconds without scanning the entire terabyte volume.

---

3. Comprehensive Filesystem Comparison

IT support technicians frequently configure local workstations, enterprise servers, external storage media, and cross-platform shared drives.

---

4. Partitioning Schemes: MBR vs. GPT

Before a physical disk can be formatted with a filesystem, it must be divided into logical regions called Partitions using a partition table standard.

+-----------------------------------------------------------------------------+
|                         MBR VS. GPT DISK LAYOUT                             |
|                                                                             |
|   [MBR DISK LAYOUT - Legacy BIOS]                                           |
|   +-----------+--------------------+------------------------------------+   |
|   | Sector 0  | Partitions 1 to 4  | 2.2 TB Addressable Limit           |   |
|   | (MBR/Boot)| (Max 4 Primary)    | (32-bit Logical Block Addressing)  |   |
|   +-----------+--------------------+------------------------------------+   |
|                                                                             |
|   [GPT DISK LAYOUT - Modern UEFI]                                           |
|   +-----------+-----------+--------------------+---------------+-----------+ |
|   | LBA 0     | LBA 1     | LBA 2-33           | Partition     | LBA -33 to| |
|   | Protective| Primary   | Partition Entries  | Allocations   | Backup GPT| |
|   | MBR       | GPT Header| (128 Partitions)   | (Up to 9.4 ZB)| Header/Tab| |
|   +-----------+-----------+--------------------+---------------+-----------+ |
+-----------------------------------------------------------------------------+

Master Boot Record (MBR)

Introduced in 1983 with IBM PC DOS:

• Sector 0 Structure: Located at the first 512-byte sector of the drive (LBA 0). Contains 446 bytes of Master Boot Code, a 64-byte Partition Table (four 16-byte partition records), and a 2-byte boot signature (0x55AA).
• 2.2 TB Capacity Limit: Uses 32-bit sector addressing. With 512-byte sectors, 2^32 * 512 bytes = 2.19 TB. Any drive capacity beyond 2.2 TB is completely inaccessible under MBR.
• Partition Limits: Supports a maximum of 4 Primary Partitions. To bypass this, one primary partition can be designated as an Extended Partition, containing multiple nested Logical Partitions.
• Single Point of Failure: If Sector 0 is corrupted, the partition table is destroyed.

GUID Partition Table (GPT)

Part of the modern Unified Extensible Firmware Interface (UEFI) standard:

• 64-bit Addressing: Supports theoretical disk capacities up to 9.4 Zettabytes (9.4 * 10^21 bytes).
• Partition Capacity: Supports at least 128 primary partitions natively in Windows without requiring extended partitions.
• Redundancy & Integrity: Stores a Primary GPT Header at LBA 1 and an exact Backup (Secondary) GPT Header at the physical end of the disk. Uses CRC32 Checksums to detect partition table corruption and automatically recover from the backup header.
• Protective MBR (LBA 0): Contains a dummy legacy MBR partition covering the entire drive to prevent older legacy disk utilities from misidentifying the GPT disk as unpartitioned raw space and overwriting data.

---

5. Linux Storage Administration, Mounting & LVM

Linux systems treat physical storage drives as block device nodes under /dev (e.g., /dev/sda for first SATA/SCSI drive, /dev/nvme0n1 for first NVMe SSD).

+-----------------------------------------------------------------------------+
|                        LINUX STORAGE COMMAND WORKFLOW                       |
|                                                                             |
|   1. Identify Disks  --> 2. Partition  --> 3. Format Filesystem --> 4. Mount|
|      lsblk / fdisk -l     fdisk / gdisk     mkfs.ext4 /dev/sdb1        mount|
+-----------------------------------------------------------------------------+

Core Storage Utilities

• lsblk: Lists all block storage devices, partitions, sizes, and mount points in a visual hierarchy (lsblk -f shows filesystems and UUIDs).
• fdisk /dev/sdb: Interactive MBR/GPT partitioning utility (commands: p print, n new partition, d delete, w write changes).
• gdisk /dev/sdb: Dedicated GPT partitioning utility.
• parted /dev/sdb: Scriptable partition tool supporting disks > 2 TB (parted /dev/sdb mklabel gpt).
• mkfs (Make Filesystem): Formats a partition with a filesystem.
  • sudo mkfs.ext4 -L "DataVolume" /dev/sdb1 (Format ext4 with label)
  • sudo mkfs.xfs /dev/sdb2 (Format XFS)
  • sudo mkfs.vfat -F 32 /dev/sdc1 (Format FAT32)
• df -h (Disk Free): Displays mounted filesystem disk space usage in human-readable gigabytes/megabytes.
• du -sh /var/log (Disk Usage): Displays the total disk space consumed by a specific directory and its contents.
• mount & umount:
  • sudo mount /dev/sdb1 /mnt/data (Attaches filesystem to directory)
  • sudo umount /mnt/data (Safely unmounts filesystem)

Persistent Mounting via /etc/fstab

To ensure filesystems automatically mount when the Linux system boots, entries are added to /etc/fstab. Each line contains 6 distinct fields:

# /etc/fstab structure:
# <Device / UUID>                       <Mount Point>  <Fstype>  <Options>         <Dump> <Pass>
UUID=4f3a2b1c-8e9d-4c3b-2a1f-0e9d8c7b6a5f  /data          ext4      defaults,noatime   0      2

• Testing /etc/fstab without rebooting: sudo mount -a (Mounts all filesystems listed in fstab; if syntax errors exist, it displays errors immediately rather than crashing the next server reboot).

Logical Volume Manager (LVM)

LVM abstracts physical storage drives into flexible, resizable virtual disk volumes.

+-----------------------------------------------------------------------------+
|                        LVM THREE-TIER ARCHITECTURE                          |
|                                                                             |
|   PHYSICAL DISKS:      [ /dev/sdb1 (500GB) ]    [ /dev/sdc1 (500GB) ]       |
|                                  |                        |                 |
|                                  v                        v                 |
|   PHYSICAL VOLUMES (PV):   [  PV /dev/sdb1  ]      [  PV /dev/sdc1  ]       |
|                                  |                        |                 |
|                                  +------------+-----------+                 |
|                                               |                             |
|                                               v                             |
|   VOLUME GROUP (VG):               [ VOLUME GROUP: vg_data (1000 GB) ]      |
|                                               |                             |
|                                  +------------+-----------+                 |
|                                  |                        |                 |
|                                  v                        v                 |
|   LOGICAL VOLUMES (LV):  [ LV lv_web (400GB) ]      [ LV lv_db (600GB) ]    |
|                                  |                        |                 |
|   FILESYSTEMS / MOUNT:      mount /var/www           mount /var/lib/mysql   |
+-----------------------------------------------------------------------------+

• Physical Volumes (PV): Raw disks or partitions initialized for LVM (pvcreate /dev/sdb1, pvs, pvdisplay).
• Volume Groups (VG): Pools of storage created by aggregating multiple PVs into a single storage pool (vgcreate vg_data /dev/sdb1 /dev/sdc1, vgs, vgextend vg_data /dev/sdd1).
• Logical Volumes (LV): Virtual partitions carved out of a Volume Group that can be formatted with filesystems and dynamically expanded while online (lvcreate -L 200G -n lv_web vg_data, lvs).
• Online Volume Expansion:

  # Extend Logical Volume by 50GB and resize underlying ext4 filesystem automatically:
  sudo lvextend -r -L +50G /dev/vg_data/lv_web
  

---

6. Windows Storage Management, Diskpart & Storage Spaces

Windows provides administrative tools for partitioning physical drives, managing RAID arrays, and pooling storage.

Disk Management (diskmgmt.msc)

The graphical MMC snap-in used to initialize disks (MBR or GPT), create Simple/Spanned/Striped/Mirrored volumes, shrink/extend existing NTFS volumes, and assign drive letters.

Command-Line Disk Partitioning: diskpart.exe

diskpart is a powerful interactive command-line partitioning utility:

:: Diskpart script to wipe, convert to GPT, partition, format, and assign drive letter:
diskpart
list disk
select disk 1
clean
convert gpt
create partition primary size=102400
format fs=ntfs quick label="EnterpriseData"
assign letter=E
exit

PowerShell Storage Cmdlets

Modern Windows administrators automate disk provisioning using PowerShell:

# Initialize a raw disk as GPT, partition using maximum space, format NTFS, assign drive letter:
Get-Disk -Number 1 | 
  Initialize-Disk -PartitionStyle GPT -PassThru | 
  New-Partition -UseMaximumSize -AssignDriveLetter | 
  Format-Volume -FileSystem NTFS -NewFileSystemLabel "AppStorage" -Confirm:$false

Windows Storage Spaces

Storage Spaces is a software-defined storage virtualization technology built into Windows 10/11 and Windows Server that groups commodity physical disks into Storage Pools and creates virtual disks with selectable resiliency:

• Simple Space (No Resiliency - RAID 0): Stripes data across drives. High performance; zero fault tolerance (a single drive failure loses all data).
• Two-Way Mirror (RAID 1): Writes two copies of data across at least 2 physical drives. Protects against 1 drive failure.
• Three-Way Mirror: Writes three copies across at least 5 physical drives. Protects against simultaneous 2-drive failures.
• Parity Space (RAID 5): Stripes data alongside parity information across at least 3 drives. Balances storage capacity efficiency with single-drive failure protection.