5.7 Archiving and Compression: tar, gzip, bzip2, xz, cpio & dd (103.3)

5.7 Archiving and Compression: tar, gzip, bzip2, xz, cpio (103.3)

Quick Summary: Linux separates archiving (combining multiple files, directories, and metadata into a single container stream) from compression (reducing raw byte size). The tar (Tape Archive) and cpio utilities create archive bundles, while gzip, bzip2, and xz perform data compression. Understanding integrated flags (-z, -j, -J), compression ratios, and pipeline mechanics is essential for Topic 103.3.

---

1. Archiving vs. Compression Concepts

• Archiving: Aggregates hundreds or thousands of individual files, directory hierarchies, permissions, ownerships, and timestamps into a single uncompressed stream (e.g., .tar or .cpio).
• Compression: Applies mathematical entropy encoding algorithms (DEFLATE, Burrows-Wheeler, LZMA2) to reduce file size. Standard Linux compression utilities (gzip, bzip2, xz) compress only single files at a time; they do not package directory structures.
• Combined Workflow (.tar.gz, .tar.bz2, .tar.xz): tar packages the directory structure into a single archive file, which is then compressed using a compression tool.

---

2. tar (Tape Archive) Deep Dive

tar Command Anatomy & Option Rules:
tar  [Operation Mode: MUST choose ONE]  [Modifiers & Compression]  -f [Archive File]  [Targets]
tar             -c                            -z -v             -f backup.tar.gz    /etc /var/log

Primary Operation Modes (Choose Exactly ONE)

Operation Modifiers & Flags

• -f <file> (--file): Specifies the archive file name or device. CRITICAL SYNTAX RULE: The -f flag must immediately precede the archive filename. In tar -czvf archive.tar.gz /data, -f is last in the cluster so archive.tar.gz is parsed as the file. If written tar -czfv archive.tar.gz /data, v would be treated as the archive filename!
• -v (--verbose): Lists file names as they are archived or extracted.
• -p (--preserve-permissions): Preserves original file permissions (default for superuser).
• -C <dir> (--directory): Changes directory to <dir> before performing operations (e.g., tar -xzf app.tar.gz -C /opt/).
• --exclude=<pattern>: Excludes files matching globbing pattern from the archive.
• -W (--verify): Attempts to verify the archive after writing.

Built-in Compression Modifiers

# Creating a compressed tarball with gzip:
sudo tar -czvf /backup/etc_backup.tar.gz /etc

# Listing contents of a bzip2 compressed archive without extracting:
tar -tjvf /backup/logs.tar.bz2

# Extracting an xz compressed archive into a specific target directory:
sudo tar -xJvf release.tar.xz -C /opt/production/

3. Compression Utilities Comparison: gzip, bzip2, xz

Common Compression Operational Flags

• Replaces original by default: When running gzip file.txt, the original file.txt is deleted and replaced by file.txt.gz.
• -d (--decompress): Decompresses the file (e.g., gzip -d file.gz is identical to gunzip file.gz).
• -c (--stdout): Writes output to standard output, preserving the original file (gzip -c file.txt > file.txt.gz).
• -k (--keep): Keeps (does not delete) input files during compression or decompression (supported in gzip, bzip2, xz).
• -1 to -9: Compression levels (-1 or --fast is fastest with lowest ratio; -9 or --best is slowest with maximum ratio; default is usually -6).
• -l (--list): Lists uncompressed size, compressed size, and ratio (available in gzip and xz).

# Inspecting compressed log files directly without decompressing to disk:
zcat /var/log/syslog.2.gz | grep "CRON"
zgrep "Failed password" /var/log/auth.log.*.gz
xzless /usr/share/doc/kernel/changelog.xz

💡 LPIC-1 Exam Fill-in-the-Blank Alert: To decompress a .bz2 file from the command line while keeping the original file, use bunzip2 -k or bzip2 -dk. To view the contents of a gzip-compressed text file directly to stdout, use zcat.

4. cpio: Copy In/Out Archiving

The cpio (Copy In/Out) tool processes archives by reading file lists from standard input (stdin) or writing archive streams to standard output (stdout).

Primary Operation Modes (Choose Exactly ONE)

Essential cpio Modifiers

• -v (--verbose): Lists file names processed.
• -t (--list): Prints table of contents (used with -i: cpio -it < archive.cpio).
• -d (--make-directories): Creates leading directories where needed during extraction.
• -u (--unconditional): Replaces existing files without prompting.
• -m (--preserve-modification-time): Preserves original file modification timestamps.
• --null / -0: Reads null-delimited (\0) filenames from stdin (pairs with find -print0).
• -H <format> (--format): Specifies archive format. Standard formats:
  • newc: SVR4 portable format with CRC/inodes (mandatory standard for Linux initramfs images!).
  • crc: SVR4 portable format with computed CRC checksum.
  • tar / ustar: Standard POSIX tar format.
  • odc: Old POSIX.1 portable format.

# Creating a modern portable cpio archive using find pipeline:
$ find . -depth -print0 | cpio --null -ov -H newc > /backup/tree.cpio

# Viewing table of contents of a cpio archive:
$ cpio -itv < /backup/tree.cpio

# Extracting all files and creating directories as needed:
$ cpio -idmv < /backup/tree.cpio

# Direct directory tree cloning using pass-through mode:
$ find /var/data -depth -print0 | cpio --null -pdmv /mnt/storage/

---

5. dd: Raw Block-Level Copying

Objective 103.3 names three archiving utilities: tar, cpio and dd. dd is the odd one out — it does not understand files, directories, permissions, or ownership. It copies blocks of bytes from an input to an output, which is precisely why it can duplicate a boot sector, clone a whole disk, or manufacture a swap file when no filesystem-aware tool can.

Operand Syntax (Not Options)

dd uses operand=value syntax rather than dashes — writing dd -if=/dev/sda is a syntax error, and the exam tests this.

Suffixes multiply the number: bs=1K (1024), bs=1M, bs=1G.

Canonical Exam Scenarios

# 1. Create a 2 GiB zero-filled file (the standard swap-file first step)
$ sudo dd if=/dev/zero of=/swapfile bs=1M count=2048 status=progress

# 2. Back up the 512-byte MBR (446 bytes of boot code + 64-byte partition table + 2-byte signature)
$ sudo dd if=/dev/sda of=/root/mbr.bak bs=512 count=1

# 3. Back up ONLY the boot code, leaving the partition table untouched
$ sudo dd if=/dev/sda of=/root/bootcode.bak bs=446 count=1

# 4. Clone one disk onto another of equal or larger size
$ sudo dd if=/dev/sda of=/dev/sdb bs=64M conv=noerror,sync status=progress

# 5. Write an installer image to a USB stick (target the DISK, not a partition)
$ sudo dd if=debian-12.5.0-amd64-netinst.iso of=/dev/sdb bs=4M conv=fdatasync status=progress

# 6. Securely overwrite a partition with random data before disposal
$ sudo dd if=/dev/urandom of=/dev/sdc1 bs=1M status=progress

conv= Values Worth Memorising

• conv=noerror — continue after read errors instead of aborting. Essential when imaging a failing disk.
• conv=sync — pad short input blocks with NUL bytes so the output stays block-aligned. Almost always paired with noerror, because skipping a bad block without padding would shift every subsequent byte.
• conv=notrunc — do not truncate the output file. Required when patching bytes into the middle of an existing image.
• conv=fdatasync — flush the write cache before dd exits, so the command does not report success while data is still queued in RAM.

Monitoring a Long Transfer

# Modern GNU coreutils: built-in progress
$ sudo dd if=/dev/sda of=/backup/sda.img bs=64M status=progress

# Portable alternative: SIGUSR1 makes dd print statistics without stopping
$ sudo kill -USR1 $(pgrep -x dd)

Exam Trap — of= Overwrites Without Warning: dd has earned the nickname "disk destroyer." There is no confirmation prompt. Writing of=/dev/sda instead of of=/dev/sdb destroys the partition table and filesystem of the running system instantly. Always confirm the target with lsblk immediately before pressing Enter.

Exam Rule: Larger bs= values generally raise throughput up to a point (1M–64M is the practical sweet spot), because each block is one read syscall and one write syscall. The default bs=512 is correct only when you genuinely need 512-byte granularity, such as MBR work.