13.3 Lossless and Lossy Compression: Run-Length Encoding, Huffman Coding, and Media Formats

What this competency asks

Under "bits as the universal medium," ETS asks you to describe lossy and lossless data compression. ETS's sample question describes Huffman coding, which assigns shorter codes to frequent values and longer codes to rare ones, and asks for its appropriate use: lossless compression. The distinction is the heart of this topic.

Why compress?

Raw media is huge. About 10 MB for one minute of CD-quality audio, and roughly 12 gigabits per second for uncompressed 4K video at 60 frames per second. Compression reduces the bits needed, which saves storage and transmission time.

Compression ratio = original size ÷ compressed size. A 10:1 ratio means the compressed file is one-tenth the size.

Lossless compression

Lossless compression removes redundancy so that decompression restores the original exactly, bit for bit.

Run-length encoding (RLE)

Replace each run of identical symbols with a count and the symbol.

RLE works well for images with large areas of one color, such as simple graphics, icons, and faxed documents. It works poorly, and can even expand the data, when there are few runs, as in photographs and ordinary text.

Huffman coding

Huffman coding builds variable-length prefix codes from symbol frequencies. Frequent symbols get short codes, and rare symbols get long ones. No code is the beginning of another, so the bit stream decodes unambiguously.

Worked example: the message AAAAAAABBC has A × 7, B × 2, and C × 1.

1. Combine the two least frequent: C (1) + B (2) forms a node of weight 3.
2. Combine that node (3) with A (7) to form the root (10).
3. Label branches 0 and 1: A = 0, B = 10, C = 11.

Decoding 0010110 reads 0 → A, 0 → A, 10 → B, 11 → C, 0 → A, giving AABCA. The code table, or the frequencies, must be stored or agreed on so the receiver can decode.

Why the prefix rule matters. Suppose the codes were A = 0, B = 01, and C = 1 instead. The bits 01 could mean B, or A followed by C, so the message is ambiguous. In a prefix code, no codeword is the beginning of another codeword. The decoder reads bits from left to right and emits a symbol the moment the bits read so far match a codeword, with no separators needed.

Dictionary methods

Lempel–Ziv methods replace repeated sequences with references to earlier occurrences or to dictionary entries. LZW is used in GIF images. DEFLATE, which combines LZ77 with Huffman coding, is used in ZIP, gzip, and PNG.

Lossless formats: ZIP and gzip archives, PNG and GIF images, FLAC audio.

Lossy compression

Lossy compression gets much higher ratios by permanently discarding information that people are unlikely to notice.

Typical results: photos at 10:1 or better, and music at around 128–256 kbps instead of CD's 1,411 kbps. Video compression often exceeds 100:1.

Video codecs get those ratios by storing occasional key frames (I-frames), each compressed like a standalone photo, and between them predicted frames (P-frames and B-frames) that record only motion and differences. A lecture recording with a still slide compresses far better than confetti or fast sports action, where almost every pixel changes from frame to frame.

Costs:

• Lost data cannot be recovered.
• Artifacts appear at high compression: blocky images, smeared edges, "swishy" audio.
• Generation loss: decompressing, editing, and recompressing a lossy file again and again compounds the damage. Editors keep a lossless master.

Calculating ratios and savings

Space savings = 1 − (compressed size ÷ original size). A 2:1 ratio saves 50%, a 4:1 ratio saves 75%, and a 10:1 ratio saves 90%. The uncompressed sizes below come from the formulas in Section 13.2. The JPEG and ZIP sizes are illustrative.

For audio at a constant bit rate, the ratio is simply the ratio of bit rates: 1,411.2 kbps ÷ 128 kbps ≈ 11.

Why no lossless method shrinks every file

A counting argument, the pigeonhole principle, shows the limit. There are 2ⁿ different files of n bits, but only 2ⁿ − 1 files shorter than n bits (1 + 2 + 4 + … + 2ⁿ⁻¹). If a lossless compressor made every n-bit file shorter, at least two different files would have to share the same compressed output, and decompression could not tell them apart. So every lossless method leaves some inputs the same size or larger, as run-length encoding did with ABCD. In practice, random data and files that are already compressed (JPEG, MP3, ZIP) barely shrink when they are zipped again. Compressing a second time does not keep helping.

Choosing lossless or lossy

Compression is not encryption

Compression makes data smaller. It does not make it secret. Anyone with the decompression method can read the data. Protecting data requires encryption (Section 13.4). Compressed data is often encrypted afterward, because encrypted data looks random and does not compress well.