14.3 Files, Databases, and Storage Requirements: Scale, Redundancy, and Backup
Key Takeaways
- A file is a named collection of data on storage; a database is an organized collection of related data managed by software that supports queries, concurrent users, and integrity rules.
- In a relational database, a table holds records (rows) with fields (columns); a primary key uniquely identifies each record, and a foreign key links to a record in another table.
- Unplanned redundancy (the same fact stored in many places) wastes space and causes inconsistencies, while planned redundancy (mirrored drives, replicas, backups) protects against failure.
- A common backup guideline is 3-2-1: keep three copies of important data, on two different types of media, with one copy off-site.
- Storage planning must account for scale: how much data each item needs, how many items there are, and how fast they grow, plus room for backups and copies.
What this competency asks
ETS asks you to be familiar with methods to store, manage, and manipulate data:
- Use terminology and concepts of files and databases.
- Identify measures of file size (byte, kilo, mega, giga, tera, peta). Section 13.1 covers the units.
- Identify issues connected with the storage requirements of computing applications, including scale, redundancy, and backup.
Files
A file is a named collection of data stored on a device. The operating system organizes files in a hierarchy of folders (directories) and records metadata about each one: size, type, dates, and permissions.
| Concept | Meaning |
|---|---|
| Text file | Human-readable characters (.txt, .csv, .html, source code) |
| Binary file | Data in a non-text format (.jpg, .mp3, executables); needs a program that understands the format |
| File format / extension | Tells programs how to interpret the bytes (.png, .docx, .json) |
| CSV | "Comma-separated values": one record per line, with fields separated by commas; a simple way to exchange tables |
| Path | A file's location, such as /home/ms-lee/grades/period3.csv |
| Open, read, write, close | The basic operations programs perform on files |
| Sequential vs. random access | Reading from the start in order vs. jumping directly to a position |
Files work well for documents and simple data. When many users need to search, update, and relate large amounts of structured data at the same time, a database is the better tool.
Databases
A database is an organized collection of related data, managed by a database management system (DBMS). Compared with loose files, a DBMS offers fast queries, safe concurrent access by many users, integrity rules, security, and recovery tools.
Relational terminology
A relational database stores data in tables (relations).
| Term | Meaning | Example |
|---|---|---|
| Table | A set of records about one kind of thing | Students |
| Record (row) | One item | One student |
| Field (column, attribute) | One property | grade |
| Primary key | Field(s) whose value uniquely identifies each record; never null | studentId |
| Foreign key | A field that refers to a primary key in another table | Enrollments.studentId |
| Query | A request for data | "All 10th graders, sorted by name" |
| Schema | The design: tables, fields, types, keys |
Relationships: one-to-many links are made with a foreign key (one teacher has many classes). Many-to-many links use a junction table (students and courses, linked through Enrollments).
Querying with SQL
SQL is a declarative language (Section 12.2): you state what you want.
SELECT name
FROM Students
WHERE grade = 10
ORDER BY name;
Given the rows (Maya, 10), (Leo, 11), (Ava, 10), (Zoe, 10), and (Ben, 9), the result is Ava, Maya, Zoe.
WHEREfilters rows, andORDER BYsorts them.COUNT,SUM, andAVGwithGROUP BYsummarize groups.HAVINGfilters the groups.JOINcombines tables on matching keys. For example, joinEnrollmentstoCoursesto list course titles for a student. ALEFT JOINkeeps every row of the left table even when there is no match.
Avoiding unplanned redundancy
Storing the same fact in many places causes update anomalies. Suppose a teacher's room number is copied into every student's record. Change the room, miss one copy, and the data now disagree. Normalization organizes tables so each fact is stored once and referenced by key. For example, room numbers live in a Teachers table, and student records refer to the teacher by ID.
Storage requirements: scale, redundancy, and backup
Scale
Estimate storage by multiplying the size per item by the number of items, then projecting growth:
- 30 teachers × 180 days × 2 short videos per day × 50 MB = 540,000 MB ≈ 540 GB per year.
- Five years of retention needs about 2.7 TB, before backups. Each backup copy adds that much again.
At large scale, you also must consider performance (searching billions of records needs indexes), cost, bandwidth for moving data, and retention policies, meaning which data you are allowed or required to keep, and for how long.
Redundancy: two very different meanings
| Kind | Example | Good or bad? |
|---|---|---|
| Unplanned data redundancy | The same address typed into three tables | Bad: wastes space and causes inconsistency |
| Planned (fault-tolerant) redundancy | Mirrored drives (RAID 1), database replicas in two data centers, multiple power supplies | Good: the system keeps running when one component fails |
When a question mentions redundancy "to protect against hardware failure," it means planned redundancy. When it mentions "inconsistent copies of data," it means unplanned redundancy.
Backup
A backup is a separate copy that can restore data after deletion, corruption, device failure, theft, ransomware, or disaster.
- 3-2-1 guideline: 3 copies of important data, on 2 different types of media, with 1 copy off-site (or in the cloud).
- Full vs. incremental: a full backup copies everything; an incremental backup copies only what changed since the last backup. Incremental backups are faster and smaller, but a restore needs the chain of backups.
- Versioning keeps older versions, so a file corrupted or encrypted by ransomware can be rolled back.
- Test restores. A backup that has never been restored is not proven to work.
- Sync is not backup. A synchronized folder faithfully copies deletions and corruption to every device. A true backup keeps separate, older copies.
- Mirroring (RAID) protects against a drive failure but not against accidental deletion or ransomware, because the mirror copies those changes instantly.
In a Students table, which field is the best choice for the primary key?
A table Students contains the rows (Maya, 10), (Leo, 11), (Ava, 10), (Zoe, 10), and (Ben, 9), as (name, grade). What does this query return?
SELECT name FROM Students WHERE grade = 10 ORDER BY name;
A teacher keeps all student project files in a folder that synchronizes automatically to a cloud account. Which statement about this setup is most accurate?
A district stores a copy of its student-information database in a second data center that takes over automatically if the first one fails. What is this an example of?