Section 4.1: Data Quality Dimensions

Introduction to Data Quality Dimensions

Data Quality (DQ) is defined by the Data Management Association (DAMA) as the planning, implementation, and control of activities that apply quality management techniques to data to make it fit for purpose. To measure data quality objectively, organizations must decompose it into measurable components called Data Quality Dimensions.

While various frameworks exist, the Data Management Association United Kingdom (DAMA UK) working group identified six core dimensions of data quality that are widely accepted and tested on the Certified Data Management Professional (CDMP) exam. These dimensions are accuracy, completeness, consistency, timeliness, uniqueness, and validity.

---

The Six Core Dimensions

1. Accuracy

Accuracy represents the degree to which data correctly describes the 'real-world' object or event being represented. It is the closeness of agreement between a data value and an accepted reference value (the 'ground truth').

• Measurement: Accuracy is measured by comparing data against an authoritative physical source, a gold-standard third-party database (e.g., verifying mailing addresses against the United States Postal Service database), or through direct customer confirmation.
• Example: If a customer's actual home address is '123 Maple Street', but the Customer Relationship Management (CRM) system lists it as '123 Mapple St', the data is inaccurate.
• Exam Trap: Do not confuse accuracy with validity. Data can be highly valid (conforming to all format and type rules) but completely inaccurate. For instance, a phone number stored as '+1-555-019-2834' is structurally valid but inaccurate if it belongs to someone else.

2. Completeness

Completeness refers to the proportion of data that is stored compared to the potential of 100% complete data. It assesses whether all required data values are present for a given dataset, record, or attribute.

• Measurement: Completeness is calculated as the ratio of populated fields to total expected fields: Completeness % = (Number of Populated Required Fields / Total Number of Required Fields) * 100
• Example: An e-commerce order record requires a 'shipping_address', 'payment_method', and 'email_address'. If the 'email_address' is null, the record is incomplete.
• Exam Trap: A null value does not always represent a completeness defect. If an attribute is optional (e.g., 'middle_name' or 'suffix'), a null value is correct and does not degrade completeness. Thus, completeness must always be measured against specific business rules that define which fields are required under what conditions.

3. Consistency

Consistency is the absence of difference when comparing two or more representations of a data item against its counterpart in another system or dataset. It measures if data values remain uniform across the enterprise.

• Measurement: Consistency is assessed by reconciling equivalent fields across multiple applications or databases.
• Example: In the CRM system, a customer's status is listed as 'Active'. However, in the billing system, the same customer's status is listed as 'Suspended'. This discrepancy represents a consistency failure.
• Exam Trap: Consistency does not guarantee accuracy. Two databases can consistently store the exact same incorrect information (e.g., both showing an incorrect birthdate). Consistency merely confirms alignment, not truth.

4. Timeliness

Timeliness (sometimes split into currency and latency) refers to the degree to which data represents reality at the required point in time.

• Currency measures how up-to-date the data is (e.g., is the stock price from 1 second ago or 1 hour ago?).

• Latency measures the time delay between when a real-world event occurs and when the corresponding data becomes available in the target system.

• Measurement: Timeliness is measured as the time difference between the event timestamp and the data availability timestamp, compared to a service-level agreement (SLA).

• Example: An inventory database that updates once every 24 hours may suffer from poor timeliness during peak shopping events, leading to out-of-stock items being shown as available.

• Exam Trap: Timeliness requirements are highly context-dependent. A data warehouse updating nightly might have excellent timeliness for quarterly financial reporting, but the same update frequency would be catastrophic for a real-time fraud detection engine.

5. Uniqueness

Uniqueness means that no entity or event is represented more than once within a dataset. It is the opposite of duplication.

• Measurement: Uniqueness is measured by identifying duplicate records using primary keys, unique natural keys, or fuzzy matching algorithms (e.g., Levenshtein distance on names and addresses).
• Example: A database containing two customer records for 'John H. Smith, 456 Oak Lane' with customer IDs 'C-102' and 'C-987' violates the uniqueness dimension.
• Exam Trap: Uniqueness issues directly hinder Master Data Management (MDM) efforts. The presence of duplicate records inflates customer counts, distorts reporting, and can lead to multiple marketing packages being sent to the same individual.

6. Validity

Validity is the degree to which data conforms to the syntax (format, range, type, or pattern) of its definition or business rules.

• Measurement: Validity is checked by running data against format masks (regular expressions), range checks, lookup tables, and referential integrity constraints.
• Example: A field defined as 'US_State_Code' containing the value 'XX' is invalid because 'XX' is not one of the 50 valid US state postal abbreviations.
• Exam Trap: Validity is a structural constraint, not a truth check. A date of birth stored as '1899-01-01' is technically valid (it conforms to the 'YYYY-MM-DD' date format), but it is almost certainly inaccurate for a living customer.

---

Summary Comparison of Core Dimensions

---

Secondary Dimensions and Interdependencies

While the CDMP exam focuses heavily on the six core DAMA UK dimensions, candidates should also be familiar with secondary dimensions and the interdependencies between dimensions:

• Referential Integrity: Ensuring that parent-child relationships are enforced. For example, an order record cannot contain a customer ID that does not exist in the customer master table.
• Reasonability: This measures if the data values fall within logical, plausible boundaries. For example, a utility bill of $50,000 for a 1-bedroom apartment might be valid in format (numeric) but is highly unreasonable.
• Dimension Interdependency: Dimensions do not exist in isolation. Improving one dimension can sometimes negatively impact another, or failure in one can cascade. For instance, if an ingestion engine enforces strict validity checks and rejects any record that does not conform to the rules, the rejected records will result in a completeness issue in the target database. Conversely, relaxing validity checks to improve completeness will result in a drop in validity. Understanding these trade-offs is a critical skill for Data Quality Managers.