Descriptive Statistics and Exploratory Data Analysis

Descriptive Statistics and Exploratory Data Analysis

Quick Answer: Descriptive statistics summarize the analysis-ready dataset (center, spread, shape, percentiles). Exploratory data analysis (EDA) uses those summaries plus tables and charts to understand structure, find anomalies, and generate questions before modeling. On CBDA, Domain 3 (~16%) expects you to interpret descriptions in business terms—including when imbalanced features (for example, ~96% one demographic) make broad claims invalid.

After scope, data, and decision models frame the work, analysis typically begins with description, not prediction. Description answers: What is true in this dataset as prepared? EDA is the disciplined practice of learning the data’s personality so later diagnostics and models are not built on surprises you should have caught.

This section covers measures of center and spread, distributions and percentiles, EDA goals and tools, imbalance risks for inference, and how preparation for analysis (competency awareness aligned with preparation-oriented analysis competencies such as 3.5–3.6-style expectations) keeps descriptive work honest.

Descriptive Statistics: The Language of “What Is”

Descriptive statistics reduce many rows to understandable numbers. They do not, by themselves, prove why something happened or what will happen next—that is diagnostic and predictive work. They do set the baseline every stakeholder needs.

Measures of center

CBDA habit: Report mean and median together when skew is possible. If mean days-to-close is 90 and median is 45, the business story is not “typical deal is 90 days”—a long tail is pulling the mean.

Measures of spread

High spread can mean true heterogeneity (segments differ) or process instability—or data quality noise. Description flags the fact of spread; diagnosis investigates why.

Distributions and shape

A distribution shows how values are allocated across a range or categories. Shape vocabulary you should recognize:

• Symmetric — mean ≈ median; classic “bell-shaped” intuition (not all symmetric data is perfectly normal).
• Right-skewed (positive) — long tail of high values; mean > median (common for revenue, dwell time).
• Left-skewed — long tail of low values; mean < median (less common in business metrics but possible).
• Bimodal / multimodal — multiple peaks often signal mixed populations (two product lines, two processes) that should be segmented before averaging.
• Heavy tails — extreme values more common than a thin-tailed model expects; affects risk and “average” storytelling.

Percentiles and quantiles

Percentiles locate a value relative to the rest of the data. The 90th percentile of support handle time is the value below which 90% of tickets fall. Percentiles are decision-friendly:

• SLAs often use 90th or 95th percentiles, not means.
• Capacity planning cares about tails.
• Comparing medians (50th) and 90th percentiles shows typical vs stressed performance.

Example: Median fulfillment time 2 days, 90th percentile 9 days → many customers are fine while a painful minority drives complaints. A mean alone could hide that operational story.

EDA Goals: What You Are Trying to Learn

Exploratory data analysis is purposeful investigation, not random clicking in a BI tool. CBDA-aligned EDA goals:

1. Understand shape and structure — distributions, categories, time patterns, grain sanity.
2. Spot anomalies and data issues — impossible values, unexpected spikes, duplicate mass, broken segments.
3. Check assumptions — Does the population match the scope model? Are key fields populated enough for the research question?
4. Generate questions and hypotheses — EDA feeds diagnostic questions; it should not be sold as final causal proof.
5. Guide next methods — Severe skew may push log transforms or robust stats; rare events may require stratified sampling or different metrics; multimodality may require segmentation first.
6. Communicate baselines — Stakeholders need “what is normal here” before “what drives it.”

EDA is not a substitute for the research question

Trap: “We will explore until something interesting appears, then that becomes the project.” Unbounded fishing produces false patterns and orphan insights. Time-box EDA against the approved research questions and decision model; promote surprising findings into new documented questions rather than silent scope changes.

Tools of Description: Tables, Histograms, Summary Stats

You do not need vendor-specific skills for CBDA. You need to know what each view is for and how to interpret it for business decisions.

Summary tables

• Univariate summaries — count, missing rate, mean, median, std dev, min/max, key percentiles for numeric fields; frequency and percent for categories.
• Cross-tabs — outcome vs segment (cancel rate by plan; defect rate by plant).
• Time aggregates — weekly volumes, rates, and moving baselines to see seasonality.

Always reconcile counts to the scope model: if scope says 120,000 eligible customers and the table shows 80,000, stop and explain the gap before storytelling.

Histograms and bar charts

• Histograms show distribution of a continuous variable (order value, age, cycle time).
• Bar charts show frequencies or metrics by category.
• Choose binning thoughtfully: too few bins hide shape; too many look noisy. For exam purposes, focus on what the shape implies (skew, multiple modes, cliffs at policy caps).

Boxplots (conceptual)

Boxplots summarize median, IQR, and candidate outliers. Useful for comparing spread across segments (cycle time by region). Remember: points outside whiskers are candidates for investigation, not automatic deletes (Domain 2 theme continues).

Before modeling: the descriptive gate

A practical pre-model checklist:

Skipping this gate is how teams “discover” in modeling what EDA would have shown in an hour.

Imbalanced Features and Valid Inference

Imbalance appears in outcomes (2% fraud) and in features/populations (one region dominates rows). Both affect what you can honestly claim.

IIBA-style demographic imbalance example

Imagine an employee engagement or customer survey dataset where about 96% of records belong to one demographic group (for example, one age band, region, or role family). A model or average “overall satisfaction = 4.2” is mostly a statement about that majority group. Claims that results generalize to underrepresented groups are on thin ice.

Valid inference responses CBDA favors:

• State the limitation clearly: estimates are dominated by Group A; Group B sample is too small for stable segment claims.
• Do not over-segment the tiny group into even smaller slices that create random noise dressed as insight.
• Consider stratified sampling or targeted data collection if decisions about minority segments matter.
• Avoid proxy fairness theater — reporting a single global metric that hides 96/4 imbalance can mislead HR or marketing decisions.
• Check whether imbalance is a data collection bias (only one group was invited) vs true population structure.

Exam trap: Treating a 96%-one-group sample as nationally representative because “N is large.” Large N does not fix coverage bias. Validity of inference depends on whether the sample matches the population the decision is about.

Class imbalance in outcomes

If only 3% of customers churn, a naïve model that always predicts “no churn” is 97% “accurate” and useless for retention ops. Description must report base rates. Later modeling (next chapter themes) will need appropriate metrics (precision/recall, lift, cost-weighted errors)—but the descriptive foundation is honest base-rate reporting.

Preparing Data for Analysis (Into Domain 3)

Domain 2 prepared sources; Domain 3 still requires analysis-specific preparation:

1. Apply the scope model filters consistently to every table used in description.
2. Create analysis tables at the decision grain (customer-month, encounter, opportunity).
3. Define outcome and key metrics with written business rules before charting “success.”
4. Handle categories thoughtfully (collapse rare levels for stability; do not invent false order in unordered codes).
5. Document transforms used for description (winsorization for a display chart vs exclusion from the study population—different intents).
6. Version the descriptive baseline so later diagnostic work compares to a frozen “what we saw.”

Preparation for analysis is not a one-time cleanse; it is ensuring every descriptive claim is traceable to a defined population and metric.

Worked Mini Scenario: Subscription Cancelations

Research question: Among U.S. online subscribers, which tenure and plan cohorts account for the most voluntary churn revenue loss in the last 12 months?

Descriptive path:

1. Confirm 1.2M eligible subscriber-months after scope filters; 3.1% voluntary cancel base rate.
2. Mean monthly revenue $18; median $12 → right skew from high-tier plans.
3. Histogram of tenure-at-cancel shows peaks at month 1–2 and month 12–13 (bimodal → mixed cancel reasons plausible).
4. Cross-tab: basic plan is 70% of cancels but mid-tier is 55% of lost revenue → volume ≠ value.
5. Demographic feature for a side analysis is 96% one region because of a historical marketing focus → any “regional preference” claim is mostly that region; do not over-interpret the 4% remainder.
6. Package baselines for stakeholders: base rates, revenue-weighted cohort table, distribution notes, imbalance caveats—then move to diagnostic “why” questions with eyes open.

Connecting Description to Later Domain 3 Work

Descriptive EDA feeds:

• Diagnostic analysis — where to drill, which segments differ, which anomalies need root-cause thinking.
• Predictive modeling — feature distributions, rare levels, leakage suspects, base rates.
• Interpretation (Domain 4) — the “before” picture in the data story.

If description is sloppy, diagnosis invents causes for artifacts, and prediction optimizes the wrong reality.

Study Checklist for This Section

• Define mean, median, mode and when median beats mean.
• Explain variance/std dev vs IQR at a conceptual level.
• Read a skewed or bimodal distribution for business meaning.
• Use percentiles in an SLA or tail-risk sentence.
• List EDA goals and why unbounded fishing is a trap.
• Explain why ~96% one-demographic data limits valid inference.
• Describe pre-model descriptive checks tied to scope and grain.