Sampling & Data Collection Plans

Sampling & Data Collection Plans

Quick Answer: A data collection plan defines metrics, sources, sampling method, frequency, roles, and operational definitions. Prefer random samples for unbiased estimates of the whole; use stratified samples when key subgroups must be represented. Collect with check sheets and consistent coding, then QC data for min/max violations, nulls, and erroneous values before you analyze.

Why Plans Beat Ad-Hoc Spreadsheets

Garbage in, garbage out is not a slogan—it is the most common Green Belt failure mode. Teams that “pull last month’s export” without definitions discover mid-Analyze that shifts mixed machines, inspectors changed criteria, and missing values were typed as zero. CSSGB BoK III.D.2 expects you to apply sampling concepts and collection discipline.

A complete data collection plan typically includes:

Attach the plan to the project file; update it when definitions change (and note the change date).

Sampling Concepts Green Belts Use

Population, sample, and bias

• Population — the full set you want to understand (all orders in Q2, all welds on Line 3).
• Sample — the subset you actually measure.
• Bias — systematic error from how you select or measure (only inspecting first-shift ease-of-access units).

Inferential statistics assume the sample represents the population. Convenience samples (“whatever was on the dock”) undermine that assumption.

Simple random sampling

Every unit in the frame has an equal probability of selection; selections are independent (or without-replacement rules are explicit).

How: random number generator against a numbered list; software sample tools; drawing lots.

Use when: you need an unbiased estimate of overall process performance and the population is relatively homogeneous, or you will analyze overall parameters (mean defect rate, mean cycle time).

Worked example: 5,000 closed tickets last month. Number them 1–5000; select 200 unique random IDs; pull those records. Avoid “first 200 in the export” if the export is sorted by severity or date in a way that correlates with Y.

Stratified sampling

Divide the population into strata (important subgroups)—shift, product family, region, machine—then sample within each stratum (often randomly), frequently in proportion to stratum size or with intentional oversampling of small critical strata.

Use when:

• Subgroups differ and each must be represented
• You plan to compare strata or ensure a rare but critical segment appears
• Overall estimates should not be dominated by one large easy stratum

Worked example: Returns come 70% from Product A, 25% B, 5% C—but C is high margin. A proportional stratified sample of 200 might take 140 A, 50 B, 10 C. If C’s quality is a project focus, increase C’s sample deliberately and weight overall estimates correctly.

Related ideas (awareness)

• Systematic sampling — every k-th unit after a random start; watch for cyclic processes that align with k.
• Rational subgrouping — for SPC, subgroup units produced under similar conditions so within-subgroup variation estimates common cause; this is not the same as a one-time audit sample for a mean.
• Census — measure all units; rare for high volume, common for small critical lots.

Random vs stratified (exam contrast): Random protects against selection bias for a single overall inference. Stratified guarantees coverage of known structure and often improves precision when strata means differ.

Check Sheets

A check sheet is a structured form for collecting data in real time by tally marks or simple entries. It is one of the seven basic quality tools.

Common formats:

1. Defect tally by type — columns for scratch, dent, mislabel; rows for hour or shift.
2. Defect location (concentration diagram) — outline of a part; marks where flaws appear.
3. Process distribution check sheet — bins for measured values to sketch a histogram manually.
4. Traveler / unit log — each row a unit with pass/fail and codes.

Design tips:

• One operational definition visible on the form
• Limited categories (Pareto-friendly); “Other” with comment
• Space for date, shift, operator, lot, machine
• Trial the form for a day; revise confusing fields before baseline week

Check sheets reduce free-text chaos and make stratification automatic if headers capture the X’s you care about.

Data Coding

Coding converts raw observations into analysis-ready symbols:

Rules:

1. Publish a codebook (code, meaning, examples, exclusions).
2. Never recycle codes with new meanings mid-project without a version flag.
3. Prefer codes that software can group (consistent spelling; no “scratch” vs “Scratch ” vs “SCR”).
4. Preserve original IDs so you can audit back to source.
5. For continuous data, store full resolution; do not over-round before analysis.

Poor coding creates phantom categories and breaks joins between systems (MES vs. Excel shadow logs).

QC Checks for Collected Data

Before Analyze, run a data quality gate. Minimum checks:

1. Min / max and range gates

• Compare values to physical or business limits (weight ≥ 0; % between 0 and 100; cycle time < shift length).
• Flag values outside process experience (three orders of magnitude off).
• For dates, reject futures or pre-go-live timestamps.

2. Erroneous and inconsistent values

• Unit mismatches (inches mixed with mm in one column).
• Impossible combinations (ship date before order date; pass code with defect count > 0 if definitions forbid it).
• Duplicate transaction IDs that double-count defects.
• Sentinel values used as missing (9999, -1) interpreted as real measurements.

3. Null / missing data

• Count % missing by field and by stratum (missingness may be informative—night shift skips a field).
• Decide policy: exclude row, impute only with documented method (rarely for core Y without expertise), or re-collect.
• Never silently fill missing Y with zero unless zero is a true measured value.

4. Sampling integrity checks

• Did actual n match the plan?
• Were strata filled as designed?
• Did collectors change mid-stream without MSA?

Worked QC example: Baseline pull of 500 cycle times shows three values of 0.00 and one of 50,000 minutes. Zeros were system defaults when the stop stamp was missing (nulls mis-coded); 50,000 was a clock left running over a holiday. After removal/rework per plan rules, mean dropped 12%—Analyze on the dirty file would have chased a false “average” problem.

Collection Plan in DMAIC Cadence

Green Belt Checklist

• Operational definitions written and trained
• Sampling method justified (random / stratified / rational subgroup)
• Check sheet or extract spec matches the plan
• Codebook versioned
• Min/max, null, and error screens run and documented
• Raw data archived before cleaning; cleaning log kept

Sampling chooses who enters the dataset; collection tools shape how facts are recorded; QC decides whether Analyze deserves to start. Apply all three and your statistical distributions and descriptive statistics will describe the process—not your data-entry bugs.