10.3 Evaluating Data Quality, Signal Integrity, Outliers, and Cleaning Procedures

10.3 Evaluating Data Quality, Signal Integrity, Outliers, and Cleaning Procedures

Predictive models, workload ratios, and neuromuscular profiles are only as valid as the raw signals feeding them—a principle universally summarized as "Garbage In, Garbage Out" ($\text{GIGO}$). Raw streams from force plates, linear position transducers, inertial measurement units ($\text{IMUs}$), heart rate straps, and $\text{GNSS}$ receivers inevitably contain high-frequency sensor noise, electromagnetic interference, multipath satellite reflections, and missing observations. A sport scientist must master digital signal processing ($\text{DSP}$), robust statistical outlier auditing, and principled missing-data imputation to clean datasets without erasing true biological extremes.

---

Signal Integrity and Digital Signal Processing ($\text{DSP}$) Fundamentals

Signal vs. High-Frequency Noise and Zero-Lag Butterworth Filtering

Any measured biomechanical waveform $y(t)$ is the sum of the true underlying biological signal $s(t)$ and additive noise $n(t)$:

$y(t) = s(t) + n(t)$

In human movement biomechanics, voluntary segment kinematics and center-of-mass ground reaction forces ($\text{GRF}$) reside predominantly in the low-frequency spectrum below $10\text{–}20\text{ Hz}$ (with rapid passive impact transients during sprinting or landing extending up to $40\text{–}50\text{ Hz}$). Conversely, electrical mains hum ($50\text{–}60\text{ Hz}$), cable whip, and force-plate frame resonance occupy higher frequency bands.

To isolate $s(t)$ from high-frequency noise $n(t)$, the biomechanical gold standard is a fourth-order, zero-lag (bidirectional dual-pass) low-pass Butterworth digital filter:

• Low-Pass Attenuation: Passes frequencies below a selected cutoff frequency ($f_c$) unattenuated while progressively rolling off high-frequency noise above $f_c$.
• Zero Phase Shift (Dual-Pass Bidirectionality): A standard single-pass (forward-in-time) second-order filter introduces a temporal phase lag, shifting peak force or peak velocity several milliseconds later in time. By passing the second-order Butterworth filter forward through the time series and then passing it backward through the reversed time series, the phase shifts cancel out completely (zero-lag) while doubling the effective filter steepness to fourth-order.

Cutoff Frequency Selection (Winter's Residual Analysis) and the Smoothing Paradox

Selecting the optimal cutoff frequency $f_c$ via Winter's Residual Analysis—which plots the root-mean-square ($\text{RMS}$) residual between unfiltered and filtered signals across a spectrum of candidate cutoff frequencies—requires balancing a fundamental trade-off known as the Smoothing Paradox:

• Over-Smoothing ($f_c$ Set Too Low): If a practitioner applies an overly aggressive low-pass cutoff (e.g., $f_c = 4\text{ Hz}$ or $6\text{ Hz}$) to a $1{,}000\text{ Hz}$ vertical ground reaction force trace during an isometric mid-thigh pull or countermovement jump, the filter rounds off sharp physiological force onsets. This clips true peak force and severely underestimates time-derivative explosive metrics such as Rate of Force Development ($\text{RFD} = \Delta F / \Delta t$).
• Under-Smoothing ($f_c$ Set Too High): If $f_c$ is set too high (or left unfiltered) when calculating kinematics from displacement data, residual high-frequency noise is catastrophically amplified during numerical differentiation ($\text{displacement} \rightarrow \text{velocity} \rightarrow \text{acceleration}$), because differentiation multiplies noise amplitude proportionally to its frequency ($\omega$) during the first derivative and by $\omega^2$ during the second derivative.

Heart Rate Variability ($\text{HRV}$) Ectopic Beat and Artifact Correction

Signal cleaning is equally critical in autonomic cardiac monitoring. The root mean square of successive differences ($\text{RMSSD}$) between adjacent R-R intervals ($\text{IBI}$) is highly sensitive to a single missed R-peak or ectopic beat. If an athlete has a steady resting heart rate of $75\text{ bpm}$ ($\text{R-R interval} = 800\text{ ms}$) and movement of the chest strap causes the sensor to miss a single beat, the recorded interval doubles to $1{,}600\text{ ms}$. Because $\text{RMSSD}$ squares successive differences:

$\text{RMSSD} = \sqrt{\frac{1}{N - 1} \sum_{i=1}^{N-1} \left(\text{IBI}_{i+1} - \text{IBI}_i\right)^2}$

that single $(1600 - 800)^2 = 640{,}000\text{ ms}^2$ error term can artificially inflate the athlete's $60\text{-second}$ $\text{RMSSD}$ by $> 50\text{–}100\%$, falsely portraying a fatigued athlete as parasympathetically hyper-recovered! Consequently, raw R-R series must undergo automated threshold artifact detection (flagging intervals that deviate by $> 20\%$ from a local 5-beat moving median) and cubic spline interpolation before computing $\text{lnRMSSD}$.

---

Outlier Detection: Parametric vs. Non-Parametric Rules and Plausibility Auditing

Statistical Outlier Detection Frameworks

1. Parametric $Z$-Score Rule ($|Z| > 3.0$): Flags observations falling more than $3.0$ (or $2.5$) standard deviations from the sample mean ($Z = (x - \bar{x}) / s$). Critical Limitation (The Masking Effect): Both the sample mean $\bar{x}$ and sample standard deviation $s$ have a statistical breakdown point of $0\%$—meaning a single massive sensor error (e.g., a corrupted distance value of $95{,}000\text{ m}$) inflates $s$ so dramatically that the outlier's own $Z$-score may remain below $3.0$ while completely masking legitimate moderate outliers.
2. Non-Parametric Tukey Interquartile Range ($\text{IQR}$) Fences: Uses order statistics resistant to extreme contamination. Given the $25\text{th}$ percentile ($Q_1$) and $75\text{th}$ percentile ($Q_3$), where $\text{IQR} = Q_3 - Q_1$:

$\text{Lower Fence} = Q_1 - 1.5(\text{IQR}), \quad \text{Upper Fence} = Q_3 + 1.5(\text{IQR})$

Values outside $\pm 1.5(\text{IQR})$ are flagged as mild outliers for inspection, while values beyond $Q_1 - 3.0(\text{IQR})$ and $Q_3 + 3.0(\text{IQR})$ are classified as extreme outliers. 3. Median Absolute Deviation ($\text{MAD}$): Possesses the maximum theoretical breakdown point of $50\%$:

$\text{MAD} = \text{median}\left(|x_i - \text{median}(x)|\right), \quad Z_{\text{MAD}} = \frac{0.6745 \times |x_i - \text{median}(x)|}{\text{MAD}}$

Observations exceeding $|Z_{\text{MAD}}| > 3.5$ are flagged for review.

The Cardinal Rule of Sport Science Outlier Cleaning

A statistical outlier flag is a trigger for investigation, never an automatic command for deletion. Before removing or modifying any extreme value, the sport scientist must conduct a Domain Plausibility and Contextual Audit to distinguish between two fundamentally different phenomena:

• Technical / Sensor Artifacts (Delete or Re-Interpolate): Values that violate human biological limits or stem from verifiable hardware failure. Examples include a $\text{GNSS}$ unit reporting a peak velocity of $14.2\text{ m}\cdot\text{s}^{-1}$ ($51.1\text{ km}\cdot\text{h}^{-1}$, exceeding the human world record of $12.42\text{ m}\cdot\text{s}^{-1}$) accompanied by poor satellite geometry (Horizontal Dilution of Precision [$\text{HDOP}$] $= 4.5$ and only $5$ connected satellites near a stadium roof overhang); a heart rate monitor spiking to $248\text{ bpm}$ due to dry-electrode static interference; or a countermovement jump trial where the athlete stepped off the force plate during the initial quiet weighing phase.
• True Biological or Tactical Outliers (Retain in the Dataset!): Values that are statistically extreme ($|Z| > 3.0$ or beyond Tukey's upper fence) but physiologically authentic and verified by video or session logs. Examples include a box-to-box midfielder recording $1{,}450\text{ m}$ of high-speed running ($+3.2\text{ SD}$ above team mean) because their team played $30\text{ minutes}$ of extra time shorthanded after a red card, or an elite sprinter hitting a personal-best $10.4\text{ m}\cdot\text{s}^{-1}$ peak velocity with clean satellite lock ($\text{HDOP} = 0.9$, $14$ satellites). Automatically deleting or winsorizing true biological outliers blinds the monitoring system to the exact acute workload spikes and peak neuromuscular outputs that drive injury risk and performance adaptation.

---

Missing Data Taxonomy ($\text{MCAR}$, $\text{MAR}$, $\text{MNAR}$) and Workload Imputation

When data points are genuinely missing or deleted as corrupted artifacts, the sport scientist must classify the mechanism of missingness under Rubin's Taxonomy:

1. Missing Completely at Random ($\text{MCAR}$): The probability that an observation is missing is completely independent of both observed and unobserved variables (e.g., a $\text{GNSS}$ unit battery dies mid-practice or a force-plate USB cable is accidentally unplugged).
2. Missing at Random ($\text{MAR}$): The probability of missingness is systematically related to observed contextual variables, but not to the missing outcome itself (e.g., offensive linemen are missing outdoor $\text{GNSS}$ high-speed running values on Tuesday because their position group was scheduled inside the indoor dome for walk-through pass protection).
3. Missing Not at Random ($\text{MNAR}$): The probability of missingness is directly driven by the unobserved value itself (e.g., the most severely fatigued, sleep-deprived, or muscle-sore athletes intentionally skip the morning subjective wellness survey or countermovement jump screening because they fear a poor score will cause the coaching staff to bench them).

Imputation Strategies for Longitudinal Workload Continuity

In longitudinal workload monitoring (such as 7-day acute loads, 28-day chronic loads, or Exponentially Weighted Moving Averages [$\text{EWMA}$]), leaving a missed training session blank or treating it as 0 creates a phantom recovery day that artificially inflates the subsequent week's acute-to-chronic workload ratio ($\text{ACWR}$). Practitioners should apply a structured imputation hierarchy while always setting a boolean metadata flag (is_imputed = TRUE and imputation_method):

1. Individualized Session-RPE Regression Imputation: If an athlete's $\text{GNSS}$ unit failed during a $90\text{-minute}$ practice where they reported an $\text{RPE}$ of $7$ ($\text{sRPE} = 7 \times 90 = 630\text{ AU}$), predict their missing total distance or PlayerLoad using that specific athlete's historical linear regression equation relating their $\text{sRPE}$ to external load.
2. Drill-Level Position-Group Scaling: Sum the average distance covered by the athlete's specific position group across the exact drills the athlete participated in, multiplied by that athlete's historical work-rate ratio relative to their position peers (e.g., $1.06 \times \text{Positional Drill Mean}$).
3. Multiple Imputation ($\text{MICE}$) or Kalman State-Space Interpolation: Used for multivariate research modeling, whereas Last Observation Carried Forward ($\text{LOCF}$) should be avoided across variable training days because repeating yesterday's hard session load onto today's recovery day distorts microcycle periodicity.

---

Worked Example: Tukey $\text{IQR}$ Outlier Audit & Drill-Scaled $\text{GNSS}$ Imputation

Suppose a 20-player soccer squad completes a high-intensity tactical session. The quartiles for High-Speed Running ($\text{HSR} > 5.5\text{ m}\cdot\text{s}^{-1}$) across the squad are:

• $Q_1 (25\text{th percentile}) = 420\text{ m}$
• $Q_2 (\text{Median}) = 530\text{ m}$
• $Q_3 (75\text{th percentile}) = 620\text{ m}$

Step 1 — Calculate Tukey's $\text{IQR}$ and Upper Fence:

$\text{IQR} = Q_3 - Q_1 = 620 - 420 = 200\text{ m}$

$\text{Upper Fence (1.5 IQR)} = Q_3 + 1.5(200) = 620 + 300 = 920\text{ m}$

$\text{Extreme Upper Fence (3.0 IQR)} = Q_3 + 3.0(200) = 620 + 600 = 1{,}220\text{ m}$

Step 2 — Audit Two Flagged Outliers ($> 920\text{ m}$):

• Athlete A ($\text{HSR} = 1{,}010\text{ m}$, Peak Speed $= 9.4\text{ m}\cdot\text{s}^{-1}$, $\text{HDOP} = 0.8$, Satellites $= 15$): Video confirms Athlete A (a wingback) performed extra repeated overlap recovery runs during an 11v10 tactical overload drill. Decision: Authentic biological/tactical outlier $\rightarrow$ Retain $1{,}010\text{ m}$.
• Athlete B ($\text{HSR} = 1{,}380\text{ m}$, Peak Speed $= 14.6\text{ m}\cdot\text{s}^{-1}$, $\text{HDOP} = 4.8$, Satellites $= 4$): Sensor diagnostics reveal multipath velocity spikes during a 15-minute period near the stadium bleachers. Decision: Technical artifact $\rightarrow$ Delete corrupted segment and impute.

Step 3 — Impute Athlete B's Missing Segment via Individualized Position Scaling:

During the corrupted 15-minute drill, the other three central midfielders averaged $180\text{ m}$ of $\text{HSR}$. Across the preceding 20 sessions, Athlete B's historical work-rate ratio in that drill category is $1.05\times$ the central midfield mean. Adding the imputed drill value ($180 \times 1.05 = 189\text{ m}$) to Athlete B's clean pre-failure $\text{HSR}$ ($390\text{ m}$) yields a corrected session total of $579\text{ m}$, stored with is_imputed = TRUE.