7.4 Advanced PETs: Differential Privacy, Secure Multi-Party Computation, and Homomorphic Encryption

7.4 Advanced PETs: Differential Privacy, Secure Multi-Party Computation, and Homomorphic Encryption

Quick Answer: While traditional anonymization focuses on altering stored data tables, modern privacy engineering deploys cryptographic Privacy-Enhancing Technologies (PETs) to protect data in use and in computation. Differential Privacy (DP) introduces calibrated mathematical noise to query outputs, ensuring that no observer can reliably infer whether any single individual's record was included in a dataset. Secure Multi-Party Computation (SMPC) allows multiple distrusting organizations to collaboratively compute joint functions without sharing their underlying private inputs. Homomorphic Encryption (HE) enables third-party cloud platforms to execute arbitrary mathematical operations directly on encrypted ciphertexts without ever possessing the decryption key.

---

Differential Privacy: Mathematical Foundations

Developed by Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith (2006), Differential Privacy is not a specific software tool or encryption algorithm; it is a rigorous, mathematically provable property of randomized data release algorithms.

+-----------------------------------------------------------------------------+
|                  THE DIFFERENTIAL PRIVACY BOUNDARY                          |
|                                                                             |
|   Dataset D  (Contains Individual X) ----> [ Randomized  ]                  |
|                                            [ Mechanism M ] ----> Output S   |
|   Dataset D' (Differs by Record X)   ----> [ P[M(D) in S] ]                 |
|                                                                             |
|   Mathematical Guarantee:                                                   |
|   P[M(D) in S] <= exp(epsilon) * P[M(D') in S] + delta                      |
|                                                                             |
|   Whether Individual X participates or opts out, the statistical outcome   |
|   of the query is virtually indistinguishable to an external observer!      |
+-----------------------------------------------------------------------------+

Formal Definition of $(\epsilon, \delta)$-Differential Privacy

A randomized algorithm $M$ with domain $\mathcal{D}$ and range $\mathcal{R}$ provides $(\epsilon, \delta)$-differential privacy if, for all neighboring datasets $D, D' \in \mathcal{D}$ that differ on at most one individual record (i.e., the symmetric difference $|D \Delta D'| = 1$), and for all query output subsets $S \subseteq \mathcal{R}$: $P[M(D) \in S] \le e^\epsilon \cdot P[M(D') \in S] + \delta$

Pure vs. Approximate Differential Privacy

• Pure Differential Privacy ($\epsilon$-DP): When $\delta = 0$, the definition reduces to: $P[M(D) \in S] \le e^\epsilon \cdot P[M(D') \in S]$ This provides an absolute, worst-case multiplicative bound of $e^\epsilon$ on the probability ratio. Even an adversary with unbounded computational power and complete auxiliary background knowledge about all other individuals in the dataset cannot reliably determine whether a target individual was present in $D$.
• Approximate Differential Privacy ($(\epsilon, \delta)$-DP): The parameter $\delta$ represents the probability that the strict $\epsilon$-multiplicative bound fails. In engineering practice, $\delta$ models a "catastrophic failure" probability (e.g., an unmasked record leaking). To maintain safety, $\delta$ must be cryptographically negligible and strictly smaller than the inverse of the dataset size: $\delta \ll \frac{1}{|D|}$ (typically $\delta \le 10^{-5}$ to $10^{-7}$). If $\delta \ge 1/|D|$, an algorithm could simply output one raw record unperturbed with probability $\delta$ and still formally satisfy the definition while catastrophic privacy leakage occurs.

---

The Privacy Budget ($\epsilon$) and Composition Theorems

The parameter $\epsilon$ (epsilon) is known as the privacy loss parameter or privacy budget. It dictates the strength of the privacy guarantee and controls the trade-off between privacy protection and statistical utility.

+-----------------------------------------------------------------------------+
|                   THE PRIVACY-UTILITY TRADEOFF CURVE                        |
|                                                                             |
|   Privacy (Noise) ^                                                         |
|                   |  * (epsilon = 0.1: Extremely High Privacy, High Noise)  |
|                   |    *                                                    |
|                   |      *                                                  |
|                   |        *                                                |
|                   |          * (epsilon = 1.0: Balanced Gold Standard)      |
|                   |              *                                          |
|                   |                  * (epsilon = 8.0: High Utility, Low DP)|
|                   +---------------------------------------------------->    |
|                   0.0                                          Utility      |
+-----------------------------------------------------------------------------+

Operational Meaning of Epsilon

• $\epsilon \to 0$: As $\epsilon$ approaches zero, $e^0 = 1$. The output distribution is completely independent of the underlying dataset. Privacy is mathematically absolute, but data utility is zero (output is purely random noise).
• $\epsilon \le 1.0$: The gold standard for strong privacy engineering. The probability ratio $e^1 \approx 2.718$ ensures that participating in the dataset increases an individual's inferential risk by at most a factor of 2.7.
• $\epsilon \in (1.0, 10.0)$: Common in production deployments, where the protection still bounds inference but less tightly.
• Large total budgets in practice: The US Census Bureau's 2020 redistricting data used zero-concentrated DP with a privacy-loss budget of $\rho = 2.63$ for the person file and $\rho = 0.07$ for the housing-unit file, which the Bureau reports as roughly $\epsilon = 17.14$ and $\epsilon = 2.47$ at $\delta = 10^{-10}$. Budgets this large are defensible only because they are spread over billions of tabulations and reviewed publicly; for a single query, $\epsilon > 10$ provides very weak protection.

Composition Properties: Managing the Privacy Budget

A critical strength of differential privacy over heuristic anonymization is that privacy loss is quantifiable, cumulative, and composable across multiple queries:

1. Sequential Composition: If $k$ independent randomized mechanisms $M_1, M_2, \dots, M_k$ are executed sequentially on the same underlying dataset $D$, where each mechanism $M_i$ satisfies $\epsilon_i$-differential privacy, the combined query sequence satisfies: $\epsilon_{\text{total}} = \sum_{i=1}^k \epsilon_i$ Engineering Implementation: A Differential Privacy Query Engine maintains an internal ledger of expended epsilon. When a researcher or analyst issues queries, the engine deducts $\epsilon_i$ from their total allocated budget $\mathcal{B}$. Once $\sum \epsilon_i \ge \mathcal{B}$, the system rejects all subsequent queries. This defends against reconstruction attacks: Dinur and Nissim (2003) showed that if an attacker can ask enough subset-sum queries whose answers carry noise much smaller than $\sqrt{n}$, a polynomial-time algorithm can reconstruct almost the entire database.
2. Parallel Composition: If mechanisms $M_1, M_2, \dots, M_k$ are executed on pairwise disjoint (non-overlapping) partitions of dataset $D$ ($D_1 \cap D_2 = \emptyset$), the total privacy cost is simply the maximum budget expended on any single partition: $\epsilon_{\text{total}} = \max(\epsilon_1, \epsilon_2, \dots, \epsilon_k)$ Because an individual's record can only exist in at most one disjoint partition, that individual's data contributes to only one mechanism's noise calculation.
3. Advanced Composition: Linear composition is pessimistic for many queries. The advanced composition theorem shows that $k$ mechanisms that are each $\epsilon$-DP are together roughly $\epsilon\sqrt{2k\ln(1/\delta')}$-DP (plus a small second-order term) with failure probability $\delta'$. For machine learning with DP-SGD, tighter accountants (the moments accountant and Rényi DP accounting) track privacy loss across thousands of training steps.

---

Global Sensitivity and Noise Injection Mechanisms

To ensure an algorithm satisfies $\epsilon$-differential privacy, the magnitude of the injected noise must be calibrated to the global sensitivity of the query function.

Global Sensitivity

Global sensitivity measures the maximum amount that the output of a deterministic query function $f: \mathcal{D} \to \mathbb{R}^k$ can change when evaluated on any two neighboring datasets $D, D'$ that differ by a single individual's record: $\Delta f = \max_{D, D' : |D \Delta D'| = 1} \|f(D) - f(D')\|$

• $L_1$ Global Sensitivity ($\Delta_1 f$): The maximum Manhattan distance change: $\Delta_1 f = \max \sum_{i=1}^k |f(D)_i - f(D')_i|$. For a single counting query (e.g., "How many patients have been diagnosed with leukemia?"), removing or adding one person changes the count by at most 1, so $\Delta_1 f = 1$.
• $L_2$ Global Sensitivity ($\Delta_2 f$): The maximum Euclidean distance change: $\Delta_2 f = \max \sqrt{\sum_{i=1}^k (f(D)_i - f(D')_i)^2}$.

+-----------------------------------------------------------------------------+
|                        NOISE INJECTION MECHANISMS                           |
|                                                                             |
|  1. LAPLACE MECHANISM (Pure epsilon-DP):                                    |
|     Output = f(D) + Lap(scale = Delta_1 f / epsilon)                        |
|     - Calibrated to L1 Sensitivity                                          |
|     - Symmetrical exponential distribution around zero                      |
|                                                                             |
|  2. GAUSSIAN MECHANISM (Approximate (epsilon, delta)-DP):                   |
|     Output = f(D) + N(0, sigma^2)                                           |
|     - Calibrated to L2 Sensitivity                                          |
|     - sigma = (Delta_2 f * sqrt(2 * ln(1.25 / delta))) / epsilon            |
+-----------------------------------------------------------------------------+

The Laplace Mechanism

The Laplace Mechanism achieves pure $\epsilon$-differential privacy by drawing noise from a zero-mean Laplace distribution with scale parameter $b = \frac{\Delta_1 f}{\epsilon}$: $M(D) = f(D) + \text{Lap}\left(\frac{\Delta_1 f}{\epsilon}\right)$ Where the probability density function of $\text{Lap}(b)$ is: $p(x) = \frac{1}{2b} \exp\left(-\frac{|x|}{b}\right)$

Numerical Calculation Example: Suppose an epidemiological research service executes a counting query ($"\text{COUNT patients with Type 2 Diabetes}"$) with an allocated privacy budget $\epsilon = 0.5$:

1. The global $L_1$ sensitivity of a counting query is $\Delta_1 f = 1$.
2. The scale parameter $b$ of the Laplace distribution is: $b = \frac{\Delta_1 f}{\epsilon} = \frac{1}{0.5} = 2.0$
3. The query engine calculates the true count (e.g., 1,420), samples random noise from $\text{Lap}(2.0)$, and returns the sum (e.g., $1,420 + (-1.43) = 1,418.57 \approx 1,419$).

The Gaussian Mechanism

The Gaussian Mechanism draws noise from a normal distribution $\mathcal{N}(0, \sigma^2)$ to satisfy relaxed $(\epsilon, \delta)$-differential privacy. The classic calibration below holds for $\epsilon < 1$ and scales the noise to $L_2$ sensitivity: $\sigma \ge \frac{\Delta_2 f \sqrt{2 \ln(1.25 / \delta)}}{\epsilon}$ The Gaussian mechanism is preferred in high-dimensional vector outputs (e.g., aggregating model gradients in federated machine learning) because $L_2$ sensitivity grows much more slowly than $L_1$ sensitivity as dimensionality increases.

---

Local vs. Central Differential Privacy

Differential privacy architectures are fundamentally divided by their trust boundary:

+-----------------------------------------------------------------------------------+
|                    CENTRAL VS. LOCAL DIFFERENTIAL PRIVACY                         |
|                                                                                   |
|  CENTRAL (GLOBAL) DIFFERENTIAL PRIVACY                                            |
|  [Client 1: Raw] \                                                                |
|  [Client 2: Raw] -- [Central Data Curator / DB] ---> [DP Engine] ---> Query Output|
|  [Client 3: Raw] /  (TRUSTED AGGREGATOR)            (Noise Added Once)            |
|  [High Statistical Accuracy; Single Point of Failure; High Breach Risk]           |
|                                                                                   |
|  LOCAL DIFFERENTIAL PRIVACY (LDP)                                                 |
|  [Client 1] -> [Add Noise] -> Perturbed \                                         |
|  [Client 2] -> [Add Noise] -> Perturbed -- [Untrusted Server] ----> Aggregate Est |
|  [Client 3] -> [Add Noise] -> Perturbed /  (ZERO TRUST REQUIRED)                  |
|  [No raw data at server; High noise; Needs very large samples N > 10^5]          |
+-----------------------------------------------------------------------------------+

1. Central (Global) Differential Privacy

• Architecture: Raw, unperturbed records are transmitted from edge clients over TLS and stored in a centralized database managed by a trusted data curator. The curator executes analytical queries, injects Laplace or Gaussian noise to the aggregate results, and releases the noisy outputs.
• Advantages: Superior data utility. Because noise is added once to the final aggregate sum, the error magnitude scales as $\mathcal{O}(1/\epsilon)$, independent of the number of participants $N$.
• Vulnerabilities: Requires absolute trust in the central curator. If the central repository is compromised, subpoenaed, or intercepted by a malicious insider, all raw personal records are exposed.

2. Local Differential Privacy (LDP)

• Architecture: Noise is injected directly on the user's edge client device (smartphone, laptop, browser) before data is transmitted to the collection server. The server never observes or stores raw personal attributes.
• The Randomized Response Foundation (Warner, 1965): To report a stigmatizing binary attribute (e.g., "Have you ever driven while intoxicated?"):
  1. The client flips a fair coin in secret.
  2. If Heads, the client answers honestly.
  3. If Tails, the client flips a second coin: answering "Yes" if Heads, "No" if Tails. Even if an adversary intercepts a "Yes" transmission, the participant retains plausible deniability (the response could have been forced by the second coin toss). The central server reconstructs the true population proportion through statistical de-biasing: $P(\text{True Yes}) = 2 \times P(\text{Observed Yes}) - 0.5$
• Enterprise Deployments:
  • Google RAPPOR (Randomized Aggregatable Privacy-Preserving Ordinal Response): Utilizes Bloom filters combined with permanent and instantaneous randomized response to collect browser diagnostic telemetry in Google Chrome.
  • Apple Differential Privacy: Deployed in macOS and iOS to identify popular emojis, trending search queries, and energy usage telemetry without capturing raw user keystrokes.
• Trade-offs: Local DP adds far more noise. Because every client adds independent noise, the error of an aggregate count grows on the order of $\sqrt{N}$ (its variance grows linearly in $N$), compared with a constant error under central DP. To achieve useful accuracy, LDP systems usually need very large populations ($N \ge 10^5$ to $10^7$).

---

Secure Multi-Party Computation (SMPC)

In many enterprise scenarios, multiple organizations wish to extract aggregate insights from their combined datasets but are legally or commercially prohibited from sharing raw data with one another. Secure Multi-Party Computation (SMPC) solves this challenge.

The SMPC Paradigm

A set of $n$ distrusting participants $P_1, P_2, \dots, P_n$ holding private inputs $x_1, x_2, \dots, x_n$ wish to compute an arbitrary function: $y = f(x_1, x_2, \dots, x_n)$ An SMPC protocol guarantees that all parties learn the final output $y$, but zero parties learn anything about any other party's private input $x_i$, beyond what can be inferred from the final output $y$ itself.

+-----------------------------------------------------------------------------+
|                   SHAMIR'S (t, n) THRESHOLD SECRET SHARING                  |
|                                                                             |
|   Secret S = a_0 (Secret embedded as polynomial y-intercept)                |
|   Polynomial: f(x) = S + a_1*x + a_2*x^2 + ... + a_{t-1}*x^{t-1} mod p      |
|                                                                             |
|   Shares generated:                                                         |
|   Party 1: (1, f(1))                                                        |
|   Party 2: (2, f(2))   ---> Any t shares reconstruct S via Lagrange         |
|   Party 3: (3, f(3))        interpolation!                                  |
|   Party 4: (4, f(4))   ---> Any t - 1 shares reveal ZERO information!       |
+-----------------------------------------------------------------------------+

Core SMPC Protocols

1. Shamir's Secret Sharing (Threshold Cryptography):
   • An input secret $S \in \mathbb{F}_p$ is encoded as the constant coefficient $a_0 = S$ of a random polynomial of degree $t-1$: $f(x) = S + a_1 x + a_2 x^2 + \dots + a_{t-1} x^{t-1} \pmod p$
   • The dealer evaluates $f(x)$ at $n$ distinct points and distributes share $(i, f(i))$ to participant $P_i$.
   • Properties: Any $t$ participants can pool their shares to reconstruct $S$ via Lagrange polynomial interpolation. Crucially, any subset of $t-1$ or fewer colluding parties possesses information-theoretic zero knowledge about $S$.
   • Linear Homomorphism: Shares can be added locally without network communication: $f_A(x) + f_B(x)$ yields shares of the secret sum $S_A + S_B$.
2. Yao's Garbled Circuits (Two-Party Computation - 2PC):
   • Designed for arbitrary boolean circuit evaluation between two parties: the Garbler and the Evaluator.
   • The Garbler transforms every logic gate (AND, OR, XOR) into an encrypted "garbled table," replacing 0 and 1 wire values with cryptographically random keys.
   • The Evaluator obtains the keys corresponding to their own private inputs using Oblivious Transfer (OT) (a protocol where the Evaluator receives the key without revealing their input bit to the Garbler).
   • The Evaluator evaluates the circuit gate by gate, unlocking the output wire keys to yield $f(x_1, x_2)$ without exposing intermediate values.
3. Practical Enterprise Applications:
   • Financial Fraud Detection: Rival banks execute joint graph-clustering algorithms across transaction ledgers to detect money laundering rings without disclosing customer account numbers.
   • Private Set Intersection (PSI): Advertisers and publishers compare user rosters to measure advertising conversion rates without either party sharing customer email lists.

---

Homomorphic Encryption (HE)

Traditional encryption schemes require data to be decrypted into plaintext before processing can take place in CPU registers. This exposes sensitive data to untrusted cloud infrastructure, hypervisor compromises, memory dumps, and malicious administrators. Homomorphic Encryption enables computations directly on ciphertexts.

+-----------------------------------------------------------------------------+
|                      THE HOMOMORPHIC ENCRYPTION MODEL                       |
|                                                                             |
|   [Client Device]                                                           |
|     |                                                                       |
|     v 1. Encrypts plaintext m with public key: c = Enc(m)                   |
|   [Untrusted Cloud Worker]                                                  |
|     |                                                                       |
|     v 2. Executes Evaluation Function: c_result = Eval(f, c_1, c_2)         |
|     |    (Cloud NEVER decrypts; has ZERO access to plaintext or secret key!)|
|     v 3. Returns encrypted result c_result                                  |
|   [Client Device]                                                           |
|     |                                                                       |
|     v 4. Decrypts with private key: m_result = Dec(c_result) = f(m_1, m_2)  |
+-----------------------------------------------------------------------------+

The Homomorphic Encryption Taxonomy

1. Partially Homomorphic Encryption (PHE): Supports an unlimited number of operations of only one type (either addition or multiplication):
   • Paillier Cryptosystem (Additive Homomorphism): Given ciphertexts $c_1 = \text{Enc}(m_1)$ and $c_2 = \text{Enc}(m_2)$: $c_1 \cdot c_2 \pmod{n^2} = \text{Enc}(m_1 + m_2 \pmod n)$ $c_1^k \pmod{n^2} = \text{Enc}(k \cdot m_1 \pmod n)$ Multiplying two ciphertexts together yields an encryption of the sum of the plaintexts! Paillier is widely used for secure electronic voting tallies and private telemetry aggregation because it is computationally lightweight compared to full FHE.
   • RSA / ElGamal (Multiplicative Homomorphism): Multiplying two RSA ciphertexts yields the encryption of the product of the plaintexts: $\text{Enc}(m_1) \cdot \text{Enc}(m_2) = \text{Enc}(m_1 \cdot m_2)$.
2. Somewhat Homomorphic Encryption (SHE): Supports both addition and multiplication, but only for a small, predefined circuit depth. Each multiplication introduces cryptographic noise into the ciphertext; once the noise exceeds a mathematical threshold, decryption fails.
3. Fully Homomorphic Encryption (FHE): Supports arbitrary, unbounded computations (any sequence of additions and multiplications). In 2009, Craig Gentry achieved a breakthrough by introducing bootstrapping—a technique where the encryption algorithm homomorphically evaluates its own decryption circuit to refresh and reduce noise in a ciphertext, allowing indefinite operations.

Modern Ring-LWE Lattice Schemes

Modern third- and fourth-generation FHE libraries (e.g., Microsoft SEAL, OpenFHE, Google Fully Homomorphic Encryption) are based on the Ring Learning With Errors (Ring-LWE) hard mathematical problem:

• BGV (Brakerski-Gentry-Vaikuntanathan) & BFV (Brakerski/Fan-Vercauteren): Optimized for exact integer modular arithmetic, ideal for private database querying and secure genomic matching.
• CKKS (Cheon-Kim-Kim-Song): Optimized for approximate floating-point vector arithmetic. CKKS allows controlled rounding errors in ciphertexts, making it the industry standard for Privacy-Preserving Machine Learning (PPML) inference in untrusted cloud environments.

Operational Trade-offs

• Performance Overhead: FHE computations are typically 1,000x to 100,000x slower than native plaintext processing. While simple additive aggregation executes in milliseconds, complex multi-layer deep neural network inferences require seconds or minutes.
• Ciphertext Expansion: A 4-byte plaintext integer expands into megabytes of Ring-LWE polynomial ciphertexts, imposing significant memory and bandwidth overhead.