4.2 Grouping, Reshaping & Array Manipulation ($group, $unwind, $replaceWith)

4.2 Grouping, Reshaping & Array Manipulation ($group, $unwind, $replaceWith)

In real-world applications, raw data rarely matches the exact shape required for analytical dashboards, reporting endpoints, or decoupled microservices. The MongoDB Aggregation Framework provides specialized stages to aggregate datasets by category keys ($group), flatten embedded arrays into document streams ($unwind), and elevate nested subdocuments to top-level root documents ($replaceWith / $replaceRoot).

Mastering these operators enables developers to perform complex multi-dimensional data transformations entirely within the database engine.

---

1. The $group Stage Architecture

The $group stage separates documents into groups according to a specified grouping key (defined in the mandatory _id field) and evaluates accumulator expressions for each distinct group.

+-----------------------------------------------------------------------------------------+
|                                   $group Stage Mechanics                                |
|                                                                                         |
|  Incoming Docs               Grouping Key: _id                 Accumulator Outputs      |
|  { dept: 'HR', sal: 50k }   ---> Group 'HR'   ---> { _id: 'HR', totalSal: 120k, count: 2|
|  { dept: 'IT', sal: 90k }   ---> Group 'IT'   ---> { _id: 'IT', totalSal: 200k, count: 2|
|  { dept: 'HR', sal: 70k }   
|  { dept: 'IT', sal: 110k }  
+-----------------------------------------------------------------------------------------+

Syntax

{
  $group: {
    _id: <grouping expression>,
    <field1>: { <accumulator1>: <expression1> },
    <field2>: { <accumulator2>: <expression2> }
  }
}

Grouping Key Patterns (_id)

1. Group by Single Field: Reference an existing field with "$fieldName":

   { $group: { _id: "$department", employeeCount: { $sum: 1 } } }
   

2. Group by Compound Key: Group by multiple dimensions using an embedded subdocument:

   {
     $group: {
       _id: {
         dept: "$department",
         state: "$officeLocation.state",
         yearHired: { $year: "$hireDate" }
       },
       totalPayroll: { $sum: "$salary" }
     }
   }
   

3. Collection-Wide Aggregation (_id: null): Grouping by _id: null (or any constant value like _id: 1 or _id: "total") treats all incoming documents as a single monolithic partition to compute global metrics:

   {
     $group: {
       _id: null,
       globalRevenue: { $sum: "$totalAmount" },
       averageOrderValue: { $avg: "$totalAmount" },
       totalOrders: { $sum: 1 }
     }
   }
   

Exam Trap: In $group, every calculated output field must use an accumulator operator (such as $sum, $avg, $push). You cannot perform direct field assignment (e.g., { total: "$price" }) inside $group; doing so results in a parsing error.

---

2. Comprehensive Accumulator Operators in $group

Accumulator operators compute aggregate values across all documents that share the same grouping key.

Accumulator Operator Reference Table

$push vs. $addToSet Comparison

// Consider input documents for department 'Engineering':
// { name: "Alice", skill: "Go" }
// { name: "Bob",   skill: "MongoDB" }
// { name: "Carol", skill: "Go" }

db.employees.aggregate([
  {
    $group: {
      _id: "$department",
      allSkills: { $push: "$skill" },      // Output: [ "Go", "MongoDB", "Go" ]
      uniqueSkills: { $addToSet: "$skill" } // Output: [ "Go", "MongoDB" ]
    }
  }
]);

Position-Sensitive Accumulators: $first and $last

Because MongoDB collections do not guarantee natural physical storage order, using $first or $last without an explicit prior $sort stage produces non-deterministic results:

// Find each customer's most recent order
db.orders.aggregate([
  {
    $sort: { customerId: 1, orderDate: -1 } // Sort newest first
  },
  {
    $group: {
      _id: "$customerId",
      latestOrderDate: { $first: "$orderDate" },
      latestOrderTotal: { $first: "$totalAmount" },
      earliestOrderDate: { $last: "$orderDate" }
    }
  }
]);

---

3. The $unwind Stage: Array Deconstruction

Documents frequently embed arrays of subdocuments or scalar values (e.g., an order containing multiple lineItems, or an article with a list of tags). The $unwind stage deconstructs an array field from the input documents to output one document for each element in the array.

+-----------------------------------------------------------------------------------------+
|                                 $unwind Stage Operation                                 |
|                                                                                         |
|  Input Document:                                                                        |
|  { _id: 101, customer: 'Alice', items: ['Pen', 'Notebook', 'Eraser'] }                 |
|                                                                                         |
|  After { $unwind: '$items' }:                                                           |
|  -> Doc 1: { _id: 101, customer: 'Alice', items: 'Pen' }                                |
|  -> Doc 2: { _id: 101, customer: 'Alice', items: 'Notebook' }                           |
|  -> Doc 3: { _id: 101, customer: 'Alice', items: 'Eraser' }                             |
+-----------------------------------------------------------------------------------------+

Basic Syntax vs. Advanced Document Syntax

// Short Form: Path only
{ $unwind: "$items" }

// Advanced Form: Full configuration object
{
  $unwind: {
    path: "$items",
    preserveNullAndEmptyArrays: <boolean>,
    includeArrayIndex: <string>
  }
}

Critical $unwind Options

1. path (String, Required): The field path to the target array, prefixed with $. If the field is an embedded path, use dot notation (e.g., "$order.items").
2. preserveNullAndEmptyArrays (Boolean, Optional, Default: false):
   • When false (default): If the array field is null, missing, or an empty array ([]), MongoDB discards the document completely.
   • When true: MongoDB passes the document through to the next stage, outputting null for the missing or empty array field.
3. includeArrayIndex (String, Optional): Names a new field in the output document to hold the zero-based array index of the unrolled element (0, 1, 2, ...).

// Unwinding orders with preservation and index tracking
db.orders.aggregate([
  {
    $unwind: {
      path: "$lineItems",
      preserveNullAndEmptyArrays: true, // Keep orders that have 0 line items
      includeArrayIndex: "itemPosition"  // Injects: itemPosition: 0, 1, 2...
    }
  }
]);

The Classic Pattern: Unwind -> Group -> Aggregate

To aggregate data nested inside arrays across multiple documents, unwind the array first, group on the extracted array elements, and compute summary metrics:

// Calculate total revenue and units sold per product SKU across all orders
db.orders.aggregate([
  { $match: { status: "COMPLETED" } },
  { $unwind: "$items" },
  {
    $group: {
      _id: "$items.sku",
      totalUnitsSold: { $sum: "$items.quantity" },
      totalRevenue: { $sum: { $multiply: [ "$items.quantity", "$items.unitPrice" ] } }
    }
  },
  { $sort: { totalRevenue: -1 } }
]);

---

4. Reshaping Documents: $replaceWith and $replaceRoot

When working with nested subdocuments or data resulting from joins, you frequently need to promote an embedded document to become the top-level root document, discarding the original parent structure.

MongoDB provides $replaceRoot and its convenient shortcut $replaceWith (introduced in MongoDB 4.2).

Syntax

// Using $replaceRoot
{
  $replaceRoot: {
    newRoot: <replacement expression>
  }
}

// Using $replaceWith (Direct alias to $replaceRoot with newRoot)
{
  $replaceWith: <replacement expression>
}

Example 1: Promoting an Embedded Subdocument

Consider a collection where user profiles are stored under a profile subdocument:

// Input Document:
// { _id: 101, username: "jdoe", profile: { firstName: "John", lastName: "Doe", city: "Austin" }, status: "A" }

db.users.aggregate([
  {
    $match: { status: "A" }
  },
  {
    $replaceWith: "$profile"
  }
]);

// Output Document:
// { firstName: "John", lastName: "Doe", city: "Austin" }

Example 2: Merging Root Defaults with $mergeObjects

$replaceWith is often combined with the $mergeObjects expression to combine default application settings with document overrides while retaining top-level fields:

db.settings.aggregate([
  {
    $replaceWith: {
      $mergeObjects: [
        { theme: "dark", notifications: true, timeoutSec: 300 }, // Defaults
        "$userPreferences" // Overrides from document
      ]
    }
  }
]);

Error Rule: The expression provided to newRoot or $replaceWith must evaluate to a valid BSON document. If it evaluates to a string, number, array, or missing field, the aggregation operation aborts with a runtime error.