7.3 Cursor Batching, Streaming & Memory Handling

7.3 Cursor Batching, Streaming & Memory Handling

When an application queries MongoDB for hundreds of thousands or millions of documents, transferring the entire dataset in a single monolithic payload would overwhelm network buffers, saturate client memory, and cause catastrophic application crashes. To solve this problem, MongoDB implements Cursors, Network Batching Protocols, and Server-Side Cursor Management.

A clear understanding of how cursors work, how the getMore wire protocol operates, and how to stream large datasets without triggering memory exhaustion is a core competency tested on the MongoDB Associate Developer Exam.

1. Cursor Architecture & Lazy Evaluation

A Cursor is not an in-memory array of documents. Rather, it is an active pointer to a result set residing on the database server (mongod or mongos).

Lazy Evaluation Mechanics

When a developer executes a find() query in an application driver:

// In Node.js / PyMongo / Java:
const cursor = db.collection("orders").find({ status: "COMPLETED" });

At the moment this line executes, no database query has been sent over the wire, and no documents have been transferred. The driver simply instantiates a local cursor object with the query specifications. Execution is deferred until the application attempts to read the first document (e.g., via await cursor.next(), for await (const doc of cursor), or cursor.stream()).

+---------------------------------------------------------------------------------------------------+
|                                 MongoDB Cursor Batching Protocol                                  |
|                                                                                                   |
|  1. INITIAL QUERY COMMAND                                                                         |
|     Client  ====================== find({ status: "COMPLETED" }) ======================> Server |
|     Client  <================ { cursor: { id: 8941029412L, firstBatch: [...] } } <======= Server |
|     * First Batch Limit: 101 documents OR 1 MB of BSON data (whichever limit is hit first).       |
|                                                                                                   |
|  2. SUBSEQUENT BATCH RETRIEVAL (getMore)                                                          |
|     Client  ============== getMore: 8941029412L, collection: "orders" ==================> Server |
|     Client  <==================== { cursor: { nextBatch: [...] } } <====================== Server |
|     * Subsequent Batch Limit: Up to 16 MiB of BSON data (or configured batchSize).                |
|                                                                                                   |
|  3. CURSOR CLOSURE                                                                                |
|     Client  ============== getMore: 8941029412L, collection: "orders" ==================> Server |
|     Client  <==================== { cursor: { id: 0L, nextBatch: [...] } } <============== Server |
|     * Server returns cursorId = 0, indicating all documents drained; server frees cursor state.  |
+---------------------------------------------------------------------------------------------------+

The Batching Protocol: Initial Batch vs. getMore

MongoDB divides query results into discrete batches transmitted over the network:

1. The Initial Batch (find command):

   • The server evaluates the query and returns the First Batch (firstBatch).
   • Batch Threshold: Contains a maximum of 101 documents or 1 Megabyte (1 MB) of BSON data, whichever limit is reached first.
   • If the entire query result set fits within this initial limit, the server returns cursorId: 0 immediately, closing the cursor in a single network round-trip.
   • If more matching documents remain, the server assigns a non-zero 64-bit integer cursorId and pins the cursor state in server RAM.

2. Subsequent Batches (getMore command):

   • When the client application iterates past the last document in its local buffer, the driver transparently sends a getMore command with the cursorId.
   • Batch Threshold: Subsequent batches contain up to 16 mebibytes (16 MiB) of BSON data — the BSON maximum message size — or the limit specified by batchSize(). batchSize() can only lower this ceiling, never raise it.
   • This process repeats until the query is exhausted.

3. Cursor Termination (cursorId: 0):

   • Once the last document has been transmitted, the server returns cursorId: 0, and the database engine immediately releases all server-side memory, locks, and resources allocated to the cursor.

2. Cursor Configuration & Timeout Rules

Tuning Batch Size with batchSize()

Developers can override the default batch size using the .batchSize() cursor method:

// Override batch size to 500 documents per network round-trip
const cursor = db.collection("telemetry")
  .find({ device_type: "SENSOR_V2" })
  .batchSize(500);

Architectural Tradeoffs of batchSize():

• Smaller Batch Size (e.g., batchSize(20)): Reduces memory consumption on the client and server. Useful when documents are large (several megabytes each) or when processing each document is slow, reducing wasted data transfer if the consumer aborts early.
• Larger Batch Size (e.g., batchSize(1000)): Maximizes network throughput by minimizing the number of getMore round-trips when processing millions of small documents in bulk ETL pipelines.

The 10-Minute Cursor Idle Timeout

By default, the MongoDB server automatically closes active cursors that have been idle for more than 10 minutes (600,000 ms).

Why Idle Timeouts Occur:

If an application retrieves a batch of 100 documents and performs slow, blocking processing on each item (e.g., sending an email or calling a third-party HTTP API for 10 seconds per item), processing the batch takes 100 * 10 = 1,000 seconds (~16.6 minutes). When the driver finally attempts to fetch the next batch with getMore, the server has already deleted the idle cursor, throwing a CursorNotFound exception (Error Code 43).

Disabling Timeouts with noCursorTimeout: true

For long-running batch jobs, developers can disable the 10-minute timeout:

// Node.js: Disabling cursor timeout
const cursor = db.collection("large_dataset")
  .find({}, { noCursorTimeout: true })
  .batchSize(50);

CRITICAL PRODUCTION WARNING (Memory Leak Risk): When noCursorTimeout: true is set, the MongoDB server will never automatically close the cursor. If the client application crashes, encounters an unhandled exception, or exits the loop early without draining all documents, the cursor remains permanently open on mongod, consuming server RAM and locks until the database process is restarted.

Mandatory Rule: When using noCursorTimeout: true, you MUST wrap iteration in a try...finally block and explicitly invoke cursor.close() in the finally clause.

// Correct Pattern for noCursorTimeout in Node.js
const cursor = db.collection("large_dataset").find({}, { noCursorTimeout: true });

try {
  for await (const doc of cursor) {
    await processHeavyDocument(doc);
  }
} finally {
  // Guarantees cursor is closed on mongod even if processing throws an error!
  await cursor.close();
}

# Correct Pattern for no_cursor_timeout in Python (PyMongo)
cursor = db.large_dataset.find({}, no_cursor_timeout=True).batch_size(100)
try:
    for doc in cursor:
        process_heavy_task(doc)
finally:
    cursor.close()  # Mandatory cleanup

3. Iteration Paradigms & The toArray() Memory Trap

How an application iterates over a cursor fundamentally impacts memory consumption and garbage collection behavior.

The toArray() Anti-Pattern

// DANGEROUS ANTI-PATTERN: toArray() on large result sets
const allOrders = await db.collection("orders").find({}).toArray(); // 1,000,000 documents!
res.json(allOrders);

Why toArray() Causes Production Outages:

• toArray() forces the driver to immediately fetch every single matching document across multiple getMore requests and allocate all documents into a single contiguous JavaScript array in heap memory.
• If the query matches 500,000 documents of 2 KB each, toArray() attempts to allocate over 1 GB of heap memory simultaneously. In Node.js (which defaults to a 2–4 GB heap limit), this immediately triggers a FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory crash.
• In Python, this causes high memory bloat and MemoryError; in Java, it triggers java.lang.OutOfMemoryError: Java heap space.

Recommended Iteration Patterns

Stream-Based Processing Example (Node.js)

Using Node.js streams allows piping millions of documents from MongoDB directly to an HTTP response or disk file with a near-zero memory footprint:

import { pipeline } from "stream/promises";
import { Transform } from "stream";

app.get("/api/export-orders", async (req, res) => {
  const cursor = db.collection("orders")
    .find({ year: 2026 })
    .batchSize(1000);

  // Transform BSON document stream to JSON lines
  const jsonLineTransformer = new Transform({
    objectMode: true,
    transform(doc, encoding, callback) {
      callback(null, JSON.stringify(doc) + "\n");
    }
  });

  res.setHeader("Content-Type", "application/x-ndjson");
  res.setHeader("Content-Disposition", 'attachment; filename="orders.ndjson"');

  try {
    // Handles backpressure automatically: pauses cursor when client network is slow
    await pipeline(cursor.stream(), jsonLineTransformer, res);
  } catch (err) {
    console.error("Streaming pipeline failed:", err);
  }
});