8.2 Data Ingestion & Export Utilities

8.2 Data Ingestion & Export Utilities

In enterprise application development and operations, developers frequently need to load initial seed data, ingest external datasets from third-party partners (such as CSV or JSON feeds), and export database collections for reporting, analytics, or external system integration. MongoDB provides two specialized command-line utilities within the MongoDB Database Tools suite for this purpose: mongoimport and mongoexport.

Understanding how to configure these tools, select appropriate ingestion modes (insert, upsert, merge), manage schema headers, and evaluate the trade-offs of BSON type fidelity loss versus programmatic driver bulk writes is essential for the MongoDB Certified Associate Developer Exam.

---

1. Overview of MongoDB Database Tools Architecture

Unlike mongosh, mongoimport and mongoexport are standalone command-line binaries executed from the operating system shell (e.g., Bash, Zsh, or PowerShell), not from within mongosh. They establish their own direct TCP connections to the MongoDB cluster via standard connection strings (--uri).

Common Connection Options for CLI Tools

All MongoDB Database Tools share a unified set of connection and authentication parameters:

# Standard connection using a URI string
mongoimport --uri="mongodb+srv://dbUser:SecretPass@cluster0.example.com/ecommerce" ...

# Explicit host, port, and authentication flags
mongoexport --host=localhost --port=27017 --db=inventory --collection=products \
  --username=appAdmin --password=SecretPass --authenticationDatabase=admin ...

---

2. mongoimport: High-Throughput Data Ingestion

mongoimport reads content from an input file or standard input (stdin) and inserts, upserts, or merges the data into a target MongoDB collection. It natively supports JSON, CSV (Comma-Separated Values), and TSV (Tab-Separated Values).

+-----------------------------------------------------------------------------------------+
|                                 mongoimport Ingestion Pipeline                          |
|                                                                                         |
|  [ File / stdin ] ---> [ Format Parser ] ---> [ Mode: insert|upsert|merge ] ---> [ DB ] |
|  (JSON/CSV/TSV)        (--columnsHaveTypes)   (Resolves on --upsertFields)              |
+-----------------------------------------------------------------------------------------+

Core Ingestion Modes (--mode)

A critical exam concept is understanding how mongoimport handles existing documents and primary key conflicts via the --mode parameter:

Practical Mode Comparison Example:

Suppose a collection contains an existing document:

{ "_id": 101, "sku": "LAPTOP-X", "price": 1200, "warranty": "2yr", "warehouse": "Austin" }

Now, mongoimport processes an incoming record with { "_id": 101, "sku": "LAPTOP-X", "price": 1150 }:

1. --mode=insert: The import operation detects a duplicate key error on _id: 101 and skips/logs an error. The existing document remains completely unchanged.
2. --mode=upsert --upsertFields=_id: The existing document is completely replaced. The resulting document becomes { "_id": 101, "sku": "LAPTOP-X", "price": 1150 }. The unmentioned fields (warranty, warehouse) are deleted/lost!
3. --mode=merge --upsertFields=_id: The input fields are applied via an update. The resulting document is { "_id": 101, "sku": "LAPTOP-X", "price": 1150, "warranty": "2yr", "warehouse": "Austin" }. The existing warranty and warehouse fields are safely preserved.

Critical mongoimport Command-Line Options

• --file <path>: Path to the source file. If omitted, mongoimport reads from stdin.
• --type <json|csv|tsv>: Specifies input format. Default is json.
• --drop: Drops the target collection before importing data. Invaluable for reproducible test seeding, but highly dangerous in production.
• --headerline: Used exclusively with --type=csv or --type=tsv. Instructs mongoimport to parse the first line of the input file as field names.
• --fields <f1,f2,...> / --fieldFile <path>: Specifies field names manually if the CSV/TSV lacks a header row.
• --ignoreBlanks: In CSV/TSV ingestion, ignores empty fields instead of inserting empty string fields ("") into the document.
• --jsonArray: Required when importing a single JSON file formatted as a JSON array ([ {...}, {...} ]) rather than newline-delimited JSON (NDJSON).
• --upsertFields <f1,f2,...>: Specifies the unique matching fields when --mode=upsert or --mode=merge is selected (defaults to _id).
• --columnsHaveTypes: Enables explicit type casting syntax in CSV/TSV headers (e.g., age.int32(), joined.date(YYYY-MM-DD)).

Practical mongoimport Code Examples

# 1. Importing a standard newline-delimited JSON file, dropping the old collection
mongoimport --uri="mongodb://localhost:27017/shop" \
  --collection=customers \
  --file=./customers.ndjson \
  --drop

# 2. Importing a JSON Array file with explicit merge mode on customerId
mongoimport --uri="mongodb://localhost:27017/shop" \
  --collection=customers \
  --file=./customer_updates.json \
  --jsonArray \
  --mode=merge \
  --upsertFields=customerId

# 3. Importing a CSV file using the first row as headers, ignoring blank values
mongoimport --uri="mongodb://localhost:27017/shop" \
  --collection=inventory \
  --type=csv \
  --headerline \
  --ignoreBlanks \
  --file=./warehouse_stock.csv

# 4. Typed CSV Ingestion with --columnsHaveTypes
# CSV Header: sku.string(),qty.int32(),price.decimal(),active.boolean(),created.date(2006-01-02)
mongoimport --uri="mongodb://localhost:27017/shop" \
  --collection=products \
  --type=csv \
  --columnsHaveTypes \
  --file=./typed_products.csv

---

3. mongoexport: Data Extraction & Query Filtering

mongoexport reads documents from a MongoDB collection and writes them to an output file or stdout in JSON or CSV format. It is designed for exporting datasets for human consumption or external spreadsheet/ETL tools.

Core mongoexport Options

• --collection <name> / -c <name>: Specifies the source collection.
• --out <path> / -o <path>: Output destination file path. If omitted, writes to stdout.
• --type <json|csv>: Output format (default is json). When --type=csv is selected, --fields or --fieldFile is strictly required.
• --fields <field1,field2,...> / -f: Comma-separated list of fields to include in the export.
• --fieldFile <path>: Path to a file containing one field name per line to include in the export.
• --query '<json>' / -q '<json>': BSON query filter enclosed in quotes to export a specific subset of documents.
• --sort '<json>': Sort specification to dictate export order.
• --limit <N> / --skip <N>: Restricts the total number of exported documents or skips initial records.
• --jsonFormat <canonical|relaxed>: Selects MongoDB Extended JSON specification format (Canonical vs. Relaxed).

Practical mongoexport Code Examples

# 1. Exporting filtered documents to newline-delimited JSON
mongoexport --uri="mongodb://localhost:27017/analytics" \
  --collection=events \
  --query='{ "eventType": "PURCHASE", "amount": { "$gte": 100 } }' \
  --out=./high_value_purchases.json

# 2. Exporting specific fields to CSV format
mongoexport --uri="mongodb://localhost:27017/analytics" \
  --collection=users \
  --type=csv \
  --fields="_id,email,tier,createdAt" \
  --sort='{ "createdAt": -1 }' \
  --limit=5000 \
  --out=./recent_users.csv

---

4. Extended JSON Formats & BSON Type Fidelity Loss

MongoDB stores data internally in BSON (Binary JSON), a rich binary format supporting over 20 distinct data types (including 64-bit integers, 128-bit decimal floats, dates, raw binary buffers, and ObjectIds). Standard JSON, however, natively supports only basic primitives: Strings, Numbers (all represented as IEEE 754 double-precision floats), Booleans, Arrays, Objects, and Null.

To bridge this divide, MongoDB defines the Extended JSON (v2) specification, offering two formatting modes during export:

Canonical Extended JSON vs. Relaxed Extended JSON

+-----------------------------------------------------------------------------------------+
|                        Extended JSON Formatting Modes                                   |
|                                                                                         |
|  BSON Date: ISODate("2026-09-02T10:00:00Z")                                             |
|  Canonical Mode: { "$date": { "$numberLong": "1788343200000" } } (Exact Type Fidelity) |
|  Relaxed Mode  : { "$date": "2026-09-02T10:00:00.000Z" }          (Human Readable)     |
|                                                                                         |
|  BSON Decimal128: NumberDecimal("149.99")                                               |
|  Canonical Mode: { "$numberDecimal": "149.99" }                   (Preserves Precision)|
|  Relaxed Mode  : { "$numberDecimal": "149.99" }                                        |
|                                                                                         |
|  BSON Int64: NumberLong("9223372036854775807")                                         |
|  Canonical Mode: { "$numberLong": "9223372036854775807" }        (Preserves 64-bit)   |
|  Relaxed Mode  : 9223372036854775807 or lossy float depending on standard parsers       |
+-----------------------------------------------------------------------------------------+

The Danger of BSON Type Fidelity Loss

When data is exported with mongoexport and subsequently imported with mongoimport using plain JSON or CSV:

1. Integer/Double Coercion: A 64-bit integer (NumberLong) exported to CSV or relaxed JSON may be imported back into MongoDB as a 64-bit IEEE double float (Double), losing numeric precision beyond $2^{53} - 1$.
2. Date String Coercion: A BSON Date exported to CSV becomes a raw string (e.g. "2026-09-02"). If imported without explicit --columnsHaveTypes configuration, it will be stored as a BSON String rather than a BSON Date, breaking all date-range queries ($gte, $lte) and date aggregation operators ($year, $dateToString).
3. Array and Subdocument Flattening: In CSV exports, embedded documents and arrays are converted to stringified JSON blobs within quotes, requiring post-import parsing to restore document hierarchy.

[!IMPORTANT] Exam Rule: Never use mongoexport and mongoimport for complete production database backups or disaster recovery. Because they produce text-based JSON/CSV representations, they do not preserve exact BSON byte structures, index definitions, or collation settings. Use mongodump and mongorestore for full-fidelity backups.

---

5. Decision Matrix: CLI Utilities vs. Driver Bulk Write Pipelines

Developers frequently must decide whether to use CLI utilities (mongoimport/mongoexport) or write custom application scripts using the programmatic Driver Bulk Write API (bulkWrite()).