Integrations and Crawlers

Quick Answer: Agent integrations are packaged checks configured under conf.d/<integration>.d/conf.yaml that collect metrics locally every ~15 seconds. Crawlers poll remote cloud or SaaS APIs when you cannot install an Agent on the resource itself. Both paths land in Datadog as metrics, but latency and troubleshooting differ.

Integrations are how Datadog collects infrastructure and middleware telemetry without custom code: CPU and disk from the host, query throughput from PostgreSQL, connection counts from Nginx, queue depth from RabbitMQ. The Fundamentals exam expects you to know how to enable an integration, verify it is running, and distinguish Agent-based collection from API-based crawlers.

Agent Integrations: Local Collection

The Datadog Agent ships with hundreds of integration packages. Each integration is a scheduled check the Agent runs on a defined interval.

Enabling an Integration on Linux

1. Navigate to /etc/datadog-agent/conf.d/<CHECK_NAME>.d/.
2. Copy conf.yaml.example to conf.yaml.
3. Edit conf.yaml with connection details (host, port, credentials, custom tags).
4. Restart the Agent or reload configuration.
5. Run datadog-agent status and inspect the Checks section.

On the Exam: The active file is always conf.yaml, not datadog.yaml and not the .example file. Renaming or deleting the example does not auto-generate a working config.

Default Collection Interval

Datadog's documentation states that standard Agent integrations and system metrics collect on a 15-second interval by default. That cadence makes host metrics feel near-real-time in dashboards. Some integrations allow min_collection_interval overrides in their config when less frequent polling is appropriate.

Verifying Integration Health

After configuration, confirm the check is loaded:

sudo datadog-agent status

Scroll to Checks. A healthy integration shows the check name, instance configuration, and last run status. If the check is missing entirely, the config file path or filename is wrong. If it appears but reports errors, credentials, network access, or permissions on the target system are the usual suspects.

The Integrations page in the Datadog UI provides a fleet-wide view. Status indicators help spot problems across many hosts:

Common Trap: Missing Data does not always mean the integration was never installed. It usually means Datadog has not seen fresh metrics recently — verify config, target service health, and Agent connectivity.

Crawlers and API-Based Integrations

Not every resource can run the Datadog Agent. Managed databases, SaaS platforms, and hyperscaler control planes live outside your VPC. For these, Datadog provides integrations that query remote APIs on a schedule — often called crawlers in exam language.

Why Cloud Metrics Arrive Later

Host metrics from a local Agent often appear within seconds. CloudWatch or Azure Monitor metrics may lag because:

1. The cloud provider publishes metrics on its own schedule.
2. Datadog's crawler polls the provider API periodically.
3. Network and processing add a small buffer.

Exam questions contrast "Agent on EC2" (fast, local) with "RDS CPU via AWS integration" (remote API, expected delay). Neither is broken — the collection architecture differs.

Integration vs DogStatsD vs Crawler

Worked Scenario: PostgreSQL Integration

A DBA enables the Postgres integration on a database host where the Agent already runs:

1. Install the integration package if not bundled: datadog-agent integration install datadog-postgres.
2. Create /etc/datadog-agent/conf.d/postgres.d/conf.yaml from the example.
3. Set host, port, username, password (or IAM auth per docs), and tags like env:prod.
4. sudo datadog-agent restart.
5. datadog-agent status → Checks → postgres shows OK.
6. Metrics Explorer: postgresql.connections grouped by db.

If status shows connection refused, the exam expects you to check firewall rules and whether Postgres accepts connections from localhost — not to reconfigure DogStatsD.

Autodiscovery Connection

In container platforms, Autodiscovery can generate integration configs from templates when containers match identifiers (labels, AD annotations). The Fundamentals outline lists autodiscovery under Networking & Agent Configuration, but Data Collection questions often assume you know integrations still produce the metrics you graph and alert on — only the config delivery mechanism changes.

Integration Management Commands

The datadog-agent integration command family manages packages:

These commands manage integration packages, not dashboards or DogStatsD ports — another favorite distractor pairing on the exam.

Log Collection Note

Host log collection also requires Agent configuration — path, source, and service fields in a logs config file or Autodiscovery template. Logs are not collected automatically from every file on disk. Fundamentals may cross-reference this when distinguishing metrics integrations from log tailing, but the activation pattern (create config, restart Agent, verify in UI) mirrors integration checks.