Shell, OS, and Processes

Why Shell and OS Knowledge Matters for Datadog

Datadog Fundamentals is not a Linux certification, but many questions assume you can reason about how software runs on a host. The Agent is not a desktop application you click open—it is a long-running background process managed by the operating system. When an integration fails or metrics stop, the fix often starts in a shell: confirm the service is running, verify the hostname, inspect environment variables, or read the tail of a log file. This section builds that mental model.

Operating Systems and Background Services

On Linux, the Datadog Agent is typically installed as a systemd unit named datadog-agent. You start, stop, and check it with service-management commands rather than leaving a terminal window open. On Windows, the equivalent is a Windows Service managed through Services console or PowerShell cmdlets such as Get-Service and Restart-Service.

Why does this design matter? A daemon or service starts at boot and survives user logouts. Monitoring must continue when nobody is logged in—especially on servers. Exam scenarios often describe "metrics disappeared after reboot" or "Agent only runs when admin is logged in," which points to a service not enabled at startup or an incorrect install mode.

Processes: What the Agent Actually Is

A process is an executing instance of a program. The Agent core process loads configuration, schedules checks (integrations), forwards data to Datadog's intake endpoints, and exposes a local API for diagnostics. Child processes may run checks for databases, web servers, or custom metrics.

When troubleshooting, distinguish:

• Service down — the Agent never started or was stopped
• Process crash loop — bad configuration causes repeated failures
• Running but not reporting — process alive yet API key, network, or permission issues block submission

The ps command (or ps aux on Linux) lists running processes. While you will use Datadog's own agent status in later chapters, ps remains a universal OS-level sanity check.

Shell Commands You Should Recognize

You do not need to script in Bash for the exam, but you should understand what these commands accomplish:

Hostname stability is a recurring theme: if the OS hostname changes frequently (common misconfiguration in cloud clones), Datadog may show duplicate hosts or fragmented history. The hostname command is the quickest way to see what the OS claims its name is before comparing to the Datadog UI.

Environment Variables vs Config Files

Datadog supports configuring the Agent through datadog.yaml and through environment variables prefixed with DD_. Examples:

• DD_API_KEY — authenticates Agent traffic to your organization
• DD_SITE — selects the Datadog site (datadoghq.com, datadoghq.eu, etc.)
• DD_HOSTNAME — overrides the reported hostname when needed

Environment variables are especially important in containers and orchestrated deployments where baking secrets into images is unsafe. An exam question might ask why a containerized Agent reads DD_API_KEY from the orchestrator secret store—the answer is supplying configuration without hardcoding secrets in the image or command line history.

Precedence rules can trip you up: when the same setting exists in a file and an environment variable, the environment variable typically wins. That is intentional for twelve-factor style deployments.

Programming Language Basics in Datadog Workflows

The Computer Fundamentals domain includes light programming-language awareness. You are not tested on syntax trivia, but you should know:

• YAML structures Agent configuration (datadog.yaml, integration configs under conf.d/)
• JSON is the lingua franca of Datadog's HTTP API request and response bodies
• Python is a common choice for automation scripts that call the API because of mature HTTP and JSON libraries

A typical workflow: edit YAML locally → restart Agent → verify metrics in UI. A different workflow: Python script sends JSON to api.datadoghq.com to create dashboards. Confusing which format belongs where is a common wrong-answer trap.

Exam Scenarios

Scenario A: After editing config, metrics stop. The engineer edited YAML with tab characters. Root cause: YAML is whitespace-sensitive; tabs break parsing and the Agent may refuse to start.

Scenario B: A CI container starts the Agent with secrets injected as env vars but leaves api_key blank in the file. This can work because DD_API_KEY in the environment satisfies authentication without a file entry.

Scenario C: Metrics exist only when a developer SSHs in. Likely the Agent was started manually in an interactive session instead of as a managed service enabled at boot.

Practical Habit

After any config change, adopt a two-step rhythm: restart the Agent service and verify status output (covered deeply in Chapter 2). At the OS layer, that means confirming the service transitioned to active (running) before you open the Datadog UI. Skipping the OS check wastes time chasing UI delays when the process never restarted.