3.3 Fleet Server & Elastic Agent Integration Management

Centralized Fleet Architecture vs. Standalone Beats

In legacy Elastic Stack deployments, monitoring an enterprise estate required deploying, configuring, and updating multiple standalone Beats on every server: Filebeat for logs, Metricbeat for host performance, Packetbeat for network flows, Auditbeat for Linux audit trails, and standalone Endpoint Security agents. Each Beat required its own local YAML configuration file, separate binary updates, and independent service management. Across thousands of enterprise endpoints, configuration drift and update management became major operational bottlenecks.

The Elastic Agent unifies all telemetry collection capabilities into a single binary. Rather than manually managing local configuration files on endpoints, Elastic Agents are centrally governed through Fleet in Kibana and coordinated by Fleet Server.

Fleet Architecture and Communication Model

The Fleet architecture consists of four interconnected components:

1. Kibana Fleet UI: The centralized administrative console where SOC teams configure agent policies, add integration packages, monitor agent health, and trigger remote diagnostics.
2. Fleet Server: A scalable backend control plane (deployed as an Elastic Agent process with the Fleet Server integration enabled) that brokers communication between thousands of managed Elastic Agents and Elasticsearch.
3. Elasticsearch: The central datastore that houses agent policies, enrollment tokens, integration packages, and the ingested telemetry data streams.
4. Elastic Agent: The endpoint binary installed on servers, workstations, and cloud instances. It establishes an outbound HTTPS connection to Fleet Server (default port 8220), retrieves policy revisions, executes configured integrations, and ships normalized telemetry directly to Elasticsearch (default port 9200).

Because Elastic Agents initiate outbound connections to Fleet Server, managed endpoints located in protected enclaves, branch offices, or cloud VPCs do not require inbound firewall openings.

---

Enrollment Tokens and Agent Policies

To onboard an Elastic Agent into Fleet, the endpoint must be registered using an Enrollment Token associated with an Agent Policy.

Declarative Agent Policies

An Agent Policy is a centralized configuration document that dictates an agent's runtime behavior:

• Enabled Integrations: Specific modules configured for the host (e.g., Windows Event Logs, Linux System, Elastic Defend, AWS, Zeek).
• Collection Settings: Polling intervals, target log file paths, and parsing pipelines.
• Output Destination: The target Elasticsearch cluster and credentials used for data shipping.

Policies use automated revision versioning. When an administrator modifies an integration in the Fleet UI, Fleet increments the policy revision number. Connected Elastic Agents receive the update via long-polling, validate the new specification, and apply it atomically without restarting the host or dropping active telemetry streams.

Enrollment Tokens

Enrollment tokens authenticate agents during initial onboarding:

• Each token is permanently bound to a designated Agent Policy.
• Tokens can be scoped by environment (e.g., Production-Linux-Token, DMZ-Firewall-Token).
• If an enrollment token is compromised or a deployment campaign completes, administrators can revoke the token in the Fleet UI without affecting already-enrolled agents.

To install and enroll an Elastic Agent via the command line:

sudo ./elastic-agent install \
  --url=https://fleet-server.corp.internal:8220 \
  --enrollment-token=QUJjMTIzWHl6... \
  --certificate-authorities=/etc/pki/ca.crt

---

Integrations and Data Stream Architecture

Integrations are modular software packages downloaded from the Elastic Package Registry (EPR) that provide input configuration, ingest pipelines, index mappings and, often, Kibana dashboards.

The Elastic Defend Integration

For security analysts, the Elastic Defend (formerly Endpoint Security) integration is the primary endpoint protection mechanism. Adding Elastic Defend to an Agent Policy enables:

• Malware Protection: Scans executables when they are written or run and uses an on-host machine-learning model, plus signatures and your blocklist, to detect or block malware.
• Ransomware Protection: Detects and halts unauthorized mass encryption of user documents, canary files, and filesystem structures.
• Behavioral Protection: Detects suspicious process ancestry, API hooking, memory injection (process hollowing, thread injection), and credential dumping against LSASS.
• Configuration Presets: When you add Elastic Defend for Traditional Endpoints, you choose a preset that you can customize later:
  • Data Collection: all events, no preventions.
  • Next-Generation Antivirus (NGAV): process events plus all preventions.
  • Essential EDR: process, network and file events plus all preventions.
  • Complete EDR: all events plus all preventions. Every preset except Data Collection enables the malware, ransomware, memory threat, malicious behavior and credential theft preventions by default. The separate Cloud Workloads presets target cloud Linux hosts, collect process, network and file events with session data, and start with preventions disabled.

Data Stream Naming: type-dataset-namespace

Fleet-managed agents write all telemetry into Elasticsearch Data Streams. Data streams enforce a three-part naming hierarchy:

$\text{type}-\text{dataset}-\text{namespace}$

1. type: Broad category of telemetry. Standard values are logs, metrics, synthetics, and traces.
2. dataset: The specific log or telemetry source defined by the integration (e.g., windows.sysmon_operational, system.auth, aws.cloudtrail, endpoint.events.process).
3. namespace: An administrator-defined grouping reflecting an environment, tenant, or business unit (e.g., production, pci, development, default).

For example, Windows Sysmon telemetry collected from production servers routes to: logs-windows.sysmon_operational-production

This granular naming convention allows administrators to attach distinct Index Lifecycle Management (ILM) retention policies, ingest pipelines, and Role-Based Access Controls (RBAC) to specific datasets and namespaces independently.

---

Agent Lifecycle and Troubleshooting

Fleet provides real-time visibility into the operational state of every managed endpoint.

Elastic Agent Statuses and Remediation Matrix

Endpoint CLI Diagnostics and Enterprise Proxy Support

When investigating an unhealthy or non-responsive agent directly on an endpoint:

• Inspect Component Health:

  sudo elastic-agent status
  

  Displays the granular status of each managed component process (e.g., filebeat, metricbeat, endpoint-security).
• Generate Diagnostic Bundle:

  sudo elastic-agent diagnostics
  

  Writes a diagnostics .zip archive containing configuration snapshots, component logs and runtime metrics. Administrators can also collect this remotely from Fleet with the agent's Request diagnostics .zip action, without host SSH access.
• Enterprise Proxy Configuration: For endpoints operating in restricted enclaves without direct routing to Fleet Server, agents support HTTP/HTTPS forward proxies, set at install time or in Fleet proxy settings:

  sudo ./elastic-agent install --url=https://fleet-server.corp.internal:8220 \
    --enrollment-token=QUJjMTIzWHl6... --proxy-url=http://proxy.corp.internal:8080