6.1 Lens Architecture & Multi-Layer Telemetry Visualizations

In modern Security Operations Centers (SOCs), threat detection and incident triage require rapid visual synthesis of disparate telemetry streams. When investigating an ongoing security incident, analysts cannot afford to spend critical minutes hand-crafting complex Elasticsearch aggregation JSON payloads or managing rigid visualization builders. Kibana Lens serves as the primary visual analysis framework within the Elastic Stack, combining an intuitive drag-and-drop interface with automatic chart suggestions for fast exploratory analysis.

Understanding the internal architecture of Lens, its multi-layer composition capabilities, and its dual Y-axis scaling mechanics is essential for building actionable SOC dashboards, conducting exploratory threat hunting, and succeeding on the Elastic Certified SIEM Analyst examination.

---

The Modern Kibana Lens Paradigm in SecOps

Historically, the Elastic Stack provided multiple specialized visualization tools within the Visualize Library, including Coordinate Maps, TSVB (Time Series Visual Builder), Timelion, and legacy Aggregation-Based visualizations. While capable, these legacy tools operated in functional silos: a chart built in TSVB could not be converted to a data table, and combining multiple indices onto a single time-series canvas required obscure script syntaxes or separate panel hacks.

+---------------------------------------------------------------------------------------------------+
|                                    KIBANA LENS WORKSPACE LAYOUT                                   |
+---------------------------------------------------------------------------------------------------+
| [Data View: logs-* v]  [KQL: event.category: "network" or event.category: "authentication"     ] |
| [Time Picker: Last 24 Hours (2026-09-29T12:00:00Z to 2026-09-30T12:00:00Z)                    ] |
+-----------------------------------+-----------------------------------+---------------------------+
| FIELD LIST & DATA VIEW            | CENTRAL INTERACTIVE CANVAS        | OPERATION DOCK (Right)    |
| Search fields: [ user.          ] |                                   | Chart Type: [ XY Chart v] |
| --------------------------------- |   Layer 1: Network Bytes (Area)   | ------------------------- |
| # @timestamp             [date]   |   Layer 2: Auth Failures (Line)   | X-Axis:                   |
| t user.name           [keyword]   |                                   |  - @timestamp (Date Hist) |
| t user.target.name    [keyword]   |   [  /\       /\            ]     | Left Y-Axis (Bytes):      |
| # network.bytes        [number]   |   [ /  \  /\ /  \   /\  /\  ]     |  - sum(network.bytes)     |
| t event.outcome       [keyword]   |   [=====\/==v====\=/==\/==\ ]     | Right Y-Axis (Count):     |
| t host.name           [keyword]   |                                   |  - count(failures)        |
| --------------------------------- |                                   | Break down by:            |
| Top Values for user.name:         |                                   |  - user.name (Top 5)      |
| 1. svc-admin (42%)                |                                   | Appearance:               |
| 2. backup_op (18%)                |                                   |  - Stacking: Stacked      |
| 3. root      (12%)                |                                   |  - Missing values: Linear |
+-----------------------------------+-----------------------------------+---------------------------+

Kibana Lens unifies these capabilities into a single, cohesive authoring environment characterized by four primary architectural zones:

1. Data View Selector & Search Header: Located at the upper-left, this control establishes the primary data context (logs-*, .alerts-security.alerts-default, or metrics-*). Crucially, Lens allows different layers on the same canvas to bind to distinct Data Views.
2. Field List & Cardinality Inspector: Positioned on the left panel, the field list displays all mapped Elastic Common Schema (ECS) fields. Each field displays an icon representing its indexed data type (date, keyword, number, ip, boolean). Selecting any field displays a summary card with its top values and distribution within the active time window, giving a quick feel for its cardinality.
3. Central Interactive Canvas: The visualization workspace where chart components render in real time. Lens dynamically dispatches optimized aggregation requests to Elasticsearch as fields are manipulated, utilizing client-side debounce caching to maintain UI fluidity.
4. Configuration & Operation Dock: Positioned on the right panel, this inspector exposes dimensional bindings (X-axis, Y-axis, Breakdown, Metric, Rows, Columns), layer management controls, chart type switchers, color palettes, and axis extent settings.

---

The Smart Suggestions Engine & Schema-Driven Prototyping

A hallmark architectural feature of Kibana Lens is its Smart Suggestions Engine. When an analyst drags a field from the left panel onto the central canvas, Lens does not force the user to define Elasticsearch aggregation pipelines manually (e.g., configuring terms aggregations nested inside date_histogram aggregations).

Instead, the engine inspects the field's underlying Lucene mapping and distribution characteristics:

• Temporal Fields (date): Dragging @timestamp automatically creates a date_histogram aggregation, intelligently selecting an interval (e.g., 30 seconds, 5 minutes, 1 hour) based on the current time picker window.
• Categorical Fields (keyword, ip): Dragging user.name or destination.ip generates a terms aggregation proposing horizontal bar charts, donut charts, or treemaps displaying the top 5 values.
• Numeric Metrics (number, long, float): Dragging network.bytes suggests summary metric cards, average lines, or histogram distributions.
• Multi-Field Combinations: Dragging both @timestamp and event.outcome simultaneously prompts Lens to suggest a stacked bar chart or a multi-line time series showing event outcomes over time.

While suggestions provide rapid 1-click prototyping, security analysts retain full manual control in the right-hand operation dock. Analysts can override suggested intervals, configure custom date ranges, adjust term bucket sizes from top 5 to top 50, enforce "Group other values as 'Other'", or inject custom mathematical formulas.

---

Multi-Layer Visualization Architecture

In real-world security investigations, single-source visualizations fail to convey true operational context. For example, viewing a surge in external network traffic alone does not reveal whether the traffic represents legitimate media streaming or active data exfiltration following a credential compromise. Conversely, inspecting authentication failures in isolation does not show whether an attacker successfully bypassed perimeter barriers.

Lens resolves this through its Multi-Layer Architecture, which enables an analyst to combine multiple independent data streams or index patterns on a unified coordinate canvas (XY Chart).

+---------------------------------------------------------------------------------------------------+
|                         LENS MULTI-LAYER COORDINATE CANVAS ARCHITECTURE                           |
+---------------------------------------------------------------------------------------------------+
|                                                                                                   |
|  LEFT Y-AXIS (Bytes)                                                    RIGHT Y-AXIS (Count)      |
|  [logs-network.*]                                                       [logs-auth.*]             |
|                                                                                                   |
|  50 GB |-------------------------------------------------------------| 500 Failures               |
|        |                       *** (Layer 2: Auth Failure Peak)      |                            |
|  40 GB |                      *   *                                  | 400 Failures               |
|        |                     *     *                                 |                            |
|  30 GB |           /\       *       *                                | 300 Failures               |
|        |          /  \     *         *                               |                            |
|  20 GB |         /    \   *           *        /\                    | 200 Failures               |
|        |        /      \ *             *      /  \                   |                            |
|  10 GB |       / (Layer 1: Network Bytes)    /    \                  | 100 Failures               |
|        |______/__________*_____________\____/______\_________________|                            |
|   0 GB +-------------------------------------------------------------+ 0 Failures                 |
|       00:00    04:00    08:00    12:00    16:00    20:00    00:00                                 |
|                                  @timestamp                                                       |
|                                                                                                   |
|  LAYER 1: Area Chart | logs-network.* | Metric: sum(network.bytes)       | Bound to Left Y-Axis   |
|  LAYER 2: Line Chart | logs-auth.*    | Metric: count(outcome='failure') | Bound to Right Y-Axis  |
+---------------------------------------------------------------------------------------------------+

Constructing Multi-Source SecOps Overlays

To construct an end-to-end multi-layer security visualization in Lens:

1. Base Layer (Perimeter Traffic):
   • Select Data View: logs-network.*.
   • Drag @timestamp to the X-axis (Date Histogram).
   • Drag network.bytes to the Y-axis, configured as sum(network.bytes).
   • Set Chart Type to Area Chart to establish the background baseline of network egress volume.
2. Secondary Layer (Authentication Telemetry):
   • Click + Add layer in the operation dock.
   • In the layer configuration, switch the Data View to logs-auth.*.
   • Set Metric to count() with a KQL filter: event.outcome: "failure".
   • Set Chart Type to Line Chart with prominent styling (e.g., solid crimson line with visible points).
3. Tertiary Layer (Alert Annotations):
   • Click + Add layer.
   • Switch Data View to .alerts-security.alerts*.
   • Set Metric to count() filtered by kibana.alert.rule.rule_id: *.
   • Set Chart Type to Bar Chart or point scatter to highlight when automated detection rules fired relative to the raw network and authentication spikes.

This multi-layering capability eliminates the need for expensive ingestion-time data flattening or cross-index ETL joins. Elasticsearch coordinates the independent aggregations across shards, and Kibana Lens renders them synchronously aligned to the shared @timestamp horizontal axis.

---

Configuring Dual Y-Axes for Disparate Metric Scales

A critical technical challenge when layering security telemetry is the scale disparity problem. Consider the scenario above:

• In Layer 1, total network egress is measured in gigabytes ($10^9$ bytes), reaching values of $40,000,000,000$ bytes per hour.
• In Layer 2, failed SSH authentication attempts are measured in units or hundreds ($10^1$ to $10^2$), reaching peak values of $350$ failures per hour.

If both layers are rendered against a single shared Y-axis, the $350$ authentication failures will be plotted at the extreme bottom of the scale ($350 / 40,000,000,000 \approx 0.000000875\%$), appearing as an invisible flat line resting directly on the zero baseline. The analyst would completely miss the brute-force attack.

Dual Y-Axis Configuration Steps in Lens

Lens overcomes this limitation through independent Y-axis assignment and scale configuration:

1. Axis Binding:
   • In the Operation Dock for Layer 1 (network.bytes), navigate to Y-axis and assign it to Left axis.
   • In the Operation Dock for Layer 2 (failures), navigate to Y-axis and assign it to Right axis.
2. Unit Formatting:
   • Configure the Left Y-Axis format to Bytes (automatically scaling raw integers to KB, MB, GB, or TB).
   • Configure the Right Y-Axis format to Number (formatted with integer separators, e.g., 1,000).
3. Independent Scale Modes:
   • Linear Scale: The standard default, mapping values proportionally to vertical distance.
   • Logarithmic Scale: Available under advanced axis settings. In telemetry sets where values range across multiple orders of magnitude (e.g., 1 event to 10,000,000 events), enabling logarithmic scaling prevents massive volume spikes from compressing baseline activity.
   • Square Root Scale: Useful for attenuating extreme outliers while maintaining zero-origin continuity.
4. Custom Extents (Bounds):
   • Analysts can set explicit minimum and maximum bounds for either axis. For example, pinning the minimum to 0 prevents Lens from auto-zooming into minor fluctuations, preserving baseline stability.

---

Dynamic Chart Type Switching

During active security triage, an analyst's investigative needs evolve rapidly. An analyst might start with a high-level summary line chart to spot a temporal anomaly, transition to a stacked bar chart to identify which hosts contributed to the spike, and finally switch to a data table to export specific source IP addresses for firewall blocking.

In legacy Kibana tools, changing visualization paradigms required discarding the existing panel, navigating back to the Visualize Library, selecting a new chart builder, and re-configuring all aggregations from scratch.

In Kibana Lens, Chart Type Switching is dynamic and lossless:

+---------------------------------------------------------------------------------------------------+
|                         DYNAMIC CHART TYPE SWITCHING PIPELINE IN LENS                             |
+---------------------------------------------------------------------------------------------------+
| Underlying Aggregation Definition:                                                                |
|   - Time Dimension: @timestamp (1h interval)                                                      |
|   - Categorical Dimension: user.name (Top 5 terms)                                                |
|   - Primary Metric: count(kql='event.outcome: "failure"')                                         |
+---------------------------------------------------------------------------------------------------+
                                                  |
          +---------------------------------------+---------------------------------------+
          |                                       |                                       |
          v                                       v                                       v
   [ XY BAR CHART ]                        [ DONUT CHART ]                         [ DATA TABLE ]
   X: @timestamp                           Slices: user.name                       Rows: @timestamp, user.name
   Y: count()                              Size: count()                           Columns: count()
   Stack: user.name                        (Aggregates total window)               (Tabular exportable audit)

When an analyst toggles the Chart Type dropdown in the Lens Operation Dock, Lens intelligently rebinds existing dimension slots:

• Bar to Area / Line: Maintains @timestamp on the X-axis and metric on the Y-axis. The analyst can toggle between stacked area (showing cumulative volume) and multi-line (showing individual entity trajectories) with zero re-query latency.
• XY Chart to Donut / Treemap: Collapses the temporal X-axis and maps the breakdown categorical dimension (user.name) to donut slices or treemap rectangles, instantly displaying overall entity distribution across the entire investigation window.
• XY Chart to Data Table: Converts the X-axis date buckets and breakdown categories into tabular rows, and the Y-axis aggregations into numeric columns. This facilitates rapid sorting, multi-column filtering, and CSV export for external reporting.
• XY Chart to Metric (Single Stat): Summarizes the metric aggregation into a single large-font KPI card, often used at the top of SOC overview dashboards to show total failed logins or active compromised hosts.

---

Multi-Field Breakdown Dimensions

To identify the root cause of an anomalous telemetry spike, analysts must dissect high-level metrics across categorical attributes. Lens provides Breakdown Dimensions, enabling multi-field categorical splitting.

Managing High-Cardinality Telemetry

Security data frequently exhibits extreme cardinality. For instance, breaking down network traffic by destination.ip in an enterprise network might return tens of thousands of unique addresses. Attempting to render 10,000 distinct lines on an XY chart creates visual chaos and exhausts browser memory.

Lens handles high cardinality through robust term controls:

• Top N Values: Analysts specify the exact number of top terms to evaluate (e.g., Top 5, Top 10, Top 20) based on metric volume.
• Group Other Values as 'Other': When enabled, Lens calculates the aggregate sum of all values falling outside the Top N threshold and groups them into a distinct, neutrally colored "Other" category. This guarantees mathematical accuracy: the chart displays total event volume without dropping missing records.
• Directional Sorting: Analysts can sort terms descending (to highlight top attackers or top talkers) or ascending (to hunt for rare, low-frequency anomalies such as unusual child processes or rare user agents).

Nested Breakdowns

Lens supports multi-level breakdowns in tables and charts. For example, an analyst can configure a primary breakdown by host.name, followed by a secondary nested breakdown by process.name. In an XY bar chart, this renders grouped or stacked bars displaying which specific processes are driving resource consumption or threat alerts across individual endpoints.

---

Architectural Comparison: Lens vs. Legacy Visualization Builders

The following table highlights the architectural differences between Kibana Lens and legacy Kibana visualization tools:

Exam Preparation Note: Elastic's objective is to construct Lens visualizations for security use cases. Because the SIEM Analyst exam is a knowledge-based (cognitive) exam, expect questions that ask which Lens feature or setting produces a given result, such as layers, axis settings, breakdowns, chart switching or formulas.