10.3 Nanolog Streaming Service (NSS) & SIEM Integration

10.3 Nanolog Streaming Service (NSS) & SIEM Integration

Quick Answer: The Nanolog Streaming Service (NSS) is Zscaler's enterprise log forwarding fabric. It extracts high-volume transactional logs from the cloud-native Nanolog engine and streams them in real time over encrypted TLS connections to enterprise Security Information and Event Management (SIEM) and Security Operations Center (SOC) platforms (e.g., Splunk, Microsoft Sentinel, IBM QRadar, Datadog). Organizations deploy either on-premises NSS Virtual Appliances (with dedicated Management and Service network interfaces) or serverless Cloud NSS for direct cloud-to-cloud ingestion.

---

1. The Nanolog Architecture & Patented Compression

Traditional on-premise proxies and firewalls write raw text log records directly to local disk arrays. In high-throughput enterprise networks processing hundreds of thousands of transactions per second, disk I/O becomes an immediate bottleneck, leading to dropped log records, high appliance latency, and exorbitant storage costs.

Zscaler solves this with its patented Nanolog Technology:

+-----------------------------------------------------------------------------+
|                        NANOLOG COMPRESSION PIPELINE                         |
+-----------------------------------------------------------------------------+

  [User Web / Firewall Traffic] 
                |
                v
  [ZIA Public Service Edge] -- (Single-Scan Multi-Action Processing)
                |
                v (Raw Transactional Telemetry)
  +-------------------------------------------------------------------------+
  |                      ZSCALER NANOLOG CLUSTER                            |
  |  * In-Memory Streaming Tokenization                                     |
  |  * Columnar Dynamic Compression (~10:1 to 50:1 Ratio)                   |
  |  * Indexed Real-Time Global Telemetry Fabric                            |
  +-------------------------------------------------------------------------+
         /                                                     \
        /                                                       \
       v                                                         v
  [Interactive Web/FW Insights]                       [NSS Streaming Outbound]
  (Instant query response in GUI)                     (Real-time push to SIEM)

Core Properties of Nanolog:

• Extreme Compression Ratio: Nanolog compresses raw transaction logs by a factor of 10:1 to 50:1 in memory before persistent storage, reducing exabytes of raw text into compact binary telemetry.
• Zero Disk Bottlenecks: Nanolog operates primarily in high-speed volatile memory arrays across Zscaler data centers, enabling real-time query responses in admin portal dashboards regardless of transaction volume.
• Decoupled Architecture: User traffic forwarding is completely decoupled from log persistence. Even if an enterprise SIEM or logging pipeline experiences an outage, user web transactions are never delayed or dropped.

---

2. On-Premises NSS Virtual Appliance Architecture

For organizations streaming logs to an on-premises SIEM collector (such as a local Splunk Heavy Forwarder, QRadar Event Collector, or Syslog daemon), Zscaler provides the NSS Virtual Appliance (packaged as an OVA for VMware ESXi or VHD for Microsoft Hyper-V, and deployable in AWS EC2 or Azure VMs).

+-----------------------------------------------------------------------------+
|                    NSS VIRTUAL APPLIANCE DUAL-NIC DESIGN                    |
+-----------------------------------------------------------------------------+

  +-------------------------------------------------------------------------+
  |                          ZSCALER NANOLOG CLOUD                          |
  +------------------------------------+------------------------------------+
                                       |
                                       | Outbound TLS Tunnel
                                       | (TCP Port 443)
                                       v
  +=========================================================================+
  |                      NSS VIRTUAL APPLIANCE (VM)                         |
  |                                                                         |
  |  +-------------------------------------------------------------------+  |
  |  | Management Interface (eth0 / MGT)                                 |  |
  |  | - IP: 192.168.10.50 (Internal Management Subnet)                  |  |
  |  | - Function: DNS, NTP, SSH Management, Outbound TLS to Nanolog    |  |
  |  +-------------------------------------------------------------------+  |
  |                                  | Internal Shared Memory Ring          |
  |  +-------------------------------------------------------------------+  |
  |  | Service Interface (eth1 / SMINT)                                  |  |
  |  | - IP: 10.100.20.75 (Dedicated Logging Subnet)                     |  |
  |  | - Function: Streams Formatted Logs (TCP/Syslog/TLS) to SIEM       |  |
  |  +-------------------------------------------------------------------+  |
  +====================================+====================================+
                                       |
                                       | Formatted Syslog / TCP Stream
                                       | (e.g., TCP Port 514 / 6514 / 9997)
                                       v
  +-------------------------------------------------------------------------+
  |                  ENTERPRISE SIEM / LOG COLLECTOR                        |
  |            (Splunk Heavy Forwarder, QRadar, ArcSight)                   |
  +-------------------------------------------------------------------------+

Critical Dual-Interface Requirements:

1. Management Interface (eth0 / MGT):
   • Used for administrative SSH access, DNS resolution, NTP synchronization, and establishing the outbound control and data tunnel to the Zscaler Nanolog Cloud.
   • Initiates an outbound-only TLS connection on TCP Port 443 to the Zscaler cloud. No inbound firewall ports are ever opened from the internet into the customer network.
   • Authenticates mutually with the Zscaler cloud using an X.509 SSL certificate generated during NSS provisioning in the admin portal.
2. Service Interface (eth1 / SMINT):
   • Dedicated strictly to transmitting outbound formatted log streams to the enterprise SIEM or syslog server.
   • Operates on a dedicated, high-speed network segment to prevent logging bursts from saturating management traffic.

[!CAUTION] NSS Dual-NIC Routing Rule: The Management Interface (eth0) and Service Interface (eth1) must reside on different IP subnets. Assigning both interfaces to the same subnet causes asymmetric kernel routing faults within the FreeBSD-based NSS operating system, breaking the Nanolog data feed.

---

3. NSS Types & Cloud NSS (Serverless Architecture)

Zscaler supports multiple specialized NSS deployment models depending on log types and destination architecture:

+-----------------------------------------------------------------------------+
|                        NSS DEPLOYMENT MODEL MATRIX                          |
+-----------------------------------------------------------------------------+

  1. ON-PREMISES NSS FOR WEB     --> Streams L7 HTTP/HTTPS, Web Threats, DLP
  2. ON-PREMISES NSS FOR FW      --> Streams L3/L4 Sessions, DNS, Cloud IPS
  3. ON-PREMISES NSS FOR ALERTS  --> Streams Admin Audit & Security Alerts
  4. CLOUD NSS (SERVERLESS)      --> Direct Cloud-to-Cloud API/HTTPS Streaming

+-----------------------------------------------------------------------------+
|                        CLOUD NSS SERVERLESS PIPELINE                        |
+-----------------------------------------------------------------------------+

  [Zscaler Nanolog Cloud] 
            |
            | Direct HTTPS REST API Stream (TLS 443)
            | (Token-Authenticated Webhook)
            v
  +-------------------------------------------------------------------------+
  |                       CLOUD SIEM / DATA LAKE                            |
  |  * Microsoft Sentinel (via Azure Log Analytics / Event Hub)             |
  |  * Splunk Cloud (via HTTP Event Collector - HEC)                        |
  |  * AWS S3 / Datadog / Google Chronicle Data Ingestion                   |
  +-------------------------------------------------------------------------+

---

4. ZPA Log Streaming Service (LSS)

In Zscaler Private Access (ZPA), log streaming is handled by the Log Streaming Service (LSS). Unlike ZIA (which uses standalone NSS VMs), ZPA embeds LSS functionality directly into ZPA App Connectors or dedicated LSS App Connector instances.

+-----------------------------------------------------------------------------+
|                          ZPA LSS LOG STREAMING PATH                         |
+-----------------------------------------------------------------------------+

  [User Accesses Private App] ---> [ZPA Public Service Edge Broker]
                                                 |
                                                 v (Encrypted Telemetry)
  [ZPA Cloud Central Authority] -----------------+
         |
         v (Streams Log Context to Designated App Connector)
  +-------------------------------------------------------------------------+
  |                   ZPA APP CONNECTOR (LSS ENABLED)                       |
  |  * Formats raw private app session data into CEF, JSON, or Syslog       |
  |  * Establishes outbound TLS/TCP connection to internal SIEM             |
  +------------------------------------+------------------------------------+
                                       |
                                       | Formatted Log Output (TCP / TLS)
                                       v
                       [Enterprise Internal SIEM / SOC]

ZPA LSS Supported Log Streams:

1. User Activity Logs: Granular connection events for private applications (User SAML ID, Client IP, Target Application Segment, App Connector used, Connection Status, Bytes Transferred).
2. User Status Logs: Tracks user authentication, enrollment, ZCC version updates, and logout/re-authentication cycles.
3. App Connector Status Logs: Telemetry on App Connector CPU/RAM health, TLS tunnel status, and round-trip times.
4. Audit Logs: Administrative configuration modifications in the ZPA portal.

---

5. Custom Feed Formatting & SIEM Parsing Strings

NSS and LSS allow administrators to customize output feed formats to match the exact ingestion schemas of target SIEM platforms, including Common Event Format (CEF), JSON, and Key-Value Pairs (Splunk CIM).

+-----------------------------------------------------------------------------+
|                      SAMPLE CUSTOM NSS LOG FEED FORMATS                     |
+-----------------------------------------------------------------------------+

  1. JSON FORMAT (Cloud SIEMs, Elasticsearch, Datadog):
  {
    "datetime": "%s{time}",
    "user": "%s{user}",
    "department": "%s{dept}",
    "client_ip": "%s{cip}",
    "url": "%s{url}",
    "action": "%s{action}",
    "threat_name": "%s{threatname}",
    "bytes_received": %d{respsize}
  }

  2. CEF FORMAT (ArcSight, Microsoft Sentinel):
  CEF:0|Zscaler|ZIA|6.2|%s{action}|%s{url}|%d{respcode}|src=%s{cip} 
  suser=%s{user} request=%s{url} act=%s{action} cs1=%s{threatname} 
  cs1Label=ThreatName in=%d{reqsize} out=%d{respsize}

Key Format String Specifiers:

• %s{user}: The authenticated username or email address.
• %s{cip} / %s{sip}: Client source IP address / Server destination IP address.
• %s{url} / %s{host}: Full HTTP request URL / Destination hostname.
• %s{action}: Policy action applied (Block, Allow, Quarantine, Caution).
• %s{threatname}: Name of detected malware, virus, or threat family.
• %d{respsize} / %d{reqsize}: Response payload bytes / Request payload bytes.

---

6. Sizing, Rate Limiting & High Availability

+-----------------------------------------------------------------------------+
|                   NSS HIGH AVAILABILITY & LOAD RESILIENCY                   |
+-----------------------------------------------------------------------------+

               +-----------------------------------------------+
               |             ZSCALER NANOLOG CLUSTER           |
               +-----------------------+-----------------------+
                                      / \
                   Outbound TLS 443  /   \  Outbound TLS 443
                                    /     \
                                   v       v
                 +-------------------+   +-------------------+
                 |   NSS VM 1 (HQ)   |   |   NSS VM 2 (DR)   |
                 |   (Active Feed)   |   |   (Active Feed)   |
                 +---------+---------+   +---------+---------+
                            \                     /
                             \                   /
                              v                 v
                 +-------------------------------------------+
                 |     ENTERPRISE SIEM LOAD BALANCER         |
                 +-------------------------------------------+

Sizing & Performance Guidelines:

• Standard NSS Instance: 2 Virtual CPUs (vCPUs), 8 GB RAM, 500 GB Disk --> Supports up to 10,000 to 15,000 Events Per Second (EPS).
• High-Capacity NSS Instance: 4–8 vCPUs, 16–32 GB RAM --> Supports up to 30,000+ EPS.
• Local Buffer Queue: The NSS VM maintains an in-memory buffer ring. If the SIEM collector experiences temporary network downtime, NSS queues logs locally for up to several hours (depending on transaction volume) without losing events.
• High Availability (HA): Deploy redundant NSS appliances in distinct physical clusters. In the ZIA admin portal, duplicate feed configurations can be bound to separate NSS nodes or mapped to a load-balanced VIP.

---

7. Exam Tips & High-Yield Distinctions

[!IMPORTANT] Inbound vs. Outbound Ports for NSS: A common exam question tests NSS firewall requirements. An NSS Virtual Appliance requires NO inbound ports open on the perimeter firewall. The NSS VM initiates an outbound-only TLS connection on TCP port 443 to the Zscaler Nanolog cloud cluster.

[!TIP] Cloud NSS vs. NSS VM Selection: If a scenario specifies that an enterprise operates a 100% cloud-native architecture using Microsoft Sentinel or Splunk Cloud and wants to eliminate all on-premise virtual machine management, the correct architectural answer is Cloud NSS.