15.1 Azure Monitor Data Collection Rules & Network Security Event Monitoring
Key Takeaways
- The Azure Monitor Agent collects nothing until a data collection rule is associated with the machine through a data collection rule association.
- One machine can have multiple data collection rules, and their data sources are additive rather than mutually exclusive.
- Windows security events are filtered with XPath queries and Linux syslog with facility and severity selections, which is the primary cost and noise control.
- Ingestion-time transformations written in KQL can drop rows, redact columns, and route data before it is billed and stored.
- VNet flow logs and Traffic Analytics supply network security telemetry that data collection rules do not gather from inside the guest.
Why This Object Exists
The retired Log Analytics agent was configured at the workspace level: every machine reporting to a workspace collected the same things. Azure Monitor Agent (AMA) inverts that with data collection rules (DCRs) — reusable objects that define what to collect and where to send it, bound to specific machines through data collection rule associations (DCRAs).
The consequence you must internalize: installing the agent collects nothing. A machine with AMA installed and no DCR association produces no data at all. "The agent is installed but the workspace is empty" is almost always a missing association.
Anatomy of a Data Collection Rule
| Element | Purpose |
|---|---|
| Data sources | What to collect: Windows event logs, Linux syslog, performance counters, IIS logs, custom text or JSON logs, Prometheus metrics |
| Streams | The named schema each data source produces, such as Microsoft-SecurityEvent, Microsoft-Syslog, Microsoft-Perf |
| Destinations | Where it goes: one or more Log Analytics workspaces, Azure Monitor Metrics, or a storage/event hub destination for supported streams |
| Data flows | Which stream goes to which destination, and which transformation applies |
Transformation (transformKql) | A KQL statement applied at ingestion, before billing and storage |
A data collection endpoint (DCE) is additionally required for scenarios such as custom logs, ingestion over Private Link, and some transformation configurations.
Associations are additive. Give a machine three DCRs — one for security events, one for performance counters, one for a custom application log — and it collects all three. This is how you build a shared security baseline DCR applied to every server, plus narrow DCRs for particular workloads, without duplicating configuration.
Filtering: Where Cost and Signal Are Decided
Windows events with XPath
Windows event collection is expressed as XPath queries, which lets you take exactly the event IDs you want:
Security!*[System[(EventID=4624 or EventID=4625 or EventID=4672)]]
System!*[System[(Level=1 or Level=2 or Level=3)]]
The Sentinel Windows Security Events via AMA connector wraps this with named tiers — All Events, Common, Minimal, or Custom — where Common and Minimal are curated event-ID sets that Microsoft maintains for security value per gigabyte. Choosing "All Events" on a domain controller is the classic way to burn an ingestion budget in a week.
Linux syslog by facility and severity
Linux collection selects facilities (auth, authpriv, cron, daemon, kern, syslog, user, and the local facilities) and a minimum severity per facility. For security monitoring, auth and authpriv at info level and above carry most of the value; debug on every facility carries almost none at high volume.
Ingestion-time transformations
A transformation is a KQL fragment operating on the incoming stream, applied before the data is stored and billed:
source
| where EventID in (4624, 4625, 4672, 4720, 4726, 4732)
| extend Account = tolower(Account)
| project-away UnusedColumn
Use them to drop noisy rows, redact or hash sensitive columns before storage (a privacy control as much as a cost control), normalize values, and route different subsets to different destinations. Because they run at ingestion, a transformation that drops data is irreversible — validate against sample data first.
Collecting Network Security Telemetry
DCRs collect from inside the guest. Network security events largely come from elsewhere, and the exam expects you to combine them:
| Source | How it is collected | What it shows |
|---|---|---|
| VNet flow logs (superseding NSG flow logs) | Network Watcher writes to a storage account; Traffic Analytics processes into Log Analytics | Allowed and denied 5-tuple flows, with threat intelligence enrichment |
| Azure Firewall logs | Diagnostic settings to a workspace | Application and network rule hits, IDPS alerts, threat intel matches |
| Application Gateway / WAF logs | Diagnostic settings | Blocked requests, matched rules, request detail |
| NVA and firewall syslog / CEF | AMA on a Linux forwarder with a syslog/CEF DCR | Third-party appliance events |
| Guest network events | AMA with a Windows event or syslog DCR | Host-side connection and authentication events |
| Performance counters | AMA performance data source in a DCR | Network interface throughput, connection counts for anomaly baselining |
The CEF pattern is the one to remember for third-party devices: appliances forward syslog to a Linux forwarder VM running AMA, and a DCR on that VM collects and parses the CEF stream into the workspace, where Sentinel analytics rules consume it.
Operational Checklist
- Create a baseline security DCR (Windows Security Events at the Common tier, Linux auth/authpriv) and associate it with every server through a policy-driven assignment so new machines inherit it.
- Add workload-specific DCRs rather than expanding the baseline.
- Apply transformations to strip high-volume, low-value rows and to redact personal data before storage.
- Enable diagnostic settings on network resources — they are not part of DCRs and are frequently forgotten.
- Monitor the pipeline itself: agent health, DCR association coverage, and ingestion volume per table, with an alert for a sudden drop that would indicate collection has silently stopped.
The Azure Monitor Agent has been installed on 50 virtual machines through an Azure Policy assignment, but no security events appear in the Log Analytics workspace. What is the most likely cause?
A security team must collect only Windows sign-in and privilege-assignment events from domain controllers to control ingestion cost. Which configuration achieves this?
An organization must remove a personally identifiable column from log records before those records are stored in the Log Analytics workspace. What should be configured?