Formats and normalization

About 13 min read

This page covers the plumbing under the behavioral surfaces: the formats telemetry arrives in, the platform sources that generate it, and the normalization that makes it all queryable as one dataset. It is not a surface of adversary behavior. It is the layer that determines whether you can read the surfaces at all.

The four format families

What the raw records look like

A console shows you its parsed view of a record. Before you trust that view, read one raw record per source and find the fields you will rely on. The samples below are trimmed and fictional; the field names are real. Raw records carry what the parsed view hides, including passwords, tokens and session cookies in command lines, URLs and headers: redact secrets before you paste a raw record into a ticket, a chat or an AI assistant, and keep the unredacted original in the evidence store.

<EventData>
  <Data Name="UtcTime">2026-03-10 14:22:04.871</Data>
  <Data Name="ProcessGuid">{6a1f0c3e-2b7d-65f0-4c1a-000000001a00}</Data>
  <Data Name="ProcessId">8128</Data>
  <Data Name="Image">C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe</Data>
  <Data Name="CommandLine">powershell.exe -nop -w hidden -enc SQBFAFgAIAAoAE4AZQB3AC0ATwBiAGoAZQBjAHQA...</Data>
  <Data Name="User">CORP\jdoe</Data>
  <Data Name="LogonId">0x3e7a21</Data>
  <Data Name="IntegrityLevel">Medium</Data>
  <Data Name="ParentProcessGuid">{6a1f0c3e-2b7c-65f0-3b19-000000001a00}</Data>
  <Data Name="ParentImage">C:\Program Files\Microsoft Office\root\Office16\WINWORD.EXE</Data>
  <Data Name="ParentCommandLine">"WINWORD.EXE" /n "C:\Users\jdoe\Downloads\remittance.docm"</Data>
</EventData>

<134>1 2026-03-10T14:22:05.003Z fw01.corp.example filterlog 2211 - - pass out tcp 10.20.4.17:51544 -> 198.51.100.23:443
<134>Mar 10 14:22:05 fw01 filterlog[2211]: pass out tcp 10.20.4.17:51544 -> 198.51.100.23:443

CEF:0|ExampleVendor|SecureGateway|5.2|url-allowed|URL allowed|3|rt=1773152525003 suser=jdoe src=10.20.4.17 dst=198.51.100.23 request=https://cdn-relay.example/upd cat=Newly Registered Domain act=allowed

type=SYSCALL msg=audit(1773152530.412:8812): arch=c000003e syscall=59 success=yes exit=0 ppid=2140 pid=2187 auid=1003 uid=1003 comm="bash" exe="/usr/bin/bash" key="exec"
type=EXECVE msg=audit(1773152530.412:8812): argc=3 a0="bash" a1="-c" a2=69643B20756E616D65202D61

Where the telemetry comes from, per platform

Formats are only half the story. The behavioral surfaces in this chapter (process trees, command lines, file activity) depend on a telemetry source actually recording them, and the source differs by platform. Knowing which source feeds your console tells you what the data can and cannot show.

Why standardization matters

A single intrusion almost always produces telemetry across more than one format. Imagine an attacker phishes a user on a Windows laptop, the laptop reaches out through the corporate firewall to an external domain, the secure web gateway logs the same request, and the cloud identity provider records the resulting authentication. That is four formats describing one event:

Four formats. Four different field names for the same concept. Four different timestamp conventions. Without standardization, the analyst has to write four parallel parsers in their head just to ask the basic question: did the same user account that triggered the laptop alert show up in the cloud identity log within the next ten minutes?

Normalize to UTC and write times in RFC 3339 form with an explicit offset (2026-03-10T14:22:05Z), so nobody downstream has to guess the zone. Record whether each timestamp is event time or ingest time, and note any known clock skew between sources: a ten-minute correlation window is meaningless across a five-minute skew.

That question is trivial when every event uses the same field names. It is hard when each format has its own vocabulary. Standardization is the engineering investment that makes the trivial version possible at scale.

One shape: schema normalization

Normalization Normalization Converting logs and events from different sources into a common field schema, such as OCSF, ECS, Splunk CIM, Microsoft Sentinel ASIM or Google SecOps UDM, so a username, IP, or timestamp means the same thing regardless of which tool recorded it. SIEM correlation rules and cross-source queries depend on it; fields that were never normalized are where searches silently miss. maps every source’s vendor-specific fields into a common schema, so a malicious PowerShell PowerShell Microsoft's shell and scripting language: Windows PowerShell 5.1 on .NET Framework, cross-platform PowerShell 7 on .NET. The most abused interpreter on Windows (ATT&CK T1059.001). Triage tells: -EncodedCommand or -enc (Base64 of UTF-16LE text), -NoProfile, -WindowStyle Hidden, IEX with a download cradle, and an Office or browser parent. Script Block Logging (Event ID 4104) records de-obfuscated script content. execution reported by the EDR and the related connection logged by the firewall land in the same structure with the same field names. These are the event schemas you will meet. Your console almost certainly speaks one of these; learn its field names for user, host, process and source IP first.

Two standards often appear next to these and are not event schemas: STIX 2.1 Structured Threat Information Expression (STIX) OASIS's JSON language for threat intelligence (STIX 2.1): indicators, attack patterns, threat actors and their relationships, usually exchanged over TAXII. Respect the sharing marking (TLP 2.0) on what you receive. carries threat intelligence (what an IOC means); OpenC2 OpenC2 An OASIS standard language for machine-to-machine commands to cyber-defense tools (block this address, isolate this host), meant to let orchestration platforms drive devices from different vendors through one vocabulary. Adoption is still limited. expresses response actions (what to do about it). Both matter to a SOC, but neither normalizes event telemetry.

Whatever schema you adopt, normalization maps diverse vendor-specific fields into four buckets. They line up with the three buckets in Reading any alert: in schema terms, its Technical detail bucket splits into affected entities and primary observables.

Normalization also requires flattening nested structures where useful and harmonizing identifiers so automated cross-tool correlation works. Within ASSURED, prioritize normalization of detection metadata, affected entities, and primary observables. Those are the fields downstream phases depend on.

Check yourself

The Sysmon record, the firewall lines and the CEF record above describe the same minute on the same laptop. Before reading on, write down the fields you would join on and the first thing you would check about the times.

For practice on a full record, Lab 1 hands you a scheduled-task event and asks what the alert summary dropped and whether a second source agrees.