Log Formats

Log Formats#

Reference of common log formats an operator parses, ingests, or generates: syslog, JSON, NCSA Combined, journald, OpenTelemetry, Windows Event Log, and the structured-logging conventions in common languages.

For protocols context, see Network Protocols. For SIEM detection, see Sigma.

Syslog#

Standard	Notes
RFC 3164 (BSD)	legacy free-form; one line per message. `<PRI>TIMESTAMP HOSTNAME TAG: MSG`.
RFC 5424	modern; structured-data sections. `<PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID STRUCTURED-DATA MSG`.
RFC 5425 (TLS)	syslog over TLS; port 6514.
RFC 5426 (UDP)	port 514 default.
RFC 6587 (TCP)	stream framing.
RELP	Reliable Event Logging Protocol; rsyslog extension.

PRI = facility × 8 + severity:

Severity	Numeric
Emergency	0
Alert	1
Critical	2
Error	3
Warning	4
Notice	5
Informational	6
Debug	7

Facility	Numeric
kern	0
user	1
mail	2
daemon	3
auth	4
syslog	5
lpr	6
news	7
uucp	8
cron	9
authpriv	10
ftp	11
local0..7	16..23

Common log lines#

Source	Sample
NCSA Common Log	`192.168.1.1 - - [04/May/2026:14:30:01 +0000] "GET /index.html HTTP/1.1" 200 1234`
NCSA Combined Log	adds `"Referer" "User-Agent"`.
nginx access (def.)	`$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"`
nginx error	`2026/05/04 14:30:01 [error] 1234#0: *1 ...`
Apache access	similar to NCSA.
HAProxy	`Mar 25 08:46:24 lb1 haproxy[12345]: 192.0.2.1:32768 [25/Mar/2026:08:46:24.123] frontend backend/server1 0/0/0/123/124 200 1024 - - --VN 5/3/0/0/0 0/0 "GET /api/users HTTP/1.1"`
Squid	`1648202784.123 124 192.168.1.1 TCP_MISS/200 1024 GET http://example.com - HIER_DIRECT/93.184.216.34 text/html`
Postfix	`mail.example.com postfix/qmgr[1234]: A1B2C3D4: from= <user@example.com>, size=12345, nrcpt=1 (queue active)`

JSON / structured#

Convention	Notes
JSON Lines (NDJSON)	one JSON object per line; the modern default.
ECS (Elastic	opinionated field schema; used by Beats / Logstash /
Common Schema)	OpenSearch.
OTel logs	OpenTelemetry log data model; SDK + Collector.
GCP Cloud Logging	structured payload; severity, labels, traces.
AWS CloudWatch Logs	lines or structured (JSON) per stream.
Azure Monitor Log	KQL-queryable structured tables.
Windows ETW	binary; high-throughput; Event Tracing for Windows.
Honeycomb / Datadog	/ Splunk / Sumo vendor JSON variants.

Recommended JSON log fields:

Field	Notes
`timestamp`	RFC 3339; UTC with timezone offset.
`level`	`DEBUG` / `INFO` / `WARN` / `ERROR` / `FATAL`.
`message`	human-readable description.
`service`	emitting service name.
`version`	service version / build.
`env`	environment (prod / staging / dev).
`host`	host or pod name.
`request_id`	/ `trace_id` / `span_id` (W3C TraceContext).
`user_id` / `account_id`	caller identifier (PII-class, mind retention).
`error.*`	`error.type` + `error.stack`; standard exception.
`http.*`	`http.method` + `http.url` + `http.status_code` (OTel HTTP semantic conventions).
`net.*`	`net.peer.ip` + `net.peer.port` (OTel network).

journald (systemd)#

Field	Notes
__REALTIME_TIMESTAMP	microseconds since epoch.
__MONOTONIC_TIMESTAMP	host monotonic clock.
__BOOT_ID	unique-per-boot UUID.
PRIORITY	syslog priority (above).
SYSLOG_FACILITY	per-syslog facility.
_PID, _UID, _GID	caller process / user / group.
_COMM, _EXE	process name / executable path.
_CMDLINE	full command-line.
_SYSTEMD_UNIT	`foo.service` etc.
_HOSTNAME	host.
MESSAGE	free-form text.
MESSAGE_ID	RFC 5424 STRUCTURED-DATA-style message-class UUID.

Query: journalctl -u nginx -p err --since "1h ago" -o json.

Windows Event Log#

Channel	Notes
Application	per-app events.
Security	audit; cred + login + privilege use.
System	kernel + driver + OS.
Setup	install events.
Forwarded Events	WEC subscriber-collected.
Microsoft-Windows-*	hundreds of subscribed channels (PowerShell, Sysmon, Defender, etc.).
ETL files	Event Tracing binary; .etl.
EVTX	XML serialised event log; .evtx.
Sysmon	configurable detailed process / file / network / registry events.

Important Event IDs (Security):

ID	Meaning
4624	successful logon.
4625	failed logon.
4634	logoff.
4647	user-initiated logoff.
4648	explicit-cred logon (RunAs).
4663	object access (audit).
4672	special privileges assigned.
4688	process creation.
4697	service install.
4720	user account created.
4724	password reset.
4732	added to local group.
4756	added to universal group.
4768	Kerberos AS-REQ (TGT).
4769	Kerberos TGS-REQ.
4776	NTLM credential validation.

Sysmon Event IDs:

ID	Meaning
1	Process create.
3	Network connection.
5	Process terminated.
7	Image loaded.
10	Process accessed (LSASS read alarm).
11	File created.
12-14	Registry events.
15	FileCreateStreamHash.
22	DNS query.
23	File delete.

Linux audit (auditd)#

Concept	Notes
`/etc/audit/audit.rules`	rule definitions.
ausearch / aureport
Syscall auditing	per-syscall match (open, execve, …).
Watch rules	file / dir-based audits.
SELinux / AppArmor	separately log AVC denials.
auditd rules per benchmark	CIS / DISA STIG benchmarks ship rule-sets.

Tooling#

Tool	Notes
rsyslog	Linux syslog daemon; default on most distros.
syslog-ng	alternative; more programmable.
journald	systemd-bundled; binary log store.
Vector (Datadog)	fast Rust log shipper; aggregator.
Fluent Bit / Fluent	d CNCF; container-friendly.
Logstash	Elastic; JVM-heavy.
Filebeat / Winlogbe	at Elastic shippers.
Promtail (Loki)	Grafana log shipper.
OpenTelemetry Collector	Collector for logs / metrics / traces.
auditd-monitor	SOC integrations.
Wazuh	OSS XDR; ingests + correlates.
Kafka / Pulsar / Redpanda	The durable transport bus for logs at scale.

Operator notes#

Always log in UTC (or RFC 3339 with offset); the parsing pipeline normalises.
Structure beats text, emit JSON Lines; SIEM parsing, field extraction, and search are an order of magnitude cheaper.
PII / PHI, mind retention regulations; redact at the edge if possible (Vector / Fluent Bit can do this in flight).
Field naming, pick ECS or OTel semantic conventions; do not invent custom names (user_id vs userid vs uid fragments queries forever).
Correlate with trace IDs (W3C TraceContext) so logs and traces line up in OTel-aware backends.
Retention vs cost, hot tier (≤90 days, full search), warm (≤1y, restricted), cold (compliance-only). Compliance regimes (HIPAA 6y, SOX 7y, GDPR shorter) drive minimums.