ETL file format: Buffers

Microsoft has not published the layout of an ETL file. What follows comes from the structures Microsoft did publish (in the Windows SDK and in the NTWMI.H header that shipped with two editions of the Windows 10 Driver Kit) and from Geoff Chappell’s study of them, checked against Airbus CERT’s etl-parser, an open-source reader (etl/etl.py, etl/wmi.py), which reads the file the same way. Field names are Microsoft’s. Where a meaning has not been established, this page says so. All numbers are little-endian.

File layout

A trace session collects events in memory buffers of one fixed size, set when the session starts. When a buffer is full, or is flushed, it is written to the file whole, header and all. An ETL file is therefore nothing but a run of buffers, one after another:

0Buffer 0: a WMI_BUFFER_HEADER, then events; its first event holds the log file headerBufferSize
BufferSizeBuffer 1: a WMI_BUFFER_HEADER, then eventsBufferSize
…more buffers to the end of the fileBufferSize each

An ETL file is a sequence of equal-size buffers.

Inside a buffer, events follow the 0x48-byte header one after another. Each starts with one of the event headers, whose first bytes give its size; events are aligned to 8 bytes. SavedOffset says how many bytes of the buffer are in use; the rest is padding.

WMI_BUFFER_HEADER

0x48 bytes on both 32-bit and 64-bit Windows. The layout below is the one used from Windows 7 (version 6.1) on; older versions differ, as noted.

Offset	Size	Type	Name	Description
`0x00`	4	uint32	`BufferSize`	The size of the buffer in bytes. In a file, the distance from this header to the next one.
`0x04`	4	uint32	`SavedOffset`	Bytes of the buffer that hold data, this header included.
`0x08`	4	uint32	`CurrentOffset`	Where the next event would have gone while the buffer was in use. Its meaning in a file is not established.
`0x0C`	4	int32	`ReferenceCount`	Used while the buffer is in memory.
`0x10`	8	int64	`TimeStamp`	A time for the buffer (from version 6.1).
`0x18`	8	int64	`SequenceNumber`	The buffer’s sequence number.
`0x20`	8	bit fields	`ClockType`, `Frequency`	From Windows 8 (6.2): the low 3 bits are the clock type (see clock types) and the other 61 bits the clock frequency. Earlier versions use these 8 bytes as list links or padding.
`0x28`	4	struct	`ClientContext`	An `ETW_BUFFER_CONTEXT`: see below.
`0x2C`	4	uint32	`State`	The buffer’s state while in memory (`ETW_BUFFER_STATE`).
`0x30`	4	uint32	`Offset`	Named only; not established.
`0x34`	2	uint16	`BufferFlag`	See buffer flags. In versions before Windows Server 2003 (5.2) these 4 bytes were a count of events lost.
`0x36`	2	uint16	`BufferType`	See buffer types.
`0x38`	16	struct	`ReferenceTime`	An `ETW_REF_CLOCK` from 6.1: a reference time for the buffer. In 5.0 to 5.2 a GUID was kept here.

Buffer context

ETW_BUFFER_CONTEXT, documented in the Windows SDK, says which processor and which session the buffer belongs to.

Offset	Size	Type	Name	Description
`0`	1	uint8	`ProcessorNumber`	The processor whose events are in the buffer.
`1`	1	uint8	`Alignment`	The alignment of events in the buffer.
`0`	2	uint16	`ProcessorIndex`	The same two bytes read as one 16-bit processor index, in later versions of the structure. The buffer flag `ETW_BUFFER_FLAG_PROC_INDEX` appears to mark this use, but that link is not documented.
`2`	2	uint16	`LoggerId`	The trace session’s number.

Buffer flags

Values of BufferFlag, from NTWMI.H. The header gives names only; the meanings here follow the names.

Value	Name	Meaning
`0x0000`	`ETW_BUFFER_FLAG_NORMAL`	An ordinary buffer.
`0x0001`	`ETW_BUFFER_FLAG_FLUSH_MARKER`	Marks a flush.
`0x0002`	`ETW_BUFFER_FLAG_EVENTS_LOST`	Events were lost.
`0x0004`	`ETW_BUFFER_FLAG_BUFFER_LOST`	A buffer was lost.
`0x0008`	`ETW_BUFFER_FLAG_RTBACKUP_CORRUPT`	A real-time backup buffer is corrupt.
`0x0010`	`ETW_BUFFER_FLAG_RTBACKUP`	A real-time backup buffer.
`0x0020`	`ETW_BUFFER_FLAG_PROC_INDEX`	The context holds a processor index.
`0x0040`	`ETW_BUFFER_FLAG_COMPRESSED`	The buffer’s contents are compressed. The compression method is not published.

Buffer types

Values of BufferType, from NTWMI.H; names only.

Value	Name	Meaning
0	`ETW_BUFFER_TYPE_GENERIC`	Ordinary events.
1	`ETW_BUFFER_TYPE_RUNDOWN`	Rundown events (the state of the system at the start or end of a trace).
2	`ETW_BUFFER_TYPE_CTX_SWAP`	Context-switch events.
3	`ETW_BUFFER_TYPE_REFTIME`	Reference time.
4	`ETW_BUFFER_TYPE_HEADER`	The header buffer.
5	`ETW_BUFFER_TYPE_BATCHED`	Batched events.
6	`ETW_BUFFER_TYPE_EMPTY_MARKER`	An empty marker.
7	`ETW_BUFFER_TYPE_DBG_INFO`	Debug information.
8	`ETW_BUFFER_TYPE_MAXIMUM`	One more than the highest type; not a real type.

.ETL File Format – Buffers

File layout

WMI_BUFFER_HEADER

Buffer context

Buffer flags

Buffer types