The Enhanced RTMP specification from Veovera extends the FLV tag bodies to carry modern codecs, identified by FourCC instead of the old 4-bit IDs, as well as HDR metadata, multichannel audio and multiple tracks. The file header and tag header do not change.
Extended video header
The first byte of a video tag is read differently when its top bit is set.
| Bits | Name | Meaning |
|---|---|---|
| 7 | isExVideoHeader | 1 for this layout; 0 for the legacy layout. |
| 6 to 4 | videoFrameType | As the legacy frame type. |
| 3 to 0 | videoPacketType | See below. The legacy codec ID is not read. |
Then, in order: any ModEx blocks; for Multitrack, a byte with the 4-bit
multitrack type and the 4-bit packet type; a 4-byte FourCC, unless the track list carries
its own (in ManyTracksManyCodecs) or the packet is metadata or a command;
then the body. CodedFrames has an SI24 composition offset
first for AVC, HEVC and VVC, while CodedFramesX has none. In a multitrack
packet each track carries a UI8 track ID and, unless it is the only track, a
UI24 size.
| videoPacketType | Name |
|---|---|
| 0 | SequenceStart |
| 1 | CodedFrames |
| 2 | SequenceEnd |
| 3 | CodedFramesX |
| 4 | Metadata, an AMF object whose only defined key is colorInfo |
| 5 | MPEG2TSSequenceStart |
| 6 | Multitrack |
| 7 | ModEx |
| 8 to 15 | Reserved |
| VideoFourCc | Codec |
|---|---|
vp08 | VP8 |
vp09 | VP9 |
av01 | AV1 |
avc1 | AVC (H.264) |
hvc1 | HEVC (H.265) |
vvc1 | VVC (H.266) |
| VideoCommand | Name |
|---|---|
| 0 | StartSeek |
| 1 | EndSeek |
| 2 to 255 | Reserved |
Extended audio header
If the top four bits of the first byte, SoundFormat, are 9, the rate, size
and type bits are not read. The low four bits become AudioPacketType.
| AudioPacketType | Name |
|---|---|
| 0 | SequenceStart |
| 1 | CodedFrames |
| 2 | SequenceEnd |
| 4 | MultichannelConfig |
| 5 | Multitrack |
| 7 | ModEx |
| 3, 6 | Reserved |
| 14, 15 | Reserved |
Values 8 to 13 were not visible in the part of the specification read for this page.
| AudioFourCc | Codec |
|---|---|
mp4a | AAC |
ac-3 | AC-3 |
ec-3 | E-AC-3 |
Opus | Opus |
.mp3 | MP3 |
fLaC | FLAC |
ModEx
A ModEx block gives a modifier for the packet that follows it: a data size as
UI8 plus 1, the data, then a 4-bit ModEx type and the next packet type. A
size of 256 means a UI16 plus 1 follows.
| Mod type | Name | Meaning |
|---|---|---|
| 0 | TimestampOffsetNano | A UI24 nanosecond offset that applies only to the current message. |
Multitrack and multichannel
| AvMultitrackType | Name |
|---|---|
| 0 | OneTrack |
| 1 | ManyTracks |
| 2 | ManyTracksManyCodecs |
| 3 to 15 | Reserved |
| AudioChannelOrder | Name |
|---|---|
| 0 | Unspecified |
| 1 | Native |
| 2 | Custom |
| 3 to 15 | Reserved |
AudioChannelMask bits name the channels below; the channel index is the
bit number.
| Index | Mask | AudioChannel |
|---|---|---|
| 0 | 0x000001 | FrontLeft |
| 1 | 0x000002 | FrontRight |
| 2 | 0x000004 | FrontCenter |
| 3 | 0x000008 | LowFrequency1 |
| 4 | 0x000010 | BackLeft |
| 5 | 0x000020 | BackRight |
| 6 | 0x000040 | FrontLeftCenter |
| 7 | 0x000080 | FrontRightCenter |
| 8 | 0x000100 | BackCenter |
| 9 | 0x000200 | SideLeft |
| 10 | 0x000400 | SideRight |
| 11 | 0x000800 | TopCenter |
| 12 | 0x001000 | TopFrontLeft |
| 13 | 0x002000 | TopFrontCenter |
| 14 | 0x004000 | TopFrontRight |
| 15 | 0x008000 | TopBackLeft |
| 16 | 0x010000 | TopBackCenter |
| 17 | 0x020000 | TopBackRight |
| 18 | 0x040000 | LowFrequency2 |
| 19 | 0x080000 | TopSideLeft |
| 20 | 0x100000 | TopSideRight |
| 21 | 0x200000 | BottomFrontCenter |
| 22 | 0x400000 | BottomFrontLeft |
| 23 | 0x800000 | BottomFrontRight |
Index 0xFE means unused and 0xFF unknown.
Metadata
audiocodecid and videocodecid in onMetaData can
hold a FourCC. The maps audioTrackIdInfoMap and
videoTrackIdInfoMap describe the tracks of a multitrack stream.