.TXT File Format

Introduction

EBCDIC is IBM’s family of 8-bit encodings, from the System/360 (1963 to 1964), and still the native text of z/OS and IBM i. It is not based on ASCII: the space is 40, the digits are F0 to F9, and the letters come in three runs. “Hi” is C8 89. There are hundreds of EBCDIC code pages. No file says which one it uses, so the explorer recognises EBCDIC from its bytes, guesses a code page, and shows the file in every page it knows.

Files are normally converted to ASCII when they leave the mainframe, so EBCDIC .txt files are rare on PCs, and no browser decodes EBCDIC. A file dropped on any page of this site opens in the explorer, which decodes it.

Structure

The file is characters and line endings, with no header. Records of fixed length with no line endings are common on mainframes, and such a file shows as one long line here, because nothing in it marks a record’s length.

Byte order mark

EBCDIC has no byte order mark, and nothing in the file names the code page.

Encoding

Each byte is one character. Letters are in three runs (A–I, J–R, S–Z), with a gap between each, and lower case at 81 to A9. Code page 037 is the usual choice for US English; 500 for international; 1047 for z/OS Unix.

Lines

A line is the bytes up to a line ending. Each character is one byte, so counts of characters and bytes are the same. Records of fixed length without line endings read as one long line.

Line endings

End of the file

The text simply stops at the end of the file. EBCDIC text has no end-of-file mark of its own, but trailing NUL bytes show as Zero padding, and a trailing 1A as an End-of-file mark, as in other text.

Problems

Telling it apart

There is no mark or signature. The explorer tries this format after UTF-8, UTF-16, UTF-32 and GB18030, and before the legacy code pages. It takes a file when, in the first 64 KB:

• the file is at least 16 bytes long;
• the space byte 40 makes up at least one byte in twenty;
• the bytes of the space, the EBCDIC letters and the digits together make up at least seven in ten;
• bytes in the ASCII letter ranges (41–5A, 61–7A) are at most one in five.

Code page 037 is assumed, unless the square brackets are at AD and BD rather than BA and BB, which point to 1047. National pages cannot be told apart from their bytes; the explorer shows the file in each of them (037, 500, 1047, 273, 277, 278, 280, 284, 285, 297 and 871) so that a reader can pick.

Variants

• UTF-8 text – one to four bytes a character, ASCII unchanged
• UTF-16LE text – two or four bytes a character, low byte first
• UTF-16BE text – two or four bytes a character, high byte first
• UTF-32LE text – four bytes a character, low byte first
• UTF-32BE text – four bytes a character, high byte first
• Legacy code page text – a pre-Unicode code page that the file does not name
• UTF-7 text – Unicode as pure ASCII, with Base64 runs
• ISO-2022 and HZ text – 7-bit Japanese, Korean and Chinese with escape sequences
• GB18030 text – China’s national encoding, one, two or four bytes a character
• EBCDIC text – IBM mainframe text, not based on ASCII

Links