Every browser reads every HTML file with one algorithm, chapter 13.2 of the HTML Living Standard. It never rejects a document: anything that is not valid is a parse error with a defined recovery. There are two stages, the tokenizer and tree construction.
Overview
- The bytes are decoded to characters and normalised: CR and CRLF become LF.
- The tokenizer turns the characters into tokens: DOCTYPE, start tag, end tag, comment, processing instruction, character, end of file.
- Tree construction takes each token and, depending on the current insertion mode, adds nodes to the document, using a stack of open elements and a list of active formatting elements.
The tree can change what the tokenizer does next: after a <script> start
tag the tokenizer switches to its script data state, after
<title> to RCDATA, and so on.
Tokenizer states (13.2.5)
| # | State | What it does |
|---|---|---|
| 1 | Data | Ordinary text; & starts a reference, < a tag |
| 2 | RCDATA | Text inside title and textarea: references, but no tags |
| 3 | RAWTEXT | Text inside style, noscript, noframes, iframe and noembed |
| 4 | Script data | Text inside script, with its special comment rules |
| 5 | PLAINTEXT | Everything to the end of the file (plaintext element) |
| 6 | Tag open | After <: a tag, a markup declaration, or text |
| 7 | End tag open | After </ |
| 8 | Tag name | Reading a tag name, lower-casing it |
| 9–11 | RCDATA less-than sign, end tag open, end tag name | Looking for the matching end tag in RCDATA |
| 12–14 | RAWTEXT less-than sign, end tag open, end tag name | Looking for the matching end tag in RAWTEXT |
| 15–17 | Script data less-than sign, end tag open, end tag name | Looking for </script |
| 18–22 | Script data escape start, escape start dash, escaped, escaped dash, escaped dash dash | After <!-- inside a script: the text is "escaped" until --> |
| 23–25 | Script data escaped less-than sign, escaped end tag open, escaped end tag name | End tag check while escaped |
| 26–31 | Script data double escape start, double escaped, dash, dash dash, less-than sign, double escape end | A nested <script> inside the comment: </script> no longer ends the element until it unwinds |
| 32 | Before attribute name | Skips white space; / and > end the tag |
| 33 | Attribute name | Reading an attribute name, lower-cased |
| 34 | After attribute name | White space, then = means a value follows |
| 35 | Before attribute value | Picks quoted or unquoted |
| 36 | Attribute value (double-quoted) | To the closing " |
| 37 | Attribute value (single-quoted) | To the closing ' |
| 38 | Attribute value (unquoted) | To white space or > |
| 39 | After attribute value (quoted) | Expects white space, / or > |
| 40 | Self-closing start tag | After / in a tag |
| 41 | Bogus comment | Text to the first > becomes a comment |
| 42 | Markup declaration open | After <!: --, DOCTYPE or [CDATA[ |
| 43–44 | Comment start, comment start dash | The start of a comment; > ends it at once |
| 45 | Comment | Comment text |
| 46–49 | Comment less-than sign, bang, bang dash, bang dash dash | Looking for <!-- nested in a comment |
| 50–52 | Comment end dash, comment end, comment end bang | Looking for --> or --!> |
| 53–56 | DOCTYPE, before DOCTYPE name, DOCTYPE name, after DOCTYPE name | Reading <!DOCTYPE name |
| 57–62 | After DOCTYPE public keyword, before public identifier, public identifier double- and single-quoted, after public identifier, between public and system identifiers | The PUBLIC form |
| 63–67 | After DOCTYPE system keyword, before system identifier, system identifier double- and single-quoted, after system identifier | The SYSTEM form |
| 68 | Bogus DOCTYPE | The rest of a malformed DOCTYPE |
| 69–71 | CDATA section, bracket, end | Inside <![CDATA[ … ]]> in foreign content |
| 72–76 | Processing instruction open, target, after target, data, questionable | <?target data?> up to the first > |
| 77–78 | Character reference, named character reference | Reading an & reference |
| 79 | Ambiguous ampersand | & followed by letters that are not a name |
| 80–84 | Numeric character reference, hexadecimal start, decimal start, hexadecimal, decimal, numeric end | Reading &#…; and validating the code point |
Insertion modes (13.2.6.4)
Each token is handled according to the current insertion mode:
| Mode | Handles |
|---|---|
| initial | Before anything; the DOCTYPE decides quirks mode |
| before html | Creates the html element |
| before head | Creates head |
| in head | title, meta, link, script, style, base, template, noscript |
| in head noscript | Inside noscript in head, when scripting is off |
| after head | Creates body or frameset |
| in body | The bulk of the document |
| text | Inside raw text and RCDATA elements, until the end tag |
| in table | Inside table; content that does not belong is foster-parented |
| in table text | Collects text in a table to decide what to do with it |
| in caption | Inside caption |
| in column group | Inside colgroup |
| in table body | Inside tbody, thead, tfoot |
| in row | Inside tr |
| in cell | Inside td, th |
| in template | Inside template |
| after body | After </body> |
| in frameset | Inside frameset |
| after frameset | After </frameset> |
| after after body | After </html> |
| after after frameset | After </html> in a frameset document |
There is no separate mode for select any more: its content is read by
the in body rules, with a few special cases for option,
optgroup, hr and input.
Implied structure
If the file does not have them, the parser supplies html, head
and body, tbody inside tables, and colgroup.
Elements are closed for the author in the cases on the
optional tags list: a new
p closes an open one, a new li closes an open one, a heading
closes an open p, and so on. The list of elements that close a
p is: address, article, aside, blockquote, center, details, dialog, dir,
div, dl, fieldset, figcaption, figure, footer, form, h1 to h6, header, hgroup, hr,
listing, main, menu, nav, ol, p, plaintext, pre, search, section, summary, table, ul.
Active formatting and the adoption agency
Formatting elements (a, b, big, code,
em, font, i, nobr, s,
small, strike, strong, tt,
u) are kept on a list so that they can be reopened after a block closes
them: <p><b>one</p><p>two is bold in both
paragraphs. When formatting is closed in the wrong order, the
adoption agency algorithm moves and clones elements so the result is
still a tree: <b>1<p>2</b>3</p> becomes a
b with “1”, then a p holding a second b with “2”,
and “3” after it.
Foster parenting
Text or an element inside a table but not in a cell cannot stay there. The parser
moves it to just before the table in the tree:
<table>hello<tr>…</table> puts “hello” in front of the
table. Only table-structure tags, form, hidden inputs, script and style are
allowed inside a table directly.
Foreign content and scope
Inside svg and math names are matched case-insensitively
and adjusted to their real case (for example foreignObject,
viewBox), CDATA sections are allowed, and self-closing tags work. A few
HTML elements (b, div, p, img and
others) end the foreign content when they appear. foreignObject,
desc and title in SVG, and the MathML text elements, are
integration points: HTML rules apply inside them again.
Elements such as applet, caption, html,
table, td, th, marquee,
object and template stop the search when an end tag looks
for its element, and act as scope markers.
Scripting and quirks
With scripting on, the content of noscript is raw text; with it off, it
is parsed as markup. The viewer on this site reads a page as a browser with scripting
on would. Quirks mode comes from the DOCTYPE, as listed on the
versions page; the parser changes one thing
because of it, that a table does not close an open p in quirks
mode.