Name
token — A unit of information
Synopsis
token ::=
(text | phrase
db._phrase | replaceable
| Graphic inlines | Indexing inlines | Linking inlines | Ubiquitous inlines)*
Description
A token
identifies a unit of information.
Usually, “tokens” are the result of some processing pass
that has performed lexical analysis and divided a data set into the
smallest units of information used for subsequent processing.
Exactly what constitutes a token varies by context.
Processing expectations
Formatted inline.
See Also
classname
, property
, symbol
, type
Examples
<article xmlns='http://docbook.org/ns/docbook'>
<title>Example token</title>
<para>In parsing, line ends are turned into the <token>CRLF</token>, all other
whitespace becomes <token>WHITESP</token>.
</para>
</article>