class Token

class Token

Defined at line 21 of file ../../tools/fidl/fidlc/src/token.h

A Token represents a typed view into a source buffer. That is, it

has a TokenKind, and it has a buffer representing the data

corresponding to the token. No processing is done on the data:

string or numeric literals aren't further parsed, identifiers

uniqued, and so on.

Public Methods

void Token (SourceSpan span, uint16_t leading_newlines, Kind kind, Subkind subkind)

Defined at line 59 of file ../../tools/fidl/fidlc/src/token.h

void Token ()

Defined at line 64 of file ../../tools/fidl/fidlc/src/token.h

const char * Name (KindAndSubkind kind_and_subkind)

Defined at line 66 of file ../../tools/fidl/fidlc/src/token.h

const SourceSpan & span ()

Defined at line 83 of file ../../tools/fidl/fidlc/src/token.h

std::string_view data ()

Defined at line 84 of file ../../tools/fidl/fidlc/src/token.h

const char * ptr ()

Defined at line 85 of file ../../tools/fidl/fidlc/src/token.h

uint16_t leading_newlines ()

Defined at line 86 of file ../../tools/fidl/fidlc/src/token.h

Kind kind ()

Defined at line 87 of file ../../tools/fidl/fidlc/src/token.h

Subkind subkind ()

Defined at line 88 of file ../../tools/fidl/fidlc/src/token.h

KindAndSubkind kind_and_subkind ()

Defined at line 89 of file ../../tools/fidl/fidlc/src/token.h

void set_leading_newlines (uint16_t leading_newlines)

Defined at line 91 of file ../../tools/fidl/fidlc/src/token.h

Enumerations

enum Kind : uint8_t
Name Value Comments
kNotAToken 0

Control and whitespace
kEndOfFile 1

Control and whitespace
kStartOfFile 2

Control and whitespace
kComment 3

Control and whitespace
kDocComment 4

Control and whitespace
kIdentifier 5

Identifiers and constants
kNumericLiteral 6

Identifiers and constants
kStringLiteral 7

Identifiers and constants
kLeftParen 8

Punctuation.
kRightParen 9

Punctuation.
kLeftSquare 10

Punctuation.
kRightSquare 11

Punctuation.
kLeftCurly 12

Punctuation.
kRightCurly 13

Punctuation.
kLeftAngle 14

Punctuation.
kRightAngle 15

Punctuation.
kAt 16

Punctuation.
kDot 17

Punctuation.
kComma 18

Punctuation.
kSemicolon 19

Punctuation.
kColon 20

Punctuation.
kQuestion 21

Punctuation.
kEqual 22

Punctuation.
kAmpersand 23

Punctuation.
kArrow 24

Punctuation.
kPipe 25

Punctuation.

Defined at line 23 of file ../../tools/fidl/fidlc/src/token.h

enum Subkind : uint8_t
Name Value Comments
kNone 0 --
kAs 1

Token subkinds
kAlias 2

Token subkinds
kLibrary 3

Token subkinds
kUsing 4

Token subkinds
kArray 5

Token subkinds
kRequest 6

Token subkinds
kString 7

Token subkinds
kStringArray 8

Token subkinds
kVector 9

Token subkinds
kAjar 10

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kBits 11

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kClosed 12

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kConst 13

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kEnum 14

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kOpen 15

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kProtocol 16

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kResource 17

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kResourceDefinition 18

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kService 19

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kStrict 20

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kStruct 21

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kTable 22

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kFlexible 23

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kType 24

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kUnion 25

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kOverlay 26

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kError 27

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kTrue 28

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kFalse 29

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kCompose 30

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kReserved 31

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.
kProperties 32

TOKEN_SUBKIND(Max, "MAX")
MAX is not a real keyword, but it effectively works like one. It can be used
as vector
<T
>:MAX or string:MAX. See Library::ResolveSizeBound in flat_ast.cc.

Each identifier token is assigned a subkind, which are defined below, or a sentinel value if

the token does not match one of the subkinds, to make it simple for the parser to either

consume a generic identifier or one that matches a specific subkind. This design makes it so

that there are no "reserved keywords" and any identifier can be defined by the user.

Defined at line 33 of file ../../tools/fidl/fidlc/src/token.h

Records