class ExprTokenizer
Defined at line 19 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.h
Public Methods
const std::string & input ()Defined at line 28 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.h
ExprLanguage language ()Defined at line 30 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.h
const Err & err ()The result of parsing. This will be multiline and will indicate the location of the problem.
Defined at line 33 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.h
size_t error_location ()When err is set, this will be the index into the input() string where the
error occurred.
Defined at line 37 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.h
const std::vector<ExprToken> & tokens ()When parsing is successful, this contains the extracted tokens.
Defined at line 40 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.h
std::vector<ExprToken> TakeTokens ()Defined at line 42 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.h
void ExprTokenizer (const std::string & input, ExprLanguage lang)Defined at line 65 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.cc
bool Tokenize ()Returns true on successful tokenizing. In this case, the tokens can be read from tokens(). On
failure, err() will contain the error message, and error_location() will contain the error
location.
Defined at line 68 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.cc
size_t GetNameTokenLength (ExprLanguage lang, std::string_view input)Returns the number of bytes that start at the given input that are valid name tokens.
If the input does not begin with a name token, this will return 0.
Defined at line 162 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.cc
bool IsNameToken (ExprLanguage lang, std::string_view input)Returns whether the input is a valid unescaped name token. This does no trimming of whitespace
and does not accept "$" escaping. An empty string is not a valid name token.
Defined at line 186 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.cc
std::string GetErrorContext (const std::string & input, size_t byte_offset)Returns two context lines for an error message. It will quote a relevant portion of the input
showing the byte offset, and add a "^" on the next line to indicate where the error is.
Defined at line 191 of file ../../src/developer/debug/zxdb/expr/expr_tokenizer.cc