Tokenizer Table is one of Understand's built-in reports — run it on your own code from the GUI, or headlessly with und report.
Tokenizer Table
Languages: AnyTargets: Classes, Files, Functions
Display the lexer tokens generated for a file or entity in a table.
This report shows each token as generated by the lexer as a row in the table. The table columns are:
- Token
- Text
- Start Line
- Start Column
- End Line
- End Column
- The entity if any
- Reference kind if there is a reference
- Reference scope if there is a reference
- Reference entity if there is a reference
- Inactive
See also the Tokenizer ireport which displays the same information with xml.
