Type representing acceptable formats to define token definitions for a
SimpleLexer
tokens: Record<keyof any, SimpleLexerTokenDefinition["pattern"]> | SimpleLexerTokenDefinition[]
Token definitions for this lexer. Can be specified as an array of token definition objects or
ignoreCase: boolean
Set this to true
to make the lexer patterns ignore casing.
This cannot be set on a per-token basis. If some tokens are case-sensitive and some aren't, do not enable ignore and handle this in the patterns themselves.
The "i"
flags on the RegExp
-based tokens have no effect on the
RegExp
pattern used by the lexer.
unicodeFlag: _RegExpUnicodeFlag
Sets the Unicode flag to be used in the combined RegExp
.
Valid values are: null
, "u"
, and `"v".
If this is left unspecified or undefined
, it uses the same Unicode flag
used in the first RegExp
-based token pattern specifed in tokens
property. If none of the token patterns are declared in RegExp
form,
the flag defaults to "v"
.