Constructs a whitespace state with a default idea of what characters are, in fact, whitespace.
Ignore whitespace (such as blanks and tabs), and return the tokenizer's next token.
A textual string to be tokenized.
A tokenizer class that controls the process.
The next token from the top of the stream.
Establish the given characters as whitespace to ignore.
First character index of the interval.
Last character index of the interval.
true if this state should ignore characters in the given range.
Generated using TypeDoc
A whitespace state ignores whitespace (such as blanks and tabs), and returns the tokenizer's next token. By default, all characters from 0 to 32 are whitespace.