Options
All
  • Public
  • Public/Protected
  • All
Menu

Class ExpressionTokenizer

Implement tokenizer to perform lexical analysis for expressions.

Hierarchy

Implements

Index

Constructors

constructor

Properties

Protected _lastTokenType

_lastTokenType: TokenType = TokenType.Unknown

Protected _nextToken

_nextToken: Token

Protected _scanner

_scanner: IScanner

commentState

commentState: ICommentState

decodeStrings

decodeStrings: boolean

mergeWhitespaces

mergeWhitespaces: boolean

numberState

numberState: INumberState

quoteState

quoteState: IQuoteState

skipComments

skipComments: boolean

skipEof

skipEof: boolean

skipUnknown

skipUnknown: boolean

skipWhitespaces

skipWhitespaces: boolean

symbolState

symbolState: ISymbolState

unifyNumbers

unifyNumbers: boolean

whitespaceState

whitespaceState: IWhitespaceState

wordState

wordState: IWordState

Accessors

scanner

Methods

clearCharacterStates

  • clearCharacterStates(): void

getCharacterState

hasNextToken

  • hasNextToken(): boolean

nextToken

Protected readNextToken

setCharacterState

  • setCharacterState(fromSymbol: number, toSymbol: number, state: ITokenizerState): void

tokenizeBuffer

  • tokenizeBuffer(buffer: string): Token[]

tokenizeBufferToStrings

  • tokenizeBufferToStrings(buffer: string): string[]

tokenizeStream

tokenizeStreamToStrings

  • tokenizeStreamToStrings(scanner: IScanner): string[]

Generated using TypeDoc