Class ExpressionTokenizer

Implement tokenizer to perform lexical analysis for expressions.

Hierarchy

AbstractTokenizer
- ExpressionTokenizer

Implements

ITokenizer

Index

Constructors

constructor

Properties

Accessors

scanner

Methods

Constructors

constructor

new ExpressionTokenizer(): ExpressionTokenizer

Overrides AbstractTokenizer.constructor
- Defined in calculator/tokenizers/ExpressionTokenizer.ts:13
Constructs an instance of this class.

Returns ExpressionTokenizer

Properties

Protected _lastTokenType

_lastTokenType: TokenType = TokenType.Unknown

Protected _nextToken

_nextToken: Token

Protected _scanner

_scanner: IScanner

commentState

commentState: ICommentState

decodeStrings

decodeStrings: boolean

mergeWhitespaces

mergeWhitespaces: boolean

numberState

numberState: INumberState

quoteState

quoteState: IQuoteState

skipComments

skipComments: boolean

skipEof

skipEof: boolean

skipUnknown

skipUnknown: boolean

skipWhitespaces

skipWhitespaces: boolean

symbolState

symbolState: ISymbolState

unifyNumbers

unifyNumbers: boolean

whitespaceState

whitespaceState: IWhitespaceState

wordState

wordState: IWordState

Accessors

scanner

get scanner(): IScanner
set scanner(value: IScanner): void

Inherited from AbstractTokenizer.scanner
- Defined in tokenizers/AbstractTokenizer.ts:58
Returns IScanner
Inherited from AbstractTokenizer.scanner
- Defined in tokenizers/AbstractTokenizer.ts:62
Parameters
- value: IScanner
Returns void

Methods

clearCharacterStates

clearCharacterStates(): void

Inherited from AbstractTokenizer.clearCharacterStates
- Defined in tokenizers/AbstractTokenizer.ts:54
Returns void

getCharacterState

getCharacterState(symbol: number): ITokenizerState

Inherited from AbstractTokenizer.getCharacterState
- Defined in tokenizers/AbstractTokenizer.ts:46
Parameters
- symbol: number
Returns ITokenizerState

hasNextToken

hasNextToken(): boolean

Implementation of ITokenizer.hasNextToken

Inherited from AbstractTokenizer.hasNextToken
- Defined in tokenizers/AbstractTokenizer.ts:68
Returns boolean

nextToken

nextToken(): Token

Implementation of ITokenizer.nextToken

Inherited from AbstractTokenizer.nextToken
- Defined in tokenizers/AbstractTokenizer.ts:73
Returns Token

Protected readNextToken

readNextToken(): Token

Inherited from AbstractTokenizer.readNextToken
- Defined in tokenizers/AbstractTokenizer.ts:79
Returns Token

setCharacterState

setCharacterState(fromSymbol: number, toSymbol: number, state: ITokenizerState): void

Inherited from AbstractTokenizer.setCharacterState
- Defined in tokenizers/AbstractTokenizer.ts:50
Parameters
- fromSymbol: number
- toSymbol: number
- state: ITokenizerState
Returns void

tokenizeBuffer

tokenizeBuffer(buffer: string): Token[]

Implementation of ITokenizer.tokenizeBuffer

Inherited from AbstractTokenizer.tokenizeBuffer
- Defined in tokenizers/AbstractTokenizer.ts:168
Parameters
- buffer: string
Returns Token[]

tokenizeBufferToStrings

tokenizeBufferToStrings(buffer: string): string[]

Implementation of ITokenizer.tokenizeBufferToStrings

Inherited from AbstractTokenizer.tokenizeBufferToStrings
- Defined in tokenizers/AbstractTokenizer.ts:182
Parameters
- buffer: string
Returns string[]

tokenizeStream

tokenizeStream(scanner: IScanner): Token[]

Implementation of ITokenizer.tokenizeStream

Inherited from AbstractTokenizer.tokenizeStream
- Defined in tokenizers/AbstractTokenizer.ts:159
Parameters
- scanner: IScanner
Returns Token[]

tokenizeStreamToStrings

tokenizeStreamToStrings(scanner: IScanner): string[]

Implementation of ITokenizer.tokenizeStreamToStrings

Inherited from AbstractTokenizer.tokenizeStreamToStrings
- Defined in tokenizers/AbstractTokenizer.ts:173
Parameters
- scanner: IScanner
Returns string[]

Generated using TypeDoc