Options
All
  • Public
  • Public/Protected
  • All
Menu

Class CsvTokenizer

Implements a tokenizer class for CSV files.

Hierarchy

Implements

Index

Constructors

constructor

Properties

Protected _lastTokenType

_lastTokenType: TokenType = TokenType.Unknown

Protected _nextToken

_nextToken: Token

Protected _scanner

_scanner: IScanner

commentState

commentState: ICommentState

decodeStrings

decodeStrings: boolean

mergeWhitespaces

mergeWhitespaces: boolean

numberState

numberState: INumberState

quoteState

quoteState: IQuoteState

skipComments

skipComments: boolean

skipEof

skipEof: boolean

skipUnknown

skipUnknown: boolean

skipWhitespaces

skipWhitespaces: boolean

symbolState

symbolState: ISymbolState

unifyNumbers

unifyNumbers: boolean

whitespaceState

whitespaceState: IWhitespaceState

wordState

wordState: IWordState

Accessors

endOfLine

  • get endOfLine(): string
  • set endOfLine(value: string): void
  • Separator for rows in CSV stream.

    Returns string

  • Separator for rows in CSV stream.

    Parameters

    • value: string

    Returns void

fieldSeparators

  • get fieldSeparators(): number[]
  • set fieldSeparators(value: number[]): void
  • Separator for fields in CSV stream.

    Returns number[]

  • Separator for fields in CSV stream.

    Parameters

    • value: number[]

    Returns void

quoteSymbols

  • get quoteSymbols(): number[]
  • set quoteSymbols(value: number[]): void
  • Character to quote strings.

    Returns number[]

  • Character to quote strings.

    Parameters

    • value: number[]

    Returns void

scanner

Methods

clearCharacterStates

  • clearCharacterStates(): void

getCharacterState

hasNextToken

  • hasNextToken(): boolean

nextToken

Protected readNextToken

setCharacterState

  • setCharacterState(fromSymbol: number, toSymbol: number, state: ITokenizerState): void

tokenizeBuffer

  • tokenizeBuffer(buffer: string): Token[]

tokenizeBufferToStrings

  • tokenizeBufferToStrings(buffer: string): string[]

tokenizeStream

tokenizeStreamToStrings

  • tokenizeStreamToStrings(scanner: IScanner): string[]

Generated using TypeDoc