Options
All
  • Public
  • Public/Protected
  • All
Menu

Class Tokenizer

This class takes a set of highlighting rules, and creates a tokenizer out of them. For more information, see the wiki on extending highlighters.

Hierarchy

  • Tokenizer

Index

Constructors

Methods

Constructors

constructor

  • Constructs a new tokenizer based on the given rules and flags.

    Parameters

    • rules: Object

      The highlighting rules

    Returns Tokenizer

Methods

getLineTokens

  • getLineTokens(line: string, startState: string | string[]): Token[]

  • Returns an object containing two properties: tokens, which contains all the tokens; and state, the current state.

    Parameters

    • line: string
    • startState: string | string[]

    Returns Token[]

Generated using TypeDoc