Function normalizeCodeEditorTokens

normalizeCodeEditorTokens(
    line: string,
    tokens: readonly Readonly<
        { kind: CodeEditorSyntaxTokenKind; text: string },
    >[],
): readonly Readonly<{ kind: CodeEditorSyntaxTokenKind; text: string }>[]
Sanitizes custom tokenizer output and guarantees full line coverage. Invalid/mismatched token arrays degrade to a single plain token.
Parameters
- line: string
- tokens: readonly Readonly<{ kind: CodeEditorSyntaxTokenKind; text: string }>[]
Returns readonly Readonly<{ kind: CodeEditorSyntaxTokenKind; text: string }>[]
- Defined in core/src/widgets/codeEditorSyntax.ts:1040