rzk-0.11.3: An experimental proof assistant for synthetic ∞-categories
Safe HaskellNone
LanguageHaskell2010

Language.Rzk.VSCode.Tokenize

Synopsis

Documentation

tokenizeModule :: Module -> [SemanticTokenAbsolute] Source #

tokenizeCommands :: Int -> [Command] -> [SemanticTokenAbsolute] Source #

Walk the commands tracking the section depth: an assumption outside any section is a file-wide axiom and is marked as an abstract function (see tokenizeCommand), while an assumption inside a section is a hypothesis the section abstracts over at its #end, so it keeps the parameter token type and only gains the abstract modifier.

tokenizeLanguageDecl :: LanguageDecl -> [SemanticTokenAbsolute] Source #

tokenizeCommand :: Command -> [SemanticTokenAbsolute] Source #

tokenizeDataSort :: DataSort -> [SemanticTokenAbsolute] Source #

tokenizeDataBody :: DataBody -> [SemanticTokenAbsolute] Source #

tokenizeConstructor :: Constructor -> [SemanticTokenAbsolute] Source #

tokenizeDataElim :: DataElim -> [SemanticTokenAbsolute] Source #

tokenizeDeclUsedVars :: DeclUsedVars -> [SemanticTokenAbsolute] Source #

tokenizeSectionName :: SectionName -> [SemanticTokenAbsolute] Source #

tokenizeBind :: Bind -> [SemanticTokenAbsolute] Source #

tokenizeParam :: Param -> [SemanticTokenAbsolute] Source #

tokenizePattern :: Pattern -> [SemanticTokenAbsolute] Source #

tokenizeMatchBranch :: MatchBranch -> [SemanticTokenAbsolute] Source #

A match branch: the constructor name is an enum member (as constructor uses are), the binders are parameters, the body is an ordinary term.

tokenizeTope :: Term -> [SemanticTokenAbsolute] Source #

tokenizeTerm :: Term -> [SemanticTokenAbsolute] Source #

tokenizeTerm' :: Maybe SemanticTokenTypes -> Term -> [SemanticTokenAbsolute] Source #

tokenizeRestriction :: Restriction -> [SemanticTokenAbsolute] Source #

tokenizeParamDecl :: ParamDecl -> [SemanticTokenAbsolute] Source #

tokenizeModalColon :: ModalColon -> [SemanticTokenAbsolute] Source #

tokenizeModality :: Modality -> [SemanticTokenAbsolute] Source #

tokenizeModComp :: ModComp -> [SemanticTokenAbsolute] Source #

tokenizeSigmaParam :: SigmaParam -> [SemanticTokenAbsolute] Source #

mkToken :: (HasPosition a, Print a) => a -> SemanticTokenTypes -> [SemanticTokenModifiers] -> [SemanticTokenAbsolute] Source #

Syntax highlighting from the token stream

tokenizeSyntaxSymbols :: Text -> [SemanticTokenAbsolute] Source #

Highlight the fixed syntax of the language (command names, reserved words, operators) and holes directly from the lexer token stream.

This complements tokenizeModule, which highlights identifiers and special term formers from the parsed module. Fixed symbols do not need parsing at all: a := is a := wherever it occurs, and a hole ? (or ?name) is its own lexer token. Working on the token stream means that the grammar (and the abstract syntax) does not have to track positions of keywords, and that highlighting keeps working for files that (temporarily) fail to parse.

classifyToken :: Tok -> Maybe (Text, SemanticTokenTypes) Source #

How to highlight a lexer token, if at all: fixed symbols by classifySymbol, holes as a distinct token. Identifiers are left to the AST pass (tokenizeModule), which knows their role.

classifySymbol :: Text -> Maybe SemanticTokenTypes Source #

How to highlight a fixed symbol of the grammar, if at all.

mergeTokens :: [SemanticTokenAbsolute] -> [SemanticTokenAbsolute] -> [SemanticTokenAbsolute] Source #

Combine tokens from the parsed module with tokens from the raw symbol stream. On overlap (same start position) the AST-based token wins, since it carries more precise semantics (e.g. unit as an enum member rather than a keyword). The result is sorted by position, as required for the LSP delta encoding.