Crate pomelo_lex

source ·
Expand description

A basic lexer for SML ’97.

Heavily inspired by rustc_lexer.

Re-exports

pub use token::LexKind;
pub use token::LexToken;

Modules

cursor 🔒
token
Lexical tokens and their kinds

Structs

LexedStr 🔒

Functions

is_ident_alpha_cont 🔒
is_ident_alpha_start 🔒
is_ident_symb 🔒
is_number_start 🔒
is_reserved_symbol_start 🔒
is_whitespace 🔒
lex
Convert the input src to a sequence of tokens.