Browse by Keyword: "tokenize"
c-tokenizer tokenize c/c++ source code
cljs-tokenizer Clojurescript tokenizer
css-tokens A regex that tokenizes CSS.
json-tokenizer A streaming JSON tokenizer
nlp-toolkit A little helper module to execute nlp operations
StringScanner StringScanner performs lexical scanning operations on a string.
vast vAST: A (mostly) generic abstract syntax tree generator.