glsl-tokenizer

r/w stream of glsl tokens

npm install glsl-tokenizer
6 downloads in the last week
81 downloads in the last month

glsl-tokenizer

a readable / writable stream that maps input to glsl tokens, if possible.


  var fs = require('fs')
    , tokens = require('glsl-tokenizer')()

  fs.createReadStream('some.glsl')
    .pipe(tokens)
    .on('data', function(token) {
      console.log(token.data, token.position, token.type)
    })

API

tokens = require('glsl-tokenizer')()

return a tokenizer stream instance.

emits 'data' events whenever a token is parsed with a token object as output.

tokens


{ 'type': TOKEN_TYPE
, 'data': "string of constituent data"
, 'position': integer position within the data stream }

The available token types are:

  • block-comment: /* ... */
  • line-comment: // ... \n
  • preprocessor: # ... \n
  • operator: Any operator. If it looks like punctuation, it's an operator.
  • integer
  • float: Optionally suffixed with f
  • ident: User defined identifier.
  • builtin: Builtin function
  • keyword
  • whitespace
  • eof: emitted on end; data will === '(eof)'.

License

MIT

npm loves you