eslint-plugin-max-tokens
is an ESLint plugin that enforces a maximum token length for files in your project. It helps ensure that your source code files do not exceed a specified token limit, which can be useful for optimizing code size and adhering to certain constraints.
You can install eslint-plugin-max-tokens
using npm:
npm install eslint-plugin-max-tokens --save-dev
To use the plugin, you need to add it to your ESLint configuration file (e.g., .eslintrc.json
) and enable the max-tokens/max-tokens
rule:
{
"plugins": ["max-tokens"],
"rules": {
"max-tokens/max-tokens": ["warn", { "maxTokenLength": 8192 }]
}
}
In the above example, the max-tokens/max-tokens
rule is set to "warn"
, and the maxTokenLength
option is set to 8192
. You can adjust the maxTokenLength
value according to your project's requirements.
The max-tokens/max-tokens
rule enforces a maximum token length for each file in your project. It uses the gpt-tokenizer
library to tokenize the source code and counts the number of tokens.
If the token length of a file exceeds the specified maxTokenLength
, the rule will report a warning or error (depending on the rule configuration).
The max-tokens/max-tokens
rule accepts an optional options object with the following property:
-
maxTokenLength
(number): The maximum allowed token length for a file. Default value is4096
.
{
"plugins": ["max-tokens"],
"rules": {
"max-tokens/max-tokens": ["error", { "maxTokenLength": 10000 }]
}
}
In this example, the max-tokens/max-tokens
rule is set to "error"
, and the maxTokenLength
option is set to 10000
. Files exceeding 10,000 tokens will trigger an error.
eslint-plugin-max-tokens
is licensed under the MIT License.
Contributions to eslint-plugin-max-tokens
are welcome! If you find any issues or have suggestions for improvements, please open an issue or submit a pull request on the GitHub repository.