budgeteer
Cost-based event rate limiter, scheduler, and deduplicator. Uses a Redis backend. Provides the "brain" of a simple job scheduling system.
Features:
- Event deduplication: Events that have already been processed, or are scheduled for delayed processing, are dropped.
- Cost based accounting, giving each key a token budget per day. Clients are free to model costs any way they like. Typically, execution time (in seconds) is a major input into the cost function.
- Delay and retry scheduling: An execution delay is calculated once an event's
budget turns negative. The delay is proportional to the time it takes to
even out the budget, based on the
token_budget_per_day
configuration option.
Usage example
const budgeteer = store: type: 'redis' // Default & only supported storage type so far. host: 'redishost' port: 12345 ttl: 86400 * 7 // Store rate limiting / dedup information for 7 days ;const key = 'someName'; // Token budget configuration, typically differs per event type.const token_budget = tokens_per_day: 24 // Steady state token rate. max_balance: 36 // Maximum balance available for bursts, when a job is // new or has not been executed in a long time.; return budgeteer