Application Load Balancer

This is a Fly App that works as an application load balancer. You can use it to route traffic to different backends, or put it in front of other services distribute traffic based on whatever logic you desire.

It is designed to be a ready to go load balancer library you can mix and match with other Fly App code to do interesting things like add path routing rules, authentication, caching etc.

Try it out

npm install -g @fly/fly
git clone https://github.com/superfly/load-balancer.git
cd load-balancer
yarn install
yarn start

Make some changes

cd my-balancer
open index.ts

How it works

The load balancer takes a list of fetch functions, tracks statistics for each, and attempts to intelligently distribute requests to each. The fetch functions can wrap logic for any kind of backend service, the example just makes simple HTTP proxies using @fly/proxy to example domains.

This example is written in TypeScript. Compiler enforced types are really nice for algorithms like this, since the compiler itself can help limit complexity. Have a look at the [webpack.fly.config.js](https://github.com/superfly/fly/blob/master/examples/load-balancer/webpack.fly.config.js) to see how TypeScript works with Fly.

Backend health

Backends get a health score calculated using a time decaying error rate. Each backend tracks the last 10 responses, calculates an "error percentage", and then weights it based on the age of the most recent error.

2 Random choices

The backend selector chooses two backends with the highest current health score, then "flips a coin" to decide which to send a request to. This is a basic implementation of a "2 random choices" algorithm.

Retries

For GET and HEAD requests, the balancer will attempt to retry failed requests.