Falcon for Big Data Crossfiltering

Crossfilter millions of records without latencies. This project is work in progress and not documented yet. Please get in touch if you have questions.

The largest experiments we have done so far is 10M flights in the browser and ~180M flights when connected to MapD.

Demos

• 1M flights in the browser: https://uwdata.github.io/falcon/flights/
• 7M flights in MapD: https://uwdata.github.io/falcon/flights-mapd/
• 500k weather records: https://uwdata.github.io/falcon/weather/

Big Crossfilter uses Apache Arrow and ndarray.

Developers

Install the dependencies with yarn. Then run yarn start to start the flight demo with in memory data. Have a look at the other script commands in package.json.

Experiments

First version that turned out to be too complicated is at https://github.com/uwdata/falcon/tree/complex and the client-server version is at https://github.com/uwdata/falcon/tree/client-server.