MView - Materialized Views Library
MView is a distributed datatypes library. It consumes data-feeds and produces materialized views as javascript objects. State is kept in memory, but may be persisted through the dump/load api.
API Overview
var mview =
API Spec:
mviewmviewtextoptsmviewmview
Where:
opts = noTombstones: bool // default false, turns off tombstone tracking
Tombstone tracking is a technique to ensure consistency. It adds memory overhead, but allows messages to arrive out of order. Applications which implement their own causal ordering may disable tombstones.
Using MView
MView is designed for distributed programs with reliable message broadcast. It was specifically built for secure scuttlebutt. It supports 4 different view types: registers, text buffers, sets, and ordered lists. The views are instantiated by the node's software at load, then used as follows:
- To generate view updates. These updates are broadcast on the network.
- To consume updates. As update-messages are received from the network, they are fed to the views.
- To produce the current state. When the node software needs to use the views, it asks them for their values.
In the examples below, there will be two functions which represent the networks's interface: net.broadcast()
and net.on('msg')
.
- Assume that
broadcast()
takes a message object and sends it to all nodes (including the local node). - Assume that
on('msg')
is called each time there is a new message from the local node or a remote node.
Registers
Register views are opaque values. They are designed to converge on the most recent update value, and will not merge the values given. Its algorithm is based on the Observed Remove Set CRDT.
var reg = mview
API Spec:
reg // updates the valueregtags // gives the current tagsreg // gives the current value
Registers use unique identifiers known as tags to track the partial order of updates. To generate an update, you capture the currently-set tags with reg.tags()
- these are the values used in previousTags
. The update also requires a new unique tag (the tag
parameter) which the application must generate.
Example Usage:
var cuid = { net}net
Text
Text views are strings. They are designed to merge updates so that users can make concurrent updates to its contents. Its algorithm is based on the Logoot CRDT.
var text = mviewtext
API Spec:
text // updates the string with the given difftext // generates a diff between the current string and the given stringtext // gives the current value
Text views use diffs structures to apply updates. These diffs are generated by passing the new value to diff()
.
Example Usage:
{ net}net
Set
Set views are unordered sets of unique values. Its algorithm is based on the Observed Remove Set CRDT.
var set = mview
API Spec:
set // adds the value to the setset // removes the value from the set settagsvalue // gets the tags for a given value set // gives the current valueset // returns boolset // returns the number of values in the setset
Like the register view, the set uses tags in order to track the order of updates. These tags are generated by the application.
Example Usage:
var cuid = { net} { net}net
List
List views are ordered arrays of values. Its algorithm is based on the Logoot CRDT.
var list = mview
API Spec:
list // inserts the value into the listlist // removes the value represent by tag from the list listtagsindex // gets the tags at the given position in the listlist // generates a tag which sorts greater than tagA and less than tagB list // gets the current list valuelist // gets the value at the given tag or indexlist // gets the number of values in the listlist
The list view uses tags like the set, but its tags are ordered (using the Logoot algorithm). This means the application uses between()
to generate its tags.
between()
takes two tags and generates a new tag that sorts in between them. It uses a weak random function to avoid tag-collisions between ndoes. For a strong guarantee, pass a unique id as the third param of between()
. Doing so will increase the key length, but may be necessary in some applications.
Example Usage:
var cuid = { net} { net} { net} { net}net