jsrel

JavaScript lightweight synchronous RDB

npm install jsrel
59 downloads in the last week
191 downloads in the last month

JSRel

description

JavaScript synchronous RDB (Relational database) without SQL

Available in modern browsers, Node.js and Titanium(NEW!).

This ISN'T ORM, but SQL-less RDB implemented in JavaScript!

Get it!

    $ npm install jsrel

or

    $ curl https://raw.github.com/shinout/jsrel/master/install-jsrel.sh | sh

API at a glance

First, define the schema

var JSRel = require("jsrel");
var db = JSRel.use("dbname", {schema: 
  { user: { name : true, is_activated: "on", $uniques: "name"},
    book: { title: true, price: 1, author: "user", $indexes: "title" },
}});

Second, insert data

var u1 = db.ins('user', {name: 'shinout'});
var u2 = db.ins('user', {name: 'xxxxx', is_activated: false});
var b1 = db.ins('book', {title: 'how to jsrel', price: 10, author: u1});
var b2 = db.ins('book', {title: 'JSRel API doc', price: 20, author_id: u1.id});

Find them!

var users = db.find('user', {is_activated: true});

Get one!

var shinout = db.one('user', {name: "shinout"});

Greater Than, Less Equal!

var booksGreaterThan5  = db.find('book', { price: {gt: 5} } );
var booksLessEqual15   = db.find('book', { price: {le: 15} } );

Like xxx%

var booksLikeJS = db.find('book', { title: {like$: "JS"} } );

Join!

var usersJoinBooks = db.find('user', {is_activated: true}, {join: "book"});

OrderBy! Offset! Limit!

var users = db.find('user', null, {order: "name", limit : 10, offset : 3} );

Perpetuation

db.save();

Export / Import

var str = db.$export();
    var newDB = JSRel.$import("newID", str);

dump as SQL!

var sql = db.toSQL();

suitable applications

  • rich client applications
  • tiny serverside applications
  • client caching
  • mock DB

NOT suitable for applications which require scalability.

motivation

Thinking about the separation of the Model layer.

If we connect to DB asynchronously, we must handle lots of callbacks in a model method.

model.getUserBooks = function(name, callback) {
  db.find("user", {name: name}, function(err, users) {
    db.find("book", {user_id: users[0].id}, callback);
  });
};

If we access to DB synchoronously, we can easily write human-readable model APIs.

model.getUserBooks = function(name) {
  var user  = db.find("user", {name: "xxyy"})[0];
  return db.find("book", {user_id: user.id});
};

Also, synchoronous codes have an advantage of error handling.

for those who dislike Blocking APIs

Why not making it standalone using WebWorker (browsers) or child_process.fork() (Node.js)? Then the whole calculation process doesn't affect the main event loop and we can get the result asynchronously.

I prepared another JavaScript library for this purpose.

standalone.

Then, we can access model methods like

model.getUserBooks("user01", function(err, result) {
})

by defining

model.getUserBooks = function(name) {
  var user  = db.find("user", {name: "xxyy"})[0];
  if (!user) return [];
  return db.find("book", {user_id: user.id});
};

That is, try/catch and asynchronous APIs are automatically created via standalone.

See make it standalone for detailed usage.

installation

    $ npm install jsrel

for development in Titanium or web browsers,

    $ curl https://raw.github.com/shinout/jsrel/master/install-jsrel.sh | sh

in browsers,

<script type="text/javascript" src="/path/to/SortedList.js"></script>
<script type="text/javascript" src="/path/to/jsrel.js"></script>

in Node.js or Titanium,

var JSRel = require('jsrel');

is the way to load the library.

In browsers, the variable "JSRel" is set to global.

In Web Worker,

importScripts('/pathto/SortedList.js', '/pathto/jsrel.js');

See also make it standalone.

dependencies

JSRel internally uses SortedList When installed with npm, it is automatically packed to node_modules/sortedlist Otherwise, it is recommended to run the following command to prepare jsrel and sortedlist.

    $ curl https://raw.github.com/shinout/jsrel/master/install-jsrel.sh | sh

In Titanium, you have to set jsrel.js and SortedList.js at the top of Resources directory.

JSRel API documentation

JSRel

  • JSRel.use(uniqId, options)
  • JSRel.create(uniqId, options)
  • JSRel.$import(uniqId, data_str, options)
  • JSRel.uniqIds
  • JSRel.isNode
  • JSRel.isBrowser
  • JSRel.isTitanium

instance of JSRel (jsrel)

  • jsrel.table(tableName)
  • jsrel.save()
  • jsrel.$export()
  • jsrel.on(eventName, func, options)
  • jsrel.off(eventName, func)
  • jsrel.toSQL(options)
  • jsrel.origin()
  • jsrel.drop()
  • jsrel.id
  • jsrel.name
  • jsrel.tables
  • jsrel.schema

instance of JSRel Table (table)

  • table.columns
  • table.ins(obj)
  • table.upd(obj, options)
  • table.find(query, options)
  • table.one(id)
  • table.one(query, options)
  • table.del(id)
  • table.del(query)

shortcut

  • jsrel.ins(tableName, ...)
  • jsrel.upd(tableName, ...)
  • jsrel.find(tableName, ...)
  • jsrel.one(tableName, ...)
  • jsrel.del(tableName, ...)

JSRel.use(uniqId, options)

Creates instance if not exist. Gets previously created instance if already exists.

uniqId is the identifier of the instance, used for storing the data to external system (file system, localStorage and so on). options is as follows.

key type required? description example
storage string no type of external storages. oneof "mock", file" "local" "session"
When running in Node.js or in Titanium, "file" is set by default.
uniqId is the path name to save the data to. When running in browsers, "local" is set by default.
local means "localStorage", session means "sessionStorage". When running in Web Worker, "mock" is set and no other options can be selected.
"mock" saves nothing. This is limitation of Web Worker which cannot access to Web Storages. In this case, exporting the data to the main thread, we can manually handle and store the data.
"file"
schema object required DB schema (see SCHEMA JSON)
reset boolean no (default false) if true, reset db with the given schema. true
name string no (default : the same as uniqId) the name of the db appname_test
autosave boolean no (default false) whether to auto-saving or not true

SCHEMA JSON

{
  tableName1: tableDescription,
  tableName2: { 
    columnName1 : columnDescription,
    columnName2 : columnDescription
  }
}

table description

key type description example
(columnName) columnDescription column to set.
name limitation
Cannot set [id, ins_at, upd_at] as they are already used by default.
Cannot set [$indexes, $uniques, $classes] as they make conflict in schema description.
Cannot set [str, num, bool, on, off] as they make conflict in column description.
Cannot set [join, order, limit, offset, as, where, select, explain] as they make conflict in search options.
Cannot include "," or "." as it is used in indexing or searching.
Cannot set (RelatedTableName)_id as it is automatically set.
age: "num"
$indexes Array list of indexes. child arrays are lists of columns to make an index.
If string given, converted as array with the value
[["name"], ["firstName", "lastName"]]
$uniques Array (the same as $indexes, but this means unique index) [["name", "pass"]]
$classes Array (the same as $indexes, but this means classified index) "type_num"

column description

example description
{type: "str"} type is string. type must be one of ["str", "num", "bool", (columnName)]
{type: "str", required: false} type is string, and if not given, null is set. required option is false by default
{type: "bool", _default: true} type is boolean, and if not given, true is set.
{type: "num", required: true} type is number, and if not given, an exception is thrown.
"str" type is string, and not required.
"num" type is number, and not required.
"bool" type is boolean, and not required.
true type is string, and required.
false type is string, and not required.
1 type is number, and required.
0 type is number, and not required.
"on" type is boolean, and default value is true.
"off" type is boolean, and default value is false.
{type: tableName} type is the instance of a record in tableName.
the column columnName_id is automatically created.
We can set columnName_id instead of columnName in insertion and updating.
This column is required unless you set required: false.
{type: tableName, required: false} type is the instance of a record in tableName and not required.
tableName type is the instance of a record in tableName and required.

JSRel.create(uniqId, options)

Creates instance if not exist, like JSRel.use. Throws an error if already exists, unlike JSRel.use. Arguments are the same as JSRel.use except options.reset, which is invalid in JSRel.create()

JSRel.$import(uniqId, data_str, options)

Imports data_str and creates a new instance with uniqId. data_str must be a stringified JSON generated by jsrel.$export().

key type required? description example
force boolean no (default : false) if true, overrides already-existing database of the same uniqId. otherwise throws an error. true
name string no the name of the db. If undefined, the imported name is used. appname_2
autosave boolean no whether to auto-saving or not. If undefineed, the imported autosave preference is used. true
storage string no type of external storages. see options of JSRel.use(). If undefined, the imported storage preference is used. "file"

Returns instance of JSRel.

As "import" is a reserved word in JavaScript, we first named this function "$import". However, CoffeeScript enables us to use reserved words, then we can also use JSRel.import as the alias.

JSRel.isNode

(ReadOnly boolean) if Node.js, true.

JSRel.isBrowser

(ReadOnly boolean) if the executing environment has "localStorage" and "sessionStorage" in global scope, true.

JSRel.isTitanium

(ReadOnly boolean) if Titanium, true.

instanceof JSRel (shown as jsrel)

jsrel.table(tableName)

Returns a table object whose name is tableName (registered from the schema). If absent, throws an exception.

jsrel.save()

Saves current data to the storage. Returns jsrel

jsrel.$export()

Exports current data as the format above. Returns data.

As "export" is a reserved word in JavaScript, used "$export" instead.

jsrel.on(eventName, func, options)

Registers hook functions. eventName is the name of the event to bind the function func.

events

event name emitted when arguments to be passed
ins data are inserted
  • tableName : table name
  • insObj : inserted object (with id)
ins:{tablename} data are inserted into {tablename}
  • insObj : inserted object (with id)
upd data are updated
  • tableName : table name
  • updObj : updated object
  • oldObj : object before updating
  • updColumns :updated columns (Array)
upd:{tablename} data are updated in {tablename}
  • updObj : updated object
  • oldObj : object before updating
  • updColumns :updated columns (Array)
del data are deleted
  • tableName : table name
  • delObj : deleted object
del:{tablename} data are deleted in {tablename}
  • delObj : deleted object
save:start at the start of jsrel.save()
  • origin : result of db.origin()
save:end at the end of jsrel.save()
  • data : saved data

options

option name type description default
unshift boolean registers a function to the top of the list false

jsrel.off(eventName, func)

Unregister hook functions registered in eventName. If a function func is registered in eventName hooks, it is removed. If func is null, all functions registered in eventName is removed.

jsrel.toSQL(options)

Gets SQL string from the current schema and data.

options

option name type description default example
noschema boolean if true, schema SQLs (create statements) are not generated. null true
db boolean or string if true, create database whose name is id of the db, if string given, the value is set as database's name. if not set, database creation (CREATE DATABASE xxxx) does not occur. null true
nodrop boolean if true, drop statements are not generated. null true
nodata boolean if true, data SQLs (insert statements) are not generated. null true
type string type of RDBs. Currently, "mysql" is only tested. "mysql" "mysql"
engine string MySQL engine (only enabled when options.type is "mysql") "InnoDB" "MyISAM"
rails (unstable) boolean if true, rails-like date format (created_at, inserted_at) is output. null true

jsrel.origin()

Gets the last savedata.

Unless jsrel.save() has been called at least once, null is returned.

  var savedata = jsrel.origin();
  var newdb = JSRel.$import("new_db", savedata);

jsrel.drop(tableName1, tableName2, ...)

Drops given tables. If dependencies exist, jsrel follows the following rules.

  1. throw an error if the given table group contains another reference table
  2. set all the ids of referred columns to null

jsrel.id

(ReadOnly) gets id

jsrel.name

(ReadOnly) gets name

jsrel.tables

(ReadOnly) gets list of registered tables

[table1, table2, ...]

jsrel.schema

(ReadOnly) gets a canonical schema of the database, the same format as schema passed to JSRel.use

Be careful that this property is dynamically created for every access.

var schema = db.schema; // created dynamically
var schema2 = db.schema; // created dynamically
schema  ===  schema2 // false, but deeply equal

var db2 = JSRel.use("db2", {schema: schema});  // the same structure as db

instanceof JSRel.Table (shown as table)

table.columns

(ReadOnly) gets registered columns in the table

[column1, column2, ...]

table.ins(obj)

Registers a new record. obj must be compatible with columns of the table. Otherwise it throws an exception. Returns an instance of the record. It is NOT the same as the given argument, as the new object contains "id".

Before insertion, Type checking is performed. JSRel tries to cast the data.

record object

Record objects have all columns registered in the table.

In addition, they have id, ins_at, upd_at in their key. These are all automatically set.

ins_at and upd_at are timestamp values and cannot be inserted.

id is auto-incremented unique integer.

We can specify id in insertion.

table.ins({id: 11, name: "iPhone"});

When the table already has the same id, an exception is thrown.

relation handling in insertion

OK, let's think upon the following schema.

var schema = { user: {
    nickName : true,
    fitstName: false,
    lastName : false
  },
  card: {
    title : true,
    body  : true
  },
  user_card {
    user: "user",
    card: "card",
    owner: {type : "user", required: false}
    $uniques: { user_card: ["user", "card"] }
  }
}

First, inserts users and cards.

var jsrel = JSRel.use('sample', {schema: schema});

var uTable = jsrel.table('user');
var shinout = uTable.ins({nickName: "shinout"});
var nishiko = uTable.ins({nickName: "nishiko"});
var cTable = jsrel.table('card');
var rabbit = uTable.ins({title: "rabbit", body: "It jumps!"});
var pot    = uTable.ins({title: "pot", body: "a tiny yellow magic pot"});

Then, inserts these relations.

var ucTable = jsrel.table('user_card');
ucTable.ins({ user: shinout, card: rabbit });

We can also insert these relation like

ucTable.ins({ user_id: nishiko.id, card_id: pot.id });
ucTable.ins({ user_id: 1, card_id: 2 }); // 1: shinout, 2: pot

Remember that user_id and card_id are automatically generated and it represent the id column of each instance. When we pass an invalid id to these columns, an exception is thrown.

ucTable.ins({ user_id: 1, card_id: 5 }); // 1: shinout, 5: undefined!

When a relation column is not required, we can pass null.

ucTable.ins({ user: nishiko, card_id: 1, owner_id: null });

When duplicated, xxxx_id priors to xxxx (where xxxx is the name of the original column).

ucTable.ins({ user: nishiko, user_id: 1, card_id: 1 }); // user_id => 1

inserting relations

obj.rel_table = [relObj1, relObj2, ...];
table.ins(obj);

relObj1, relObj2 are also inserted to table "rel_table" containing the new id as the external key.

If the main table is related to the rel_table multiply, you must specify the column like

obj["rel_table.relcolumn"] = [relObj1, relObj2, ...];
table.ins(obj);

table.upd(obj, options)

Updates an existing record. obj must contains id key. Only the valid keys (compatible with columns) in obj is updated. Throws no exceptions when you passes invalid keys. Throws an exception when you an invalid value with a valid key.

Returns an instance of the updated record. It is NOT the same as the given argument.

relation updates

updating related tables

obj.rel_table = [relObj1, relObj2, ...];
table.upd(obj, {append: append});

if relObj contains "id" column, updating the object. Otherwise, inserting the object. If options.append is false or not given, already existing related objects are deleted.

If the main table is related to the rel_table multiply, you must specify the column like

obj["rel_table.relcolumn"] = [relObj1, relObj2, ...];
table.upd(obj, {append: append});

table.find(query, options)

Selects records. Returns a list of records. query is an object to describe how to fetch records.

query examples

example description
{name: "shinout"} name must be equal to "shinout"
{name: ["shinout", "nishiko"]} name must be equal to "shinout" or "nishiko"
{name: {like$: "shin"}} name must be like "shin%"
{name: {$like: "inout"}} name must be like "%inout"
{name: [{$like: "inout"}, {equal: "nishiko"}] } name must be like "%inout" OR equals "nishiko"
{name: {$like: "inout", equal: "nishiko"} } name must be like "%inout" AND equals "nishiko"
{age: {gt: 24} } age must be greater than 24
{age: {gt: 24, le: 40} } age must be greater than 24 and less equal 40
{age: [{ge: 24}, {lt: 40}] } age must be greater equal 24 or less than 40
{country: {$in: ["Japan", "Korea"] } country must be one of "Japan", "Korea" (as "in" is a reserved word in JavaScript, used "$in" instead.)
{name: "shinout", age : {ge: 70 } must returns empty until shinout becomes 70

options is as follows.

key type description example
order mixed see order description { name: "asc" }
limit int the end position of the data 20
offset int offset of the results 10
join mixed see join description {records.scene: {title : {like$: "ABC"} }
select string (one of column names) get list of selected columns instead of objects "title"
select array (list of column names) get list of object which contains the given columns instead of all columns ["name", "age"]
explain object put searching information to the given object {}

order description

example description
"age" order by age asc
{age: "desc"} order by age desc
{age: "desc", name: "asc"} order by age desc, name asc

results

Returns list of instances

[ {id: 1, name: "shinout"}, {id: 2, name: "nishiko"}, ...]

join description

sample data

group

idname
1mindia
2ZZZ

user

idnameagegroup
1shinout251
2nishiko281
3xxx392

card

idtitlebody
1rabbitit jumps!
2pota tiny yellow magic pot
3PCcalculating...

user_card

idusercard
111
221
312
423
533

Fetching N:1 related objects

var result = db.table('user').find({name: "shinout"}, {join: JOIN_VALUE});
No. JOIN_VALUE description result
1 "group" get "group" column as object [{id: 1, name: "shinout", age: 25, group_id: 1, group: {id: 1, name: "mindia"}}]
2 {group : true} get "group" column as object (the same as sample1) [{id: 1, name: "shinout", age: 25, group_id: 1, group: {id: 1, name: "mindia"}}]
3 true get all the related columns as object [{id: 1, name: "shinout", age: 25, group_id: 1, group: {id: 1, name: "mindia"}}]
4 {group : {name: {like$: "mind"}}} get "group" column as object whose name starts at "mind" [{id: 1, name: "shinout", age: 25, group_id: 1, group: {id: 1, name: "mindia"}}]
5 {group : {name: "ZZZ"}} get "group" column as object whose name is equal to "ZZZ" [] // empty

Fetching 1:N related objects

var result = db.table('group').find({name: "mindia"}, {join: JOIN_VALUE});
No. JOIN_VALUE description result
6 "user.group" get "user" table objects (setting the related column in "user" table) [{id: 1, name: "mindia", "user.group": [{id: 1, name: "shinout", age: 25}, {id: 2, name: "nishiko", age: 28}]}]
7 "user" get "user" table objects (if related column is obvious) [{id: 1, name: "mindia", "user": [{id: 1, name: "shinout", age: 25}, {id: 2, name: "nishiko", age: 28}]}]
8 {"user.group" : true } get "user" table objects (the same as sample6) [{id: 1, name: "mindia", "user.group": [{id: 1, name: "shinout", age: 25}, {id: 2, name: "nishiko", age: 28}]}]
9 {"user.group" : {age : {gt: 27}} } get "user" table objects with age greater than 27 [{id: 1, name: "mindia", "user.group": [{id: 2, name: "nishiko", age: 28}]}]
10 {"user.group" : {age : {gt: 27}, as: "users"} } get "user" table objects with age greater than 27, with alias name "users" [{id: 1, name: "mindia", "users": [{id: 2, name: "nishiko", age: 28}]}]
11 {"user.group" : {where : {age : {gt: 27}}, as: "users"} } get "user" table objects with age greater than 27, with alias name "users" (the canonical expression of sample9) [{id: 1, name: "mindia", "users": [{id: 2, name: "nishiko", age: 28}]}]
12 {user : {age : {gt: 27}, as: "users"} } get "user" table objects with age greater than 27, with alias name "users" [{id: 1, name: "mindia", "users": [{id: 2, name: "nishiko", age: 28}]}]
13 {user : {age : {gt: 47}, outer: true} } outer joining. Records containing Empty 1:N subqueries can be remained with the column filled with null. [{id: 1, name: "mindia", "users": null}]
13 {user : {age : {gt: 47}, outer: "array"} } outer joining. Records containing Empty 1:N subqueries can be remained with the column filled with empty array. [{id: 1, name: "mindia", "users": [] }]

Fetching N:M related objects

var result = db.table('user').find({name: "shinout"}, {join: JOIN_VALUE});
15 {"card": {via: "user_card"} } get "card" related through "user_card" [{id: 1, name: "shinout", "card": [ {id:1, ...}, {id: 3, ...}] }]

table.one(id)

Gets one object by id.

table.one(query, options)

Gets one result by table.find().

table.del(id)

Deletes a record with a given id .

table.del(query)

Deletes records with a given query . query is the same argument as table.find(query).

relation handling in deletion

When a record is deleted, related records are also deleted.

Think upon the schema.

First, inserts users, cards and these relations.

var jsrel = JSRel.use('sample', {schema: schema});

var uTable = jsrel.table('user');
var cTable = jsrel.table('card');
var ucTable = jsrel.table('user_card');

var shinout = uTable.ins({nickName: "shinout"});
var nishiko = uTable.ins({nickName: "nishiko"});

var rabbit = uTable.ins({title: "rabbit", body: "It jumps!"});
var pot    = uTable.ins({title: "pot", body: "a tiny yellow magic pot"});

ucTable.ins({ user: shinout, card: rabbit });
ucTable.ins({ user: nishiko, card: rabbit });
ucTable.ins({ user: shinout, card: pot });

Next, delete shinout.

uTable.del(shinout);

Then, the dependent records ( shinout-rabbit, shinout-pot ) are also removed.

ucTable.find().length; // 1 (nishiko-rabbit)

shortcut

  • jsrel.ins(tableName, ...)
  • jsrel.upd(tableName, ...)
  • jsrel.find(tableName, ...)
  • jsrel.one(tableName, ...)
  • jsrel.del(tableName, ...)

are, select table via jsrel.table(tableName) in the first place. Then run the operation using the remaining arguments.

for example,

jsre.ins('user', {nickName: "shinout"});

is completely equivalent to

jsrel.table('user').ins({nickName: "shinout"});

make it standalone

standalone is a library to make a worker process / thread which can communicate with master.

Here are the basic concept.

master.js

standalone("worker.js", function(model) {

  model.getSongsByArtist("the Beatles", function(err, songs) {
    console.log(songs);
  });

});

worker.js

var db = JSRel.use("xxx", {schema: {
  artist: {name: true},
  song  : {title: true, artist: "artist"}
}});
var btls = db.ins("artist", {name: "the Beatles"});
db.ins("song", {title: "Help!", artist: btls});
db.ins("song", {title: "In My Life", artist: btls});

var model = {
  getSongsByArtist: function(name) {
    return db.find("artist", {name : name}, {join: "song", select : "song"});
  }
};
standalone(model);

In master.js, we can use "getSongsByArtist" asynchronously, catching possible errors in err.

In Node.js, standalone spawns a child process.

In browsers, standalone creates a WebWorker instance.

In Titanium, standalone is not supported.

environmental specific code

Because Node.js and WebWorker has a different requiring system, We must be careful of loading scripts.

in Node.js (worker.js)

var JSRel = require('jsrel');
var standalone = require('standalone');

This is enough.

in browsers (worker.js)

importScripts('/pathto/SortedList.js', '/pathto/jsrel.js', '/pathto/standalone.js');

Don't forget to import SortedList (which JSRel depends on).

LICENSE

(The MIT License)

Copyright (c) 2012 SHIN Suzuki shinout310@gmail.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

npm loves you