mongolfier

A Simple MySQL to MongoDB Migration Tool

npm install mongolfier
37 downloads in the last month

mongolfier

A Simple MySQL to MongoDB Migration Tool

Montgolfier Balloon

NOTE: not "montgolfier" but "mongolfier". ;)

install and run

local install and run

npm install mongolfier
node ./node_modules/.bin/mongolfier -h

global install and run

npm install -g mongolfier
mongolfier -h

usage

$ mongolfier -h
A Simple MySQL to MongoDB Migration Tool
Usage: node ./bin/mongolfier.js [options]

Options:
--configfile, -f       path to config file                     [default: "config.json"]
--logfile, -l          path to log file                      
--collection, -c       collection name to migrate            
--emptycollection, -E  make empty collection before migration  [boolean]
--bulkcopy, -B         do bulk copy if no template available   [boolean]
--failonerror, -R      stop at first failure                   [boolean]
--dryrun, -D           do not insert into collection           [boolean]
--debug, -X            show debug information                  [boolean]
--quiet, -q            be extra quiet                          [boolean]
--verbose, -v          be extra verbose                        [boolean]
--help, -h             show this message                     

configuration

config file(json)

  • NOTE: no comment allowed here. this file should conforms to strict json format.
  • NOTE: {{ and }} is just a marker. replace it with yours.

{
  "mysql": {
    "host": "{{mysql_host}}",
    "port": {{mysql_port}},
    "user": "{{mysql_user}}",
    "password": "{{mysql_password}}",
    "database": "{{mysql_database}}",
    ...
  },
  "mongo": {
    "host": "{{mongo_host}}",
    "port": {{mongo_port}},
    "user": "{{mongo_user}}",
    "password": "{{mongo_password}}",
    "database": "{{mongo_database}}",
    ...
  },
  "context": {
    "{{custom_attr_key}}": "{{custom_attr_value}}",
    ...
  },
  "before": [
    "{{before_script.js}}",
    ...
  ],
  "after": [
    "{{after_script.js}}",
    ...
  ]
  "collections": [
    {
      "collection": "{{mongo_collection_name}}",
      "template": "{{path_to_collection_template}}",
      "query": "{{mysql_select_query}}"
    },
    {
      "collection": "{{mongo_collection_name}}",
      "template": "{{path_to_collection_template}}",
      "query": [
        "{{mysql_select_query_part1}}",
        "{{mysql_select_query_part2}}",
        ...
      ]
    },
    ...
  ]
}
  • context is optional. this could be accessed via $ variable across processing all collections.
  • before is optional. these scripts are evalulated before processing the first collection.
  • after is optional. these scripts are evalulated after processing the last collection.
  • template is optional(default: {{collection name}}.json).
  • query is string or array. array will be joined to a query.

mapping template/script

NOTE: comment allowed here. this file is javascript or something. ;)

NOTE: {{ and }} is just a marker. replace it with yours.

simple mapping template


({
  // use mongodb ObjectID
  "_id": new mongo.ObjectID(),
  // use mysql column values
  "{{mongo_field_name}}": ${{mysql_column_name}},
  ...
})

complex mapping template


var id = new mono.ObjectID();
...
({
  "_id": id,
  "foo": ($bar).toUpperCase(),
  "sum": $num1 + $num2,
  "now": new Date(),
  ...
})
  • NOTE on the enclosing braces the result. do not use return keyword.

async mapping template


var d = q.defer();
var id = new mongo.ObjectID();
setTimeout(function () {
  ...
  d.resolve({
    "_id": id,
    "foo": ($bar).toUpperCase(),
    "sum": $num1 + $num2,
    "now": new Date(),
    ...
  })
}, 100);
d.promise;
  • NOTE on the last line d.promise;. do not use return keyword.

predefined objects

TBD... ;)


That's all folks.

May the SOURCE be with you...

npm loves you