mongo2mysql

Streaming data from MongoDB into MySQL

npm install mongo2mysql
1 downloads in the last week
4 downloads in the last month

MongoDB to MySQL Data Streaming

Streaming data in MongoDB to MySQL database in realtime. Enable SQL query on data in NoSQL database.

Configurations:

  1. Update the mongodb configuration in config.json
{
  "service": "mycol001",
  "mongodb": {
    "host": "127.0.0.1",
    "port": 27017,
    "db": "test",
    "collection": "blog_posts"
  },
  "mysql": {
    "host": "localhost",
    "user": "root",
    "password": "",
    "db": "test",
    "table": "blog_posts"
  },
  "sync_fields": { // fields store in MySQL
    "_id": "int",  // required field
    "field1": "int",
    "field2": "int",
    "field3": "string",
    "field4": "string"
  },
  "transform" : {
    "field_name" : { // field name changes
      "order": "_order"
    },
    "field_value" : {
      "cid": "transform_cid", // field value changes, need function
    }
  }
}
  1. Add index manually, for example:

    ALTER TABLE blog_posts ADD PRIMARY KEY (_id);

    ALTER TABLE blog_posts ADD INDEX field1 (field1);

    ALTER TABLE blog_posts ADD INDEX _order_id (_order, _id);

  2. Import the old data in MongoDB collection to MySQL table:

    node app.js import

  3. Start the daemon to streaming data

    node start app.js or forever start app.js

  4. A MySQL table mongo_to_mysql will be created to store required information.

  5. Update the transform() to change field names or modify values during streaming.

npm loves you