matryoshka

Matryoshka is a package for Cordova/PhoneGap 3.0 and above that lets you do live, on-device updating of the app's code.

npm install matryoshka
1 downloads in the last day
5 downloads in the last week
9 downloads in the last month

matryoshka

Notice: This should be considered experimental code at the moment — it may not be ready for prime time. I've been testing it on iOS and Android. Other platforms are totally untested. Testers are welcome, but please back up your code before starting and report any issues you might encounter. If you want to add a new platform, see below.

Matryoshka is for Cordova/Phonegap 3.0 and above. It would probably be possible to make it work with earlier versions.

What does it do?

For a quick overview, see the Demo Video

  1. Monitors your top-level www directory for changes and updates the platform-specific www directory accordingly (in addition to use with Matryoshka, this is handy if you're actually building the product with Xcode or Eclipse).
  2. Pushes these code updates to the device or simulator/emulator, without requiring a rebuild and reinstall. Save a change to the www folder at the top level and your device is running the new code within seconds.

Recent Changes

Building a shell app is no longer required (though it still has certain advantages, see below). Just add the new matryoshka-client.js to your HTML files, run the server, and you're good to go.

Installation

Clone the repo and run:

npm install -g

from the repo root folder.

If you want to remove it later, use:

npm uninstall -g matryoshka

You can also get the package from the npm registry (without cloning the repo) by just running:

npm install -g matryoshka

However, you will still need the matryoshka-client.js file from the repo (it goes with your own HTML files rather than the node-based server).

Basic Usage (without building a shell app)

  • Add matryoshka-client.js (from the repo) to your project, and add a script element for it to every HTML file you want to dynamically update.
  • Set the whitelist for your project so it can load code from your development machine's IP address. On Android and iOS, this is done by setting the access origin key in the top-level www/config.xml file. The key should look like this afterward:

<access origin="http://(your dev machine's IP)" />

  • If you're brave, or you need to load code from multiple sites, you can set the entry to "*" temporarily. This allows loading code from anywhere, so use caution. You'll probably want to change this before releasing your app to the public.

  • Some of the other platforms require handling the whitelist in a slightly different way. Check the Domain Whitelist Guide for details.

  • Build the project (but don't run it yet) to make sure everything is in place, all folders created, etc.

  • Open a new terminal shell and navigate to the top level of your project. Run:

matryoshka [platform] where platform is ios, android, etc.

  • You may also specify an optional port number after the platform. If not supplied, the server will run on port 8089.

  • Your system's firewall may ask for permission to receive incoming connections. If so, grant it.

  • When the Matryoshka server starts, it should tell you which port number and IP address(es) it is using. Make a note of the external IP address (i.e., the one that is not 127.0.0.1). This should be the same IP address you set in the whitelist above. If not, change the whitelist entry and rebuild the project.

  • After it starts up, the server will begin monitoring the top-level www folder for changes and should automatically propagate those changes into the platform-specific www folder (e.g., platforms/ios/www).

  • Run your app. You should get some prompts asking for the IP address and port for the Matryoshka server. Enter the same ones that the server displayed on startup.

  • Once the server and app are both running, any changes under the top-level www folder in your project (editing, adding, or deleting files) should be reflected in the running app within a few seconds.

Cool beans, huh?

When you're finished, just terminate the Matryoshka server (Control-C on most platforms).

If you need to return to the main screen in the Matryoshka app, the matryoshka-client.js file exposes a matryoshkaHome() function that you can call from within your app (or from the JavaScript console, etc.).

Known and Potential Limitations, Issues, and Workarounds

  1. If your Matryoshka server is running on a machine on the public internet, anyone will be able to see your code. It is recommended that you run the server on a machine behind a NAT (or otherwise firewalled against incoming connections from the public net). However, you also need to make sure that your devices can connect to the server (see the discussion in troubleshooting below). Note that the built-in web server in Cordova also works this way. These servers are for testing and development, and are not hardened/secured for general use on the public internet.

  2. Before building and submitting a production app to the iOS App Store, remove both the matryoshka-client.js and the script elements for it in your HTML files. I guarantee that Apple's reviewers won't be amused by finding dynamic code loading in a production app. :-)

  3. Matryoshka can only make live updates for the web portion of your code (HTML, JavaScript, CSS, web image resources, etc.). It can't update native code (e.g., plugins). For native stuff you'll still have to rebuild and reinstall before any changes are reflected.

Plugin Compatibility

In theory, most plugins should work as long as the plugin is added to both the Matryoshka app and the app under test. In practice there are some exceptions which will be noted here as they're discovered. At this early stage, "seems to work" in this list should be interpreted as "tried with one of the simple examples in the docs" rather than "rigorously tested". :-)

Anything that uses a file:// URL to load stuff from the device is likely to cause trouble. When your app is running in Matryoshka, it's being loaded as a remote site over http. Browser security restrictions prevent direct access to local files from remote web sites. Normally this is a very good thing (you don't want random web pages reading files from your hard drive) but it's troublesome for an app of this type. There may be a way around this (configuring security on the web view, e.g.) — investigating.

  • Accelerometer: seems to work on iOS. Not working on one of my Android devices (but does work on the other). Investigating.

  • Camera: partial support on both iOS and Android. Taking pictures works, as does returning the image with Camera.DestinationType.DATA_URL. Camera.DestinationType.FILE_URI partially works, in the sense that you can take a picture, but suffers from the "loading file:// from http://' issue discussed above.

  • Capture: untested. Likely also suffers from the "loading file:// from http://' issue.

  • Compass: seems to work on both Android (if the device has a compass) and iOS.

  • Connection: seems to work on both Android and iOS.

  • Contacts: seems to work on both Android and iOS.

  • Device: seems to work on both Android and iOS.

  • Events: deviceready seems to work on both Android and iOS. Others untested.

  • File: seems to work on both Android and iOS (tested using my gapfile gapfile plugin).

  • Geolocation: seems to work on both Android (if the Android device has GPS, some don't) and iOS.

  • Globalization: untested.

  • InAppBrowser: seems to work on both Android and iOS.

  • Media: untested.

  • Notification: seems to work on both Android and iOS (both dialog and vibration).

  • Splashscreen: untested.

  • Storage: seems to work on both Android and iOS. However, do note that the local storage for your app running stand-alone and the local storage for your app running in Matryoshka are going to be different.

Troubleshooting

If it doesn't work, check these possible causes:

  • Are the Matryoshka app and server able to see each other? One way to check is to go into the regular browser on the mobile device and try to load http://(server ip address):(server port)/index.html. If it comes up in the browser, you're good. If not, make sure that your device is on the same network as the Matryoshka server (e.g., on the same wireless LAN). It definitely won't work if your device is on the cellular data network and your server is behind a NAT or firewall.

  • JavaScript or other errors in your code might cause the updates to stop working. If you're using a debugging tool that lets you connect to the JavaScript console on the device (e.g., the Web Inspector in Safari, weinre, etc.) you should be able to force a manual reload of your app by running this line in the console:

window.location='http://(server IP address):(server port)/index.html'

  • If you're using a shell app (below) you can go back to the Matryoshka startup page with:

matryoshkaHome()

  • If you find yourself doing these things often, you could put temporary buttons in your app to run these code snippets without messing with the JavaScript console.

  • If things really get messed up, you may need to terminate and restart the app. On iOS 7, this is done by double-tapping on the Home button, then swiping upward on the app's mini-screenshot (not the icon). On Android, you can do this by Force Stopping from the Applications menu (YMMV on where this is located — it's under Settings on both of my Android devices).

Advanced Usage (Creating a Shell App)

It's possible to build a "universal shell app" that can dynamically load the code for any app (and in fact that's how the first release of Matryoshka worked). This is handy if you have multiple testers, or are testing on different platforms/simulators/emulators; all you have to do is install the shell app once, and multiple apps (even ones that didn't exist when the shell app was created) can be tested from the same shell app. There's no need to deploy new apps to multiple devices during development; once your testers have the shell app they can load test code from any URL. The downside is more time investment up front (however you only need to do it once unless you have to add a new plugin or the like).

  • Create a new project with a name of your choosing ("Matryoshka" is probably good, unless you have a strong preference for something else). Do any conjurations necessary to make it installable on your platforms (e.g. for iOS make sure to create it with something like cordova create Matryoshka com.contraterrene.Matryoshka Matryoshka (for cordova) or phonegap create Matryoshka --id com.contraterrene.Matryoshka --name Matryoshka (for Phonegap), where com.contraterrene.Matryoshka is replaced with the Bundle ID from an iOS development provisioning profile that you've created).
  • Change into the folder for the new project.
  • Add all the plugins to the Matryoshka project. Yes, all of them. All the standard plugins and all the third-party plugins that you normally use. Matryoshka can dynamically update your web code, but not compiled platform-specific code like plugins. If you need (or might need) a plugin, add it now (of course you can always rebuild/reinstall the Matryosha app later if you forget something or need something new, but the goal here is to minimize the number of rebuild/redeploy cycles). For your convenience, here is a list of commands to install all the standard plugins.

cordova plugin add org.apache.cordova.device-motion

cordova plugin add org.apache.cordova.camera

cordova plugin add org.apache.cordova.media-capture

cordova plugin add org.apache.cordova.device-orientation

cordova plugin add org.apache.cordova.network-information

cordova plugin add org.apache.cordova.contacts

cordova plugin add org.apache.cordova.device

cordova plugin add org.apache.cordova.battery-status

cordova plugin add org.apache.cordova.file

cordova plugin add org.apache.cordova.geolocation

cordova plugin add org.apache.cordova.globalization

cordova plugin add org.apache.cordova.inappbrowser

cordova plugin add org.apache.cordova.media

cordova plugin add org.apache.cordova.dialogs

cordova plugin add org.apache.cordova.vibration

cordova plugin add org.apache.cordova.splashscreen

  • If you're going to be using the local file system, you might also want to install: cordova plugin add https://github.com/tonyhursh/gapfile.git

  • This is my GapFile plugin. It greatly simplifies the task of using the local file system.

  • Add the code and resources from the shell-app folder in the repo to the top-level www folder in your Matryoshka project (overwriting the generated index.html, js folder, etc.).
  • Make sure the whitelist is set so the Matryoshka app can load code from your development machine's IP address. On Android and iOS, this is done by setting the access origin key in the top-level www/config.xml file. In this specific case, I recommend setting it to just an asterisk. This will allow the Matryoshka app to load code from anywhere. This isn't a good policy in general, but in this case it's going to be your app running your code on your device, and it avoids problems if your server IP address changes (due to DHCP or whatever -- the goal here is to only build and install the shell app once), or if you want the flexibility of loading the code from different locations (for instance, you might want to serve the code from an internal Matryoshka server at the early stages, then deploy it to public web server for beta testing). The key should look like this afterward:

<access origin="*" />

  • Some of the other platforms require handling the whitelist in a slightly different way. Check the Domain Whitelist Guide for details.
  • While you're in the config.xml, make any other changes needed (e.g., your name and contact info).
  • Add your build platforms.

cordova platform add ios

cordova platform add android

  • Build and install the app for all the devices and simulators/emulators that you normally use.
  • Launch the app and connect to any Matryoshka server (or, for that matter, any standard web server that can serve up the code -- you could also use this for distributing apps to remote beta testers without requiring a new app install. Just use the host name for the server (no http://) and set the port to 80).

Adding a New Platform

If you want to try it with a platform that isn't currently supported, load up lib/matryoshka.js and add entries for the proper folders for your platform in buildfolders and assetfolders. The entry in buildfolders is likely to be 'platforms/[platformname]', but the entry for assetfolders may vary. This is the folder where the generated www folder for your platform should be placed. For example, on iOS, the app's www folder is located at /platforms/ios, while for Android it's /platforms/android/assets. I have no idea for the others; firefoxos seems to follow the same pattern as iOS, but I have no way of actually testing it. Pull requests welcome on this.

Once you've added the correct folders, reinstall the matryoshka node module with npm install -g and restart any matryoshka servers you might have running.

Credits

Matryoshkya was written by Tony Hursh. Of course, major credit must go to the fine folks behind node (and all the node modules) and Cordova/PhoneGap itself. The matryoshka doll image is due to user 'jimbongo' at openclipart.org.

Contact

I can be reached at tony.hursh -- AT -- gmail.com. However, please don't email bug reports or feature requests. Use the issue tracker instead. I get a lot of email. If I can't deal with your issue at the exact moment I get the mail, there's a good chance it will sink into the uncharted depths of my inbox, never to be seen again. :-)

If you want want to hire me to do something, have a look at my profile on the PhoneGap developer portal. I'm currently available for remote contract work.

npm loves you