Stickmaker: Thimble on a thumb drive

tl;dr: Offline Webmaker tools are very blue skies, and fairly infeasible. Much less running off a thumb drive. However there are, very imperfect, ways to make it work.

[HIGHLY EXPERIMENTAL] HERE BE DRAGONS:
This post about surfacing my work, research, and findings, as well as some of my opinions on the possibility of something I keep hearing requests for. Offline Webmaker. Please take this post with the warning that this work is provided as is, no guarantees it’ll work for you.

crude visual representation of the idea

One of the many questions I’ve been hearing from the mentor community around the world is “How can I run Webmaker tools without an internet connection?”

The short answer is that its possible, but with a little pain. The long answer is something I’ve been exploring, as well as figuring out how to change that short answer, from pain, to pleasure.

Before I waste the time of those looking for a quick solution, I’d like to point out that I’ve so far failed to make it a pleasure, though I’ve made things a little easier for some.

People asking to be able to use the tools in an online context is great as it shows that we’ve definitely made something people want to use, and are comfortable using. However there are a large number of people running events in areas where internet connectivity is less than perfect, if non-existent. webmaker.org doesn’t provide much of a solution for these people. Although there are guides + kits on running offline webmaking activities.

These events range from rural areas, to schools (yes, this shocked me too) and large events where wifi is intermittent at best, due to heavy usage.

Being web-based, webmaker.org requires an internet connection, and these situations don’t provide. So how do we work around this problem?

One method is to set up a VM (Virtual Machine), and create a local network that has access to it. This way you can set up a completely offline version of webmaker.org (there are some caveats to this however).

This doesn’t work so well in schools though (due to user permissions on the machines), and requires a reasonably good knowledge of setting up a virtual machine, networking, and other non-typical-user friendly skills.

I wanted to see if there was a way to solve this problem, so I came up with a set of requirements based on some conversations with people like Tef from Code Club, and by thinking back to some of the ideas about the Webmaker USB sticks we got at the Athens train the trainer week.

My solution needed to:

1. Be easy enough to install that my mom could do it.
2. Needed to be portable enough that you could take it with you, and use on other computers without an internet connection.
3. Needed to work cross-platform.
4. Should work in schools.

Four seemingly simple problems, and one of which had mostly been taken care of by someone else.

“As I was drawing close to completion of my first iteration on the installer I popped my head into #webmaker (on IRC) and found out that someone had beaten me to writing an installer. They uploaded it to GitHub on Monday.”

William Duyck – Beaten to the finish.

Requirement one met, time to move onto the second… making it portable. I started by experimenting with simply installing Webmaker to my USB. This raised another problem. As amazingly easy as it is to use. The Webmaker Suite installer does quite a lot of I/O, and most USB sticks are quite slow at this.

My first install attempt took a little over an hour to complete. Not good.

I needed to do less I/O. So I set about scripting an installer that followed many of the same principles as what Pomax had created, but that only downloaded and setup what was needed to run Thimble.

Okay… install down to the time to make a coffee, and eat a scone. Not perfect but much more reasonable. This I could work with. Now onto making it portable.

It turns out that this is the hardest part. Webmaker requires a number of languages to work, most don’t come preinstalled on consumer devices, and I needed this to be simple to install… so avoid the user having to do any extra downloads. How about portable python, nodejs, mongodb, elasticsearch, etc…

Well they don’t all exists, and easticsearch refuses to play nice on a USB. However I did manage, with a lot of digging around the web, blood, sweat, and tears, get something that worked. Though now I needed to script an install for those as well, and I had to assume that node wasn’t installed on the user’s computer… time to fall back to something that would be.

Doing so meant changing my focus a little from the cross-platform love I wanted to lavish on this project. My knowledge of preinstalled, cross-platform languages isn’t good enough to go charging off into this world. So I defaulted to *nix systems as I know they all have shells, and almost all have bash.

After a lot of fiddling, and Stickmaker now exists, though in an extremely buggy state. I focused more on just getting it to install and run Thimble, rather than iron out bugs. Mostly to act as a proof (or disproof) of concept.

Stickmaker is my very rough attempt to address all three requirements I mentioned earlier, as proof of concept, and for the most part works. There are some pretty big bugs, however it answered a big question that I’ve had, and heard, for a while.

Can Thimble be run off a USB stick? The answer… not really, at least not in its present state (which is unlikely to change any time soon). There is a bigger question that we should be asking though. Should we try to make Thimble run off a USB?

In my opinion, the answer is kinda yes, kinda no. Stickmaker tries to run Thimble on the host machine. The reason: Most schools, and public networks don’t allow you to run VMs.

I believe that Stickmaker acts as a good proof that web/server-based tech, shouldn’t be jerry rigged into an offline application.

Is the dream of an offline Webmaker dead? Not my a long shot, however this is not the approach we should be taking. The better, more stable, and far more feasible solutions lay with VMs and potentially the RaspberryPi.

Future iterations of Stickmaker are going to focus on keeping those original four requirements, but tackling the problems of locked down systems from another attack vector.

The Future

Google Coder takes a very interesting approach to solving a very similar problem. They have a RaspberryPi based solution. This solves most of the issues that an offline Webmaker would face. Another approach that would work in a similar way is to simply have a VM with Webmaker installed, and connect to that. Both these solutions would meet all my requirements, if you’re able to change the networking settings for the machines the learners are using.

Earlier I said that VMs were a problem, and they still are. However they cause less of a problem that running multiple complied binaries for all the requirements Thimble currently has. My next move in this experiment is going to be to look at using the Webmaker Suite, and a virtual machine. How is this going to be portable though? Where does the USB fit into this new equation.

After a little digging around on the web I’ve discovered Portable-Virtualbox for Windows users, so unlike the current version of Stickmaker that is *nix only, the next version is looking to be Windows only. However as more machines are Windows (unfortunately), and due to the state of portable apps for *nix, this is going to be initial tradeoff for a more stable, more fully featured offline Webmaker.

A problem to still be addressed

One of the key draws for Webmaker is the ability to instantly share your work with others. Any current offline installation of Webmaker won’t have this functionality. To solve this we’ll have to look into data synchronisation for the source code, images, etc… This is stretch goal for this project though, it’s going to take a lot of time, and work. It might be possible to do this sensibly.

Current project status: Dormant

Right now I’m marking this project as fairly dormant. Despite having plans for it to continue on it’s a matter of weighing up priorities, and regrettably I don’t see this being a major focus for the time being.

What I will be doing however is writing another article in the next couple of weeks that will go through the process of installing Webmaker Suite into a VM.