CrowdSync - All Your Phones Are Belong To Us

Saturday, December 24, 2011 9:22 AM by nairdo

An Idea

A few months ago Kim Vehon, from our Worship team came, into our office to share a video showing a bunch of phones hanging from wires flashing on and off. It's pretty cool. She wondered if we could do something like that during our Christmas services at Central. Building a "fat-app" to control a bunch of phones wired together did not seem like a challenge nor interesting enough to be worth pursuing.

A Better Idea

But then we started thinking. What if we tried to control the congregation's phones and what if we used only their existing 3G network connection?  And, since hardly anyone would want to install an app from the market/store, what if it only used their phone's web browser?  We also thought it should be built in such a way as to reduce the dependency on the network -- in other words the phone should get what it needs from the server and then be able to loose network connection without impact to the performance. Game on. I agree with Jason (tweet), this might be one of the funnest things I've worked on in a long time. Jason and I call our software CrowdSync...

Pre-event countdown on CrowdSync 

Ultimately we also needed to generate a lights time coded "track" from a Christmas song (midi file) with each note being assigned to 1-4 colors.  A person's phone would receive the track, be randomly assigned one of the four colors, and start playing the track in sync with the band.  Sounded simple enough. Thankfully I found the C# MIDI Toolkit code from Leslie Sanford that reads midi files (thank you Leslie!) which I was able to modify in order to extract and generate our time coded light track as JSON data which looks roughly like this:

startTime: 123578916,
endTime: 12345667,
ticks: [
{ notes: ["a"], time: 27, duration: 900 },
{ notes: ["b"], time: 982, duration: 900 },
{ notes: ["c"], time: 1940, duration: 900 },
{ notes: ["d"], time: 2908, duration: 900 },


The Time Problem

Pretty quickly we eliminated web sockets and other client-server signaling technology for a variety of reasons including lack of consistent device support, chattiness, lag, and timing control.  We wanted to constrain ourselves in order to force us to think differently about certain problems -- such as the "how do we get all of these devices/clients to start at exactly the same time?" problem.

Initially we were thinking we could rely on the time from the phone's operating system.  You might think two 3G Verizon phones would have the same time, right?  Wrong. Very wrong. We saw phones that were off anywhere from several seconds to a few minutes. (Who knows how or where each phone is getting its time from? It doesn't appear they're using a common Network Time Protocol (NTP) server.)

After some medium scale client tests and experimentation, we came up with the following approach:  Each client asks our server for the current time, calculates the delta (from it's local time), and repeats this about 20 times over the course of about 20 seconds.  The error introduced because of network latency is reduced to a minimum because we use only the 'smallest' delta from our samples.

CrowdSync Time Sync Diagram

Once we've got the correct delta we calculate and shift the entire track's note times to represent the exact localized time each particular note should play (light up) on that device.  Then we basically wait until the note's time occurs and set some CSS to play that particular note's light.

Other Stuff

Since not all browsers can handle HTML5, we had to keep things simple and used basic HTML, CSS and JavaScript (CoffeeScript).  Other problems worth mentioning are: cell phone auto dimming, screen locking, and mysterious time lost as a result of certain mobile phone browser interruptions - such as screen lock and wake up.  (On my Android phone, a screen lock and re-open would cause a simple JavaScript clock to become out of sync with correct time.  It smells like a timing bug with the OS - but what do I know.)

Forward to November when it was decided that our system should also power the worship center's IMAG side screens with something cool, show an event count down timer, and the drive the band's click track.  That ended up being a real blessing because we got to work on some really fun stuff.  After more experimentation for the "something cool" part, and more ramping-it-up ™ with statements like "what if we could ...", we ended up programmatically building a Christmas Tree as colored circles on the HTML5 canvas to match our church's Luminous graphic.

Central Christian Church's Christmas Luminous Graphic

We somewhat randomly plot circles in the shape of a Christmas tree using a little Math.sin() trick to get the curve we wanted.  (Remember that stuff you learned in high school -- it really does come in handy!)  The same JSON encoded light track data would be used to control the four sets of lights on the tree.  We kept the same code base and ended up with some configuration settings to control whether the simple HTML/CSS (cell phones) or HTML5 canvas w/tree was being used (side screens).

Ultimately, the midi file click track turned into a full blown amazing score and arrangement of Carol of the Bells by Adrian Darsee in mp3 format.  There was more fun getting the audio track to play which perhaps Jason will cover in his blog.

We also created a small admin panel as an Arena module so that the A/V tech guys at each campus could store the event start time for that service once they knew exactly when they wanted it to start.

CrowdSync Admin Panel 

Time Shifting Gotchas

Late in the game we discovered something unfortunate when we put it all together with all the other equipment (QLab, the worship center's A/V systems and side screens, a full 30 minute pre-service countdown, etc.).  Not all seconds are created equal. I should have realized this, but somehow it failed to register.  Due to NTP being used on the various systems, the QLab mac, the mac running Chrome for the IMAG, and our server would slowly drift by a 1 to .5 seconds during the 40 minute countdown.  That's a big problem if you're trying to achieve millisecond synchronization.  We turned off NTP as a work-around but realize it's something we want to address in future versions of CrowdSync.  We're also going to implement all the server side stuff in Node.js.


The goal was for each person to "be the light" and "let your light shine" as a literal and symbolic expression during the worship to our King. Overall it was a success for the whole opening worship experience but when the congregation erupted into loud applause at the end I knew they enjoyed the experience too.  I hope we have some of it captured on video.

click for large image

We'll post the latest alpha CrowdSync source up on Github shortly.


    No Comments

New Comments to this post are disabled

Powered by Community Server, by Telligent Systems