Most of the system in testing prior to use

On Tuesday night, Insanity Radio did a hugely successful outside broadcast from our student’s union event night. What we did was actually revolutionary in terms of the SU’s radio, and certainly isn’t something I’ve heard of another student station doing. The result was fantastic audio, a great sound overall, and a fun night. And near-zero latency. Here’s how it worked.

We used a fairly simple setup in principle – we planned on having a wireless link to get our OB computer onto the campus-wide wireless network, then using NetJACK to send audio from the JACK server on our OB computer across the network to another JACK server located in the studio. From there, we used alsa_out to get audio out to the sound card, and from there went into the studio desk.

However, the clever bit is this- we also had JACK managing the audio for the venue as a whole. We had audio going straight out of the OB computer into a balancing line-isolator (a Sonifex RB-BL2) and from there into the venue’s PA system. JACK is so low-latency that doing this is actually fairly simple, and while the DJs noticed the latency with their monitors a bit, they were fine with headphones and didn’t complain about it at all. For comparison, we had people using microphones to address the crowd and they didn’t notice a thing. NetJACK using CELT over wifi works fine – we used a 32kbps bitrate and used 3-packet redundancy, with a mono feed. We actually ended up managing to use a wired link and thus could probably have gone further but we didn’t test wired so didn’t want to risk an unreliable link given that the low-bitrate feed was actually very listenable. We ended the 6 hour long outside broadcast with a mere 20 netxruns, none audible.

This is all well and good but you might wonder why. Well, here’s the why of it- we essentially at this point have a JACK server with the physical and network outputs on the output side, and the inputs (actually the sum of a Behringer Xenyx desk put through a line-isolator and taken in on line in) from the DJs and microphones. But we’re a radio station! We want jingles, and occasionally we want to run our own music. So let’s do that!

Set up in all it's slightly-messy glory

We sadly don’t use Rivendell as our main playout system yet, but that didn’t stop me from pushing it into service here. We used Rivendell for jingles and music, with a master log set up with enough music to do the show if a DJ didn’t turn up, plus a sound panel loaded with jingles we’d had made for the event. Rivendell can be set up to use JACK so we could handle this easily. We then had a software mixer – jack-mixer – with the desk and rivendell as two inputs. The output of this went into jack-rack with a fast lookahead limiter to stop nasty transients going out. And to make this easy to monitor we had 6 instances of JKMeter running – one for prefade and one for postfade of both rivendell and the desk, and another pair before and after the limiter. This makes for an excellent view alongside RDAirPlay, and allows rapid debugging of any problems as well as good level monitoring.

Of course, being able to see what’s going on is nice, but nothing’s as good as having a proper hardware interface for mixing. So we threw a £30 Behringer BCF2000 into the mix, plus a2jmidid to bridge the ALSA and JACK midi ports. This let us put hardware faders on software controls, for both jack-mixer and jack-rack. When doing fast, smooth transitions this is what really helps to make the system usable.

But to what end, you might wonder? So did most people, till the DJ finished a song, only to have it smoothly picked up with a “You’re listening to Insanity Radio” jingle, swiftly followed by “Coming up next: Your 2011-2012 President of the Student’s Union” to be met with massive cheers from the crowd… and suddenly in comes the announcer, loud and clear on the mic.

Ideally we’d have run this with a good external soundcard with plenty of solid I/O on balanced connectors instead of using a desk to submix to a stereo feed then putting that into a balancer; that would have let us put every single channel on it’s own jack-mixer input and apply compressors to the microphones, for example. Still, nice to know there’s room for improvement! The event was a success, overall, though we had a few interruptions from our silence detector that marred it somewhat. And we managed it all on a budget of £0, using only what we had lying around- so that’s a win!

NetJACK is very promising and as I mentioned in #rivendell and #london-hack-space on freenode, there’s a lot of potential here – particularly using the bidirectional and low-latency nature of NetJACK you could do talkback between OB and the studio, you could fairly simply set up remote monitoring of feeds at the studio, and so on.

If you want to listen to some of what we did, have a listen – the audio on that page is largely unmodified, aside from the inclusion of the first jingle and removal of all the DJ music (for licensing reasons). All the overlaid jingles and transitions are as they went to air – which is why we messed up the recording a bit, as we’d failed to disable the silence detector in the chaos after the people with the results didn’t turn up till an hour after schedule! Next time, record locally…

For the real geeks who want to do this themselves: We were using Ubuntu 11.04 Alpha (thoroughly tested beforehand) with the kxstudio PPA packages for jack, libcelt and friends. SSH was used to control the endpoint machine from the OB, and our wifi module of choice was the Ubiquiti Bullet M2HP with an 8dBi omni antenna.

Okay, so, Backchat was hugely interesting as a project. Eventually, I produced a set of graphs using the classifier that showed sentiment over time. These graphs aren’t too accurate but are fairly good at showing how things were going. However, after this I pretty much dropped the project. This was mainly due to exams cropping up and stealing my time away, but also because of how difficult it was to approach a sensible level of accuracy.

In my ‘final’ design I ended up using a bigram classifier. I added parsing of the tweets to pull out mentions of words, URLs and users, and then used this to generate my training sets, which improved things a lot. This gave me several thousand tweets for each training set, which worked okay. However, even with this classifier, which was doing a lot better than most others, my results weren’t very reliable on a tweet-by-tweet basis. Still, it wasn’t too shoddy, and the graphs on the right are fairly reliable I think in terms of general sentiment.

The AMQP-linked network of processors worked extremely well, and resulted in good throughput- I used two parsers, two classifiers and one classifier loader in the end; I was unable to achieve realtime performance due to network constraints. Sadly my ISP at home had decided that I’d used too much bandwidth and clamped me down to 128 kilobits a second. That said, thanks to the streaming API I did not (as far as I know, except for a few hundred to ratelimiting) lose any tweets, I just received them out of order and then reconstructed the correct order using the timestamps for each tweet. The machine I was using for this also pretty much went flat out on disk I/O and CPU usage, but was able to keep up- it’s a fairly old box, only a Pentium 4 with a couple of gigs of RAM.

In any case this was an interesting project and I’ll be open sourcing the data and source in the coming weeks if anyone wants to have a poke at it. While the debates are now gone and done, I’m sure people can come up with some great uses for sentiment analysis outside of UK politics.

We have got some things in the works for Dominion, and we hope you’ll find them useful; we’ve not had much time to work on EVE Metrics, and we’re being distracted by another project at the moment, but we’ll have more time to work on EVE Metrics in a few weeks time around Christmas. I’m still evaluating what we’ll spend our time on, though, and we’d like to get more feedback via the feedback button on the site- you can vote for other people’s suggestions, so please do so!

Other than that, not much to report. We’ll have Dominion items loaded into the site by release day so you can start using the site straight away with the new items. We’ve been so far very successful with some performance improvements on the site; this has mostly been tuning our database server and working on improving the performance of queries through better indexes, clustering indexes, and so on. Hopefully you’ll notice this in the form of improved page responsiveness and less ‘slow loading’ pages. Enjoy!