So, you might figure that’s a problem. You’d be, partly, right. Video involves a lot more data, loads more numbers to crunch as a result, more bandwidth, and so on. Not to mention the relevant methods of capture are immensely more expensive to implement than the equivalent-quality audio. I’d like, though, to highlight a few nice things for open source video and production.

For any broadcast worthy of the description, we want to be able to do this:

• Work with multiple camera feeds both close to and further from the computer
• Fade between cameras
• Overlay static graphics
• Overlay dynamic text
• Add decent-quality audio
• Stream to as wide an audience as possible

So let’s start with cameras. Cameras are expensive. Buying handicam type consumer-grade things won’t cut the mustard as far as I am aware. I don’t know enough to speak on the subject of HDMI capture (which things like the Blackmagic Studio can do, purportedly to Linux), but that’s the only output format those cameras present. Firewire/IEEE1394 remains a popular format of direct capture in Linux. In broadcast, SDI and HD-SDI are now the main format for video/audio transport, and affordable (I say affordable- low-end professional cameras, ~£3,500 on eBay) cameras support it as a standard and again, Blackmagic cards and other capture devices are available with Linux support. HD/SD-SDI are in my mind the ideal format to be working with because as a transport protocol it is capable of long cable runs- meaning your cameras can roam around or be placed where it most makes sense from a camerawork perspective. You pay extra, but compared to any other capture format it makes loads of sense. PC capture cards are not particularly expensive – a few hundred pounds gets you a card with SDI I/O and balanced audio I/O and analogue video I/O thrown in, too. Stick that in a PCI-E capable box and you’re good!

This brings me to topic two – CPU power. Specify the fastest box you can. Right now I’m testing an encoding-and-compositing-on-one-box setup, and with no cameras connected I’m at 80% CPU usage on a 2.8Ghz Core 2 Duo. This stuff needs brains. Intel i7 processors are the way to go right now, or high to mid end Xeons. RAM, too- at least 4 gigabytes, more wouldn’t hurt at all, especially if you use videos in your composites. Aim to keep all your data in RAM, or use an SSD. Multiple computers with gigabit networking will let you split loads across machines, as I’ll talk about in a bit.

Now so far we’ve spent a lot on cameras and computers. Fortunately, that’s all you need (audio equipment aside). Now we get everything in as a source in Linux, we can do the fun stuff- compositing our graphics and videos and text on top of the cameras, switching between them, and then broadcasting.

Composition we have a lot of choice in. We can use freej, which I am very interested in but have been unable to get working properly on Ubuntu so far. Or, we can use WebcamStudio. It’s a fairly flexible package that lets you do compositing of lots of source types, arranging things into layouts. So we might have an intro layout that has placeholder graphics describing the stream for while we’re setting up the cameras. Then we might have a couple of layouts that have our camera feeds on them. For a test setup I wrote a small Python script to get the last few tweets from our on-campus media, then told WCS to run that command and put the output on the screen as a rolling text item. Now we’ve got our video cameras with contributions from news teams overlaid. We’re starting to look professional!

Audio is a piece of cake. All we need to do is capture it and encode it with the video. The former is too complex to detail here but needless to say you end up bringing in your final mixdown of audio into a soundcard input which you can then bring into my next chunk of software for the post – Flumotion.

Flumotion is really neat. It lets you build GStreamer chains very easily to do broadcasting, and has a lot of added functionality on top of that like test chains, and crucially the ability to split tasks across computers fairly seamlessly. You can make your rig a network of 3 machines, put video compositing on one, capture on another, and combining and encoding of those streams can be delegated to another machine- for instance. There’s loads of ways to work with this. Another way I’ve thought about doing things would be to encode using Driac or another high-res HD codec the feed from each camera on a local Icecast server or HTTP server, then pull that compressed feed down into the compositor, then do the lossy for-broadcast encoding prior to distribution. This has the benefit that you can also write the Driac encoded data direct to disk, giving you clean-feeds of each camera input for later editing and production use. Perfect!

So, how about streaming? Well, turns out this is pretty simple. We use Flumotion to generate a Theora/Vorbis encoded Ogg container, throw it at Icecast2 using the shout2 module, and then use HTML5 video tags with a fallback on the Cortado Java player to embed it into pages. This works without any plugin requirements on most modern browsers, and will seamlessly fall back to Java if HTML5/Theora/Vorbis support is lacking. Perfect! Icecast2 gives us failover and relay options for expansion and redundancy for free, and doesn’t need much in the way of resources. The fact that we can push to this means that we can site this outside any networks we might be using, and we don’t have to worry about firewalls at the venue so much as long as we can send out to our remote host.

The main issue here is in making everything realtime. With video, that’s expensive. I reckon for a two-camera setup including buying computers and ancillary equipment we’re talking somewhere around £10,000 to £15,000 setup costs. But that’s actually pretty good – considering what it gets you. You’re now a real, bona-fide online TV station capable of real time coverage- something very few places can boast. For student media that’s a massive boon to both the people running the media and students involved with it, as they can now learn what amounts to the most stressful and complex aspects of TV broadcast production in a fairly safe setting, and for students consuming the media it means much better coverage, more engaging events and more fun in general.

This is a very exciting time for student media, where there is a lot of room for experimentation and improvement to try and provide the absolute best in a safe, controlled environment. Of course, student media means student budgets, but TV is rapidly becoming affordable, and is certainly within the grasp of any SU in the country who wish to go down this path.

Our electoral system is essentially a popularity contest. Manifestos, campaigning and student outreach have very little impact on the results. Many positions are uncontested and whoever runs wins by virtue of being the candidate who is running; people assume that this means that they care about the position enough to run, and that’s enough for them.

I don’t think this is a good way to run elections, and it’s not something that should be encouraged. But it’s something that can be very easily fixed, or at least I think so.

Essentially- why not make the candidates unlisted? Let’s say there’s a candidate running for a position- let’s call them John Smith. They’ve been out hard campaigning throughout the week, and you’ve just gotten canvassed by this candidate on your way to the library. You sit down and decide you should probably vote, so you grab your laptop or a public terminal and load the website up, log in… and vote.

But here’s the crucial bit. We use alternative transferable vote for most of our elections, which requires you to put a number by each candidate, ranking them in priority. There are also two meta-options: re-open nominations (RON), and no further preferences (NFP). The way this is presented to you is as a list of all candidates, followed by the NFP/RON options, and you just type in a number by each option in descending order of preference.

The lazy voter just sticks a 1 by the only candidate and hits vote. If we’re lucky (in terms of improving democracy) they vote for RON as their second preference, or maybe they even vote RON for first preference. In terms of improving democracy by encouraging people to go out and canvass for their position (which is more likely to discourage people running for the sake of running/sticking it on their CV and encourage people who care enough about the position that they’re willing to go out and get people voting for them), the ideal option is that people vote for people they’re aware of through canvassing (online or in person) and vote RON for any positions they’ve not had any contact from people over.

So how can we encourage this over the default impulse to vote for whoever’s running? Simple- we remove the candidate’s names from the ballot. “What on earth is he on?”, you may well be thinking. Well, here’s the thing- if we remove people’s names and require people to actually enter the name of a candidate they want to vote for, people cannot just vote for people because they’re running; they won’t know who is running, if anyone. Of course, they can go find out who is running by looking at the candidate and manifesto listings. But this encourages people to find out more about who they’re voting for, and discourages lazy voting- both good things for democracy. Canvassing gets your name into people’s heads (and, through flyers etc, into people’s hands, making the process of voting even easier).

And because our votes are 100% online, the process of typing in someone’s name can be made easier – if we want to vote for John Smith, we type J and are immediately autocompleted to John Smith. This entry is now an entry we can put a preference number by (or in perhaps a better UI, we could have draggable lists to make specifying preference easier). This stops people with complex names from being disadvantaged against other candidates in elections.

So with this system, the view that would greet you on any election position vote page would be “Hello, this is the election for the position of X. Type a candidate’s name in below, re-order the entries so the option you most want to win is at the top and no further preferences is above any entries you don’t want to cast a vote for, and press vote”. Voters would either just click vote to vote for re-open-nominations, or could go look at the manifestos etc and see who’s running and why, type in the people they want to vote for, drag the markers around, click vote, confirm their vote after seeing a screen detailing their vote, and then they’re done. Simple as that.

I figure I might do a quick proof of concept system that demonstrates what I’d imagine a voting UI to look like that used this system. I don’t see any major downsides to this system, but I’d love some feedback if I’m missing something obvious. And if people think it’s a good idea, who knows- maybe we’ll get this to a general meeting and see about making it the standard, or at least get it trialled.

We’ve historically relied on a RRDtool database updated with Munin and an icecast plugin. While this served us well in the single-server days, we recently added a relay server to help listeners with network problems connecting to our JANET-hosted box. Now we have to handle summing two statistics sources and compensating for the added relay connections. At this point I weighed up writing a Munin plugin versus rolling my own and decided to try whipping a solution up using Redis.

Redis is mind-blowingly fast, and exceptionally flexible to boot. We’re already planning to use it for caching, so it makes sense to use it for statistics storage. So, the goal here was:

• Fast inserts
• Very fast retrieval of arbitrary time ranges

Simple goals. I did some digging around and there’s a lot of different approaches to storing time-series data in Redis. The scheme I used in the end uses sorted sets to store data, indexed by the timestamp, and with the timestamp included in the data to allow for duplicate values. The sorted sets are partitioned by day; for a regular update interval we’re looking at ~8,000 points per day.

Updates take a bit longer because Redis has to do sorting on insert, but that’s actually scalable – O(log(N)) – and the losses there are regained tenfold when doing retrieval. We keep the datasets small by the partitioning, meaning that N in O(log(N)+M) is kept low- M is dependent on the query. I have yet to benchmark this all because I have yet to notice the extra performance hit on the pages- it’s snappy in the extreme. We’ll have to wait and see on how well it scales up, of course.

We do get a bit of overhead because we have to split the timestamp and value apart before we can use either. But that’s pretty trivial overall. We’re also putting the statistics into a GSL vector using rb-gsl-ng, which means subsequent stats operations on the dataset are fast; we can generate a page with 80-odd sparklines and statistics summaries generated from 80 different queries without adding more than 50ms to the page load time, which is completely acceptable. Overall, this is working very well indeed. I’d love to see Redis add more innate support for timeseries data with duplicate values, but for the time being this workaround is doing okay.

As an addendum to this post, redistat is another tool worth mentioning, partly for the similar application but also for the alternative method of storing key/value data in Redis, albeit in a manner more geared towards counters rather than time-series statistics data. Worth having a look at if you’re interested, in any case.