Sustainable low-budget infrastructure

June 14, 2012

This month I finish my university career and along with this move I sadly will stop working at Insanity Radio, the student (now community) radio station I’ve been running tech at for about 3 years now. Needless to say, I’m going to miss the place and the people, and the challenges that came with that environment.

Specifically: No budget (a total of £3,000 income annually, compared to the average income of £75,000 for most Community Radio stations according to Ofcom). No paid full-time staff. And a desire for 100% availability regardless.

Over the years the systems at Insanity have evolved and grown – they started out as a single computer for playout, a single encoder and streaming server running Windows Media Encoder with about 50% availability best-case, and we’re now deploying high availability clusters for streaming and encoding, have very few single points of failure, with a total of 21 computers. Back in 2009 we had significant amounts of dead air – outside of a processor failure we’ve had very few incidents since 2011.

Building systems for reliability on no money is a tricky thing to do, and it’s even harder when the people maintaining the infrastructure change on a potentially annual basis. This post is basically a quick encapsulation of some of the most important things to focus on to make such a situation work – not just from a technological perspective but from a human perspective too. read more …

Tags

Perceptual image and audio deduplication

January 7, 2012

Okay, two months without a post, won’t happen again…

So, lately I’ve been moving out from the broadcast area and getting back into webapp development, but some of the things I’ve been working on touch quite heavily on deduplication, of images and music. This is quite an interesting topic, so let’s have a look at what we can do now.

Doing exact deduplication – stopping someone uploading the same file twice to a website – is pretty easy. You just hash the uploaded file (or de-encapsulate the data and hash that if you want to be a little more resilient) with something like SHA256 or SHA512. It’s fast, effective and easy. Lookups are as fast as your RDBMS is. This works with images, audio, videos, you name it.

What’s much harder is doing perceptual deduplication, or content deduplication. If I upload two files which are the same except one’s a PNG and one’s a JPEG, I want to be able to say “Hang on, you’ve already uploaded that!” when you upload the second file. Similarly, what if we resize an image? We want something resistant to that sort of attack. read more …

IRIS – The Interchangeable Radio Ingest System

October 13, 2011

Well, wow. After nearly forgetting to actually submit it and only writing the entry a few hours before the deadline, it turns out that the system I made while at Insanity Radio 1287AM has been nominated for the Best Technical Achievement award at the Student Radio Awards. So, I figured it would be worth actually writing up a little bit about what it is and what it does. And why you can use it, too, if you’re involved with a student radio station.

IRIS was written to replace MACIS, a system I bodged up out of necessity. At Insanity, we had a computer failure weeks before we went on air at the start of the first term, and lost all the data- including the entire playout system. Lessons have been learned (I made sure we replaced that machine with a box that had RAID, for starters) since, but we had the unenviable challenge of repopulating a student radio playout system from scratch with little to no staff. Enter MACIS!

read more …

Tags