RadioDNS and RadioVIS – Getting Started

Okay, this is gonna be a weighty one. But it’s an interesting one, for me at least, and I hope for you too.

If you’ve got a smartphone, or a modern DAB/FM radio, chances are it can connect to the internet on 3G or wifi. This lets your smartphone also check your emails, and your modern radio listen to internet radio streams. But what it doesn’t let you do is find out more about the station you’re listening to. At least, until now. (Read that in a dramatic voice and let the tension sink in a little. Done? Then let us proceed!)

Continue reading RadioDNS and RadioVIS – Getting Started

Perceptual image and audio deduplication

Okay, two months without a post, won’t happen again…

So, lately I’ve been moving out from the broadcast area and getting back into webapp development, but some of the things I’ve been working on touch quite heavily on deduplication, of images and music. This is quite an interesting topic, so let’s have a look at what we can do now.

Doing exact deduplication – stopping someone uploading the same file twice to a website – is pretty easy. You just hash the uploaded file (or de-encapsulate the data and hash that if you want to be a little more resilient) with something like SHA256 or SHA512. It’s fast, effective and easy. Lookups are as fast as your RDBMS is. This works with images, audio, videos, you name it.

What’s much harder is doing perceptual deduplication, or content deduplication. If I upload two files which are the same except one’s a PNG and one’s a JPEG, I want to be able to say “Hang on, you’ve already uploaded that!” when you upload the second file. Similarly, what if we resize an image? We want something resistant to that sort of attack. Continue reading Perceptual image and audio deduplication

Django – a flying visit

So rather jokingly the other day I asked where the London Hackspace jobs board was, and 20 minutes later I’d decided to make it. I needed a little project as a break from the main commercial app I’m working on, which for the last week has involved staring at some rather involved postfix/mailer code. As I usually do, I grabbed Rails, but paused to update to Rails 3.1; I figured if I was going to start a new app, given Rails 3.1 is at rc6, I should probably go with that, knowing that a lot of the asset code would need rewriting shortly thereafter otherwise.

However, I quickly hit problems, probably just with interactions between my authentication library of choice (devise) and 3.1. Notably I got a complete appserver lockup on user registration, which sort of limited what I could do. Getting frustrated at how much had been fiddled with in 3.1 for seemingly no reason, I decided to take a look at how Pythonistas get their web fix.

Continue reading Django – a flying visit