Real Time kernels and audio on the Raspberry Pi

A year or two ago I posted about how you could send audio over the internet with Raspberry Pis using the OpenOB project. Since then the OpenOB project has taken off, with lots of contributions from the community and lots of improvement as a result.

What hasn’t aged well is the Pi. A firmware update to fix some keyboard compatibility issues caused some serious issues with audio over IP, as both the Ethernet controller and USB sound card shared a USB bus which couldn’t operate quickly enough to handle the precise timing demands. Fortunately, the Wolfson Audio Board has come along to save the day – and it’s certainly promising.

Sadly, the kernel support for the Wolfson device isn’t in the mainline kernel yet, so that means using their custom OS image, or building our own kernel. On top of that we’d quite like a preemptible kernel to allow us to get lower latencies in userspace. This is crucial for reliable jitter-free low-latency audio, but because it’s quite niche this also means we need to apply some non-mainline patches to the Raspberry Pi kernel. The Wolfson drivers will eventually make it to the Pi kernel by default, and hopefully someone familiar with packaging for Debian/Raspbian can contribute a package to provide a real-time patched version of the kernel; but in the meantime we need to get our hands dirty. Here’s how to get a stock Raspbian image turned into a low-latency audio capable flavour, broken down and explained a bit.

A quick note about the assumptions

What I’m describing below certainly helps according to benchmarks. In some multi-day-long tests I’ve experienced a 15mS average maximum on a non-rt kernel, whereas on a -rt kernel that drops to 110uS (that’s microseconds, not milliseconds), using the cyclictest program from the rt-tests kernel repository. Making that improvement translate to better audio performance is not automatic – there is additional work required to configure ALSA and any userspace programs you’re using.

Some other assumptions:

  • We’re building for the 3.10.25 kernel. If your Pi ships with a more up to date kernel (as it will if this post is older than a few months hopefully), you’ll need to locate alternative patches (the -rt patchsets are regularly updated, but the Wolfson drivers may not be)
  • We’ll build the kernel on the Pi – so you will need to leave this Pi switched on with a terminal on it for around 12 hours while it builds; you can also run the build in a tool like screen or tmux that will let you detatch/reattach your session while the build runs, so your PC/terminal doesn’t have to be on all the time
  • Your Pi is a model B with a Wolfson audio board attached. If you omit the Wolfson audio patches from the kernel you should have a working realtime system that can only use the onboard audio or USB devices. This may not be an issue but it’s no harm to build the Wolfson modules in any case!

With that disclaimer out of the way, let’s get cracking. I’m assuming you’re starting from a base Raspbian installation, and have unfettered internet access – if you’re behind a proxy then make sure to configure that using the SOCKS_PROXY/HTTP_PROXY/GIT_PROXY_COMMAND environment variables and a wrapper script for git.

Getting dependencies, patchsets and kernel sources

First up, let’s update the Pi and install some dependencies.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install bc libncurses5-dev

There’s two patchsets we’re going to apply – the Wolfson kernel patches, and the -rt patches, so let’s grab those first.

wget http://downloads.element14.com/wolfson/wolfson_drivers.tar.gz
tar zxf wolfson_drivers.tar.gz
wget https://www.kernel.org/pub/linux/kernel/projects/rt/3.10/older/patch-3.10.25-rt23.patch.gz
gunzip patch*

Now we’ve got those downloaded and extracted, look around a bit – you’ll note the Wolfson drivers come with a whole boatload of .sh scripts named things like Record_from_lineIn.sh. These are simple wrappers around ALSA’s configuration tools – if you run alsamixer after we’re done you’ll see why they’ve included them, as the ALSA interface their codec provides is very complicated compared to most codecs. You’ll also find a bunch of .patch files in a subfolder, which are what we’ll apply to the kernel sources next. First, though, we need to check out the sources.

mkdir kernel_sources
cd kernel_sources 
# This will take a while and download plenty of data. Go get a cup of tea! 
git init 
git fetch git://github.com/raspberrypi/linux.git rpi-3.10.y:refs/remotes/origin/rpi-3.10.y 
git checkout rpi-3.10.y 
# Go to the specific commit the patches are for 
git reset --hard c43739885d512c92c0aa443b5895b96df5141da0 
# Tell git who we are for our changes 
git config --global user.name "Your name"
git config --global user.email "your email in here"

With all of that out of the way we should now have a copy of the Linux kernel source tree suitable for the Raspberry Pi ready to build. This is where we start to configure things.

# If you're prompted by the following to skip any empty patches, do so as directed
# Skip this line for RT only (no Wolfson drivers)
git am -3 /home/pi/rpi_wlf_3.10_beta/*
cat /home/pi/patch-3.10.25-rt23.patch | patch -p1
# Copy in the default config for the Wolfson drivers
# If you're going for just RT, do 'make bcmrpi_defconfig' instead of this next line
cp arch/arm/configs/rpi_wolfson_sound_pi_defconfig .config
# Start the configuration menu
make menuconfig

Configuring and building your kernel

Your Pi should have a think and then give you a menu. Navigating in here is a bit fiddly so have a play around to get your bearings – if you mess up, just hit escape until prompted to save, don’t do so, and run make menuconfig again. Worst case if you save the config by accident, run the cp line above again to copy the default config back in for a clean slate.

You may be tempted to tinker with all the options in here – and to an extent, do so! If you can see things you know you’ll never need you can remove them. However, be aware that you can break things easily, and there’s not a huge upside to removing most things here.

What you do need to do, though, is find these options and set them as follows:

  • Kernel config -> CPU power management -> CPU frequency scaling -> Default governor -> performance
  • Kernel config -> CPU power management -> CPU frequency scaling: Disable the non-performance governor modules, we don’t want them!
  • Kernel config -> Kernel Features > Preemption Model -> Fully preemptible kernel (RT)

That’s it! Easy as that. What we’ve done there is forced the CPU to run at maximum power at all times (note that this is not something you should do in combination with a 1000MHz overclock – 950MHz seems to be stable, but lower frequencies are better if you can live with them), and enabled a fully preemptible kernel. Now we can build it and install the modules and the kernel:

make
make modules
sudo make modules_install
sudo cp arch/arm/boot/Image /boot/kernel-rt.img

Go make a cup of tea (or a few) while the build runs. We copy the kernel to a new location in the boot partition rather than overwriting the existing one – this makes it easy to switch back, and OS upgrades to the kernel package won’t wipe your changes out. What we need to do now, though, is tell the Pi about this kernel.

Configuring the bootloader

Edit /boot/config.txt with your editor of choice and add the following lines:

disable_splash=1
force_turbo=1
kernel=kernel-rt.img

We’re forcing on turbo mode and removing the splash screen for easier debugging, and telling the Pi where to look for its kernel. If you break anything in your kernel, removing that line will get you booted up again!

We also need to modify /boot/cmdline.txt to work around a bug which would mean the kernel couldn’t mount the SD card root partition. We also force the USB device speed to 1.1, as this appears to fix some stability issues; omit the second part of this if you feel lucky. Simply add this to the start of the line of text in cmdline.txt:

sdhci_bcm2708.enable_llm=0 dwc_otg.speed=1

Once we’ve done that, you should be good to go. Simply issue a ‘sudo reboot’ and wait for your Pi to come back to life! If it doesn’t, edit the config.txt to point at your old kernel.img and go from there; you’ll need to hook up a HDMI screen or serial console to debug it, though.

To check everything is working, issue a quick uname -a:

pi@openob-pi ~ $ uname -a
Linux openob-pi 3.10.25-rt23-openob+ #3 PREEMPT RT Sat Apr 5 11:04:35 UTC 2014 armv6l GNU/Linux

Note in this build I’ve added a -openob version string to my kernel configuration – but the crucial thing is the PREEMPT RT line and the -rt23 version suffix. If you’re also building the Wolfson Audio stuff and have the card connected, you can check to see if it’s being picked up by the kernel with the following:

pi@pi-jamesha-0 ~ $ cat /proc/asound/cards
 0 [sndrpiwsp      ]: snd_rpi_wsp - snd_rpi_wsp
                      snd_rpi_wsp
 1 [ALSA           ]: BRCM bcm2835 ALSbcm2835 ALSA - bcm2835 ALSA
                      bcm2835 ALSA

You can play around with the alsamixer control panel or the Wolfson scripts to set things up. Bear in mind that to make full use of the realtime features of the kernel you need to prioritize properly; this is documented elsewhere on the net in detail and is generic across all Debian/Raspbian systems in terms of configuration, so I won’t cover that here.

Hopefully this will help people get set up with low latency audio on the Raspberry Pi; I will try and keep on top of the documentation around this in future!

3 thoughts on “Real Time kernels and audio on the Raspberry Pi”

  1. Hello James, great howto! I’m getting better results with a non-rt kernel though than with a RT kernel. What are your findings? And personally I prefer cross-compiling the kernel on a faster machines, this saves a lot of time! And what kind of latencies do you achieve?

    1. Cross-compilation is faster, but needs a bunch of setup work and another Linux box/VM, which people don’t always have handy! This struck me as the lowest barrier to entry, and really running a compile for 12 hours every now and then isn’t a huge inconvenience.

      Results-wise, the average latency with a RT kernel is 37uS and the max was 110uS over a two-day test period where I loaded the Pi down with an OpenOB transmitter in the background from the measurement tool. Are you using chrt etc to prioritize your applications appropriately, and what application are you using?

Comments are closed.