Given what I’ve been writing for the last year and a half has been large scale distributed systems it occurred to me that I should probably look at Erlang, and in part Elixir.
Erlang, for those not in the know, is a functional programming language written by Ericsson ages back for telecoms switch programming. The key requirements were to build systems that were reliable, handled failure gracefully, and operated at huge scale with massive numbers of concurrent activities, connections and tasks.
Real reliability means distribution across hardware – you can’t do five-nines on a single computer. Erlang excels at these tasks by way of a very clever virtual machine and some interesting design choices.
The focus in Erlang is not only on functions, as you’d expect, but on processes. Erlang’s VM has a very lightweight (bytes of memory, microseconds of startup time) process and concurrency model, which distributes work across multiple cores and multiple machines almost entirely transparently. You might readily spawn a process for every single connection your service receives. Say “10,000 processes” to any developer used to practically any other language and they’ll laugh or cry. In Erlang that’s an entirely reasonable idea.
Erlang itself isn’t the whole story, though. There’s a lot of common patterns that emerge from this model and the Erlang/OTP stack was developed to encapsulate some of these. OTP stands for Open Telecoms Platform, though it’s not limited to telecoms work. This provides a simple wrapper around all this generic logic – things like making sure processes are restarted when they fail, managing state transfer to new versions of code deployed to a running system, and making hierarchies of supervisors and workers simple to implement.
Which is all well and good. These systems clearly work. So given me, a Ruby developer (mainly) who is used to the developer-friendly world of Ruby, Python and the at least well trodden territory of C/C++, how did I get on?
Erlang’s main issue for adoption is, in my mind, the lack of a friendly build tool and the lack of documentation written for mere mortals.
There are a few ways to build your code – escript, on the console with erl, erlc and so on. But taking a step back – the classic workflow for a developer writing some code is: write, compile, run, test. The latter two may be combined with unit testing etc, but manual checking is obviously something you want to do. rebar is apparently the build tool de jour, something akin to pip or a combination of bundler and rake. And indeed, ‘rebar compile’ and ‘rebar doc’ do what you’d expect. So how do we run our OTP app now we’ve built it? It’s entirely non-obvious. There’s some documentation, but it’s either so high-level you’d only be able to fully grok it after a month of picking it to bits and looking up other docs, or broken. So far I’m running things with ‘erl -pa ebin/’, then running lines like ‘application:start(testapp).’ to actually get things started. Which sort of works.
If Erlang wants to be a language people will choose to use, it really has to get its build systems sorted out to the point where human beings can go from “write some code” to “run some code” and see their system start and get a console on it without having to get lost in documentation and search the net for tips.
Elixir is a step in the right direction with its mix tool, and might be the point at which Erlang’s VM becomes approachable enough to be adopted at a larger scale. At this stage though it’s still too much like hard work for what is effectively a solved problem. It’s all the more frustrating because Erlang itself is quite approachable, straightforward and powerful, and it feels to me at least that improvements to the usual workflow to make write-build-run a simple task would improve things massively. Of course, I’m approaching this as a new Erlang developer having only spent a few days on it, so maybe I’m missing something. In the meantime, though, I think it’s onwards to Elixir to see how that works.