The internet has flourished and works so very well because it is decentralized, based on open protocols, and systems working together to let people communicate. Let’s just compare standard email with the BlackBerry flavour for a moment.If you send an email, your email client talks to a server in a protocol called SMTP, the Simple Mail Transfer Protocol. The server will then take that SMTP request and talk to the server which the target domain has nominated to be the mail server for that domain. This is found out using the open Domain Name System, and supports simple load balancing and failover by virtue of multiple MX records and priorities. The servers then have a quick conversation via SMTP, and at some point (assuming all goes to plan), your email is now sat in someone’s mailbox. Their client will talk to the server regularly (using POP3 or IMAP) and will pick up on the new email that just came in, and tell them about it.

These protocols are all completely standard, simple and free to implement. They run over the standard TCP/IP network layer that makes up the internet, and there’s no centralization (except in the case of DNS, which is still massively decentralized compared to BBM). Everything supports full end-to-end encryption using the secure socket layer, a much more robust system for securing transmissions than BlackBerry’s unencrypted transports and symmetric keying.

Now let’s look at a typical BlackBerry user and compare them to an Android user (or iPhone, or Windows Phone, or even Palm Pilots – the protocols existed back then). Both are trying to send and receive emails. Most people rely on a free provider for their email, like Gmail or Hotmail, but many people run their own mail servers or use organizational servers (university mail servers, your office’s mail server, and so on). There’s a vast web of servers, all cooperating together to make things work. Enter the BlackBerry- you now remove those protocols from the device and instead put them into a mail server at the other end. This server talks SNMP to the rest of the world, and even speaks IMAP/POP to your mail server, behind the scenes- but access to your mails from your device requires the BlackBerry go-between to be functional. If that single point of failure fails, it’s all over- no connectivity for you.

So why would, on paper, you want to use the BlackBerry service? Well, BBM is an additional feature- group messaging based on… the BlackBerry equivalent of MAC addresses? Huh. Okay, so we’ve got a comms methodology from the 90s which basically implements a dumbed-down version of IRC or Jabber for your phone. Not like we’ve had that on other platforms for as long as I can remember – I was using Jabber on my phone years ago to talk to people on the other side of the world about EVE Online fleet operations instead of paying attention in maths lessons. Again, things like IRC and Jabber are open standard protocols, free to implement, and implemented in a manner that is widely distributed with little to no single points of failure. IRC was designed for the days of acoustic couplers, and is still used widely by millions of people. The BBM protocols aren’t even secure- they’re encrypted, but the key used for encryption is shared amongst all phones. It’s a bog standard symmetric crypto cipher. This means if you can convince a phone that it’s actually another phone, you can read any other person’s messages and send messages as them. Contrast with IRC – the network is still assumed secure (just as with BBM) and unencrypted, but links between servers and links between client and server can all use the secure socket layer- a very secure protocol.

On top of this we’ve got the push email feature – which lets you get emails sort-of-instantly (actually delayed unless your organization runs a BlackBerry Messaging Server, as I understand it) pushed to your phone instead of having to have the phone fetch new emails. Granted, that’s quite a nice feature. But it’s not a game-changer to the extent where I’d be willing to sacrifice my email connectivity on my device to have it- for days at a time, no less. In addition, there’s open tools that will do this for you on Android and iPhone. I remember back when I was first using a Windows Mobile PDA, I set up push email in the course of a few days of hacking around with an off-the-shelf product. This was back in the days of the XDA, when push email was just kicking off, and it wasn’t too hard to make work. These days it’s even easier.

BlackBerry falls over because they are a quite heavily centralized company and their infrastructure is not designed for the modern age. Note that the BlackBerry announcement about their recent outages referred to “Europe, the Middle East and Africa”. One problem can hit all those users. That’s not a well-designed system, it’s a legacy system shoehorned in from the days when BlackBerry Messaging Server was the sort of thing you ran if you were a big company. It is bound to fail and when it fails it pisses off many millions of users.

The question of why people decided to go with BlackBerry remains a mystery to me- I honestly don’t know why people would choose a BlackBerry over any other phone. I have spent countless hours debugging and fixing broken BlackBerries for friends, and many more hours still trying to explain how to use them to other friends (who inevitably end up sending them back and getting a ‘droid). Perhaps there’s some secret feature I’m not seeing, but it just seems to me that the hardware’s mediocre, the software’s awful at best, and the infrastructure is flaky. Why choose that over solid and varied hardware choice, great software with flexibility, and open infrastructure with no reliance on single providers or services? I know which one of these combinations has given me years of flawless service with not one day without email.

Recently, CCP decided to ditch their old forums (ASP, around 10 years old with little to no updates in that time) and started work on a new forum system. They’d just launched EVE Gate, which is a fairly minimal but working social network for EVE, and allows you to view some in-game features out of game. Many people decried EVE Gate, saying CCP should have just focused on building a better API first- then making their own CCP-owned apps on top of the API. Still, EVE Gate was done and launched to a minor fanfare. Some people use it a lot (in-game email is a handy thing to have at work), but many people ignore it outright.

What was shouted about most on the forums was the investment of time and effort from CCP to produce a very minimal site, features-wise. EVE has a stellar (no pun intended) community, with some very bright minds in it, who have built some fantastic apps on top of a fairly limiting API, and done so in their own time as a hobby project. Sites like EVE Metrics, EVE Commander, and such were all cited as examples of this- people were whipping up massively more complex and powerful sites than EVE Gate in their spare time in a matter of months, so what took CCP so long?

Well, the same people who developed EVE Gate (to my knowledge) were applied again to a web based problem; new forums. Now, a logical thing to do would have been to build on EVE Gate, and to add forums directly to that. This means you’d have only one core framework of software to debug and worry about, your authentication (a huge consideration for games like EVE) is all in one place, and in terms of validation and security testing there’s less to go wrong and fewer places to look when the shit hits the fan.

But CCP’s Web Cell did not. They sank a colossal amount of time into the project; the number being thrown around is 72,000 man-hours, which sounds about right to me given when they started and how many people are working on it. 72,000 man-hours is huge. Months and months and months and months. Forums are not complicated beasts; they’re essentially a simple relational DB application. You have to apply some thought about formatting (implementing bbcode or something like Markdown/Textile), accounts, profiles, all that sort of thing, but everything boils down to very simple structures, and simple code. There’s nothing complex, really.

CCP has one extra thing to think about atop of most forums: Integration of EVE accounts and characters. You have to be able to log in, and select a character to post as. But with EVE Gate, they had all that already done and dusted. And, crucially, fairly well tested- exposed to the world for a good long while. But again, this is all quite simple stuff.

But the web cell still managed to botch the job. And not just a half-arsed botch. Oh no.

They didn’t decide to integrate with EVE Gate. They completely ignored it, in fact. They took an existing ASP.NET forum software package, Yet Another Forum, and then skinned it. They just made a CCP skin for it. But that’s not all they did- they also gutted parts of it to tie in the authentication system, filling massive chunks with serious security flaws.

At approximately 21:00 UTC on Friday, April 8 we were made aware of some security issues with the new EVE forums which needed to be addressed. These issues were as follows:

• We discovered that it was possible to access some forums which certain users should not have been able to access
• Users could make and edit posts as another user’s character
• It was possible to inject some HTML code into signatures

At this stage the competent web developers (and software developers in general) are staring blank-faced in incomprehensible confusion. How could a 600-man industry-leading corporation like CCP let this sort of thing slip?

Authentication was done by cookies. That means there was a client-side cookie which stored your character ID. Change that, and you could appear to be logged in as anyone. And act like it, too- this wasn’t just a display bug. Change yourself to the CEO of CCP’s character, and you could see all the private internal forums. And the admin panel. And you could ban people! Handy.

Editing posts was unauthenticated. You could be logged in as you, edit your post but then change the post ID in the edit URL to the post you wanted to edit. And hey, no questions asked – go right ahead!

And finally, at least one HTML injection flaw. Why? Because HTML is used for formatting.

Now, these are all mind-blowingly simple ‘My First Website’ cock-ups. Any competent developer who had knowledge of programming web applications would never have made any of these mistakes. So why are there no competent developers in the web cell?

Here’s a clue. CCP Games hf is based in Iceland. Chances are, you’re relocating to take a job, and that relocation is putting you in a country far, far away from wherever you used to live. That’s a massive downer for potential employees. Next, the other part of the puzzle: I currently do freelancing. With a few clients on the go at once, doing roughly 20 hours of work a week during holidays as a student, per year I earn well in excess of what full time employees of CCP’s web cell get paid. I’m not even out of university yet. There’s one other thing to consider, especially now: CCP is not the sort of company that looks good on your CV any more. There’s some incredibly smart people working at CCP on some incredible stuff. But their reputation is tarnished, almost beyond repair, by this sort of fuck-up. Having CCP on your CV is something you want to think twice about. And working somewhere that triggers that sort of thinking is not looking great, is it?

And last but not least, have a look at the CCP Jobs website. Check out the requirements for a web developer:

Required Experience/Background/Skills:

• B.Sc. in Computer Science or related field, or equivalent training and professional experience
• 2+ years of experience developing & implementing .NET based web solutions
• JQuery, JSON, JavaScript, ASP.NET, MVC, C#, Visual Studio 2010, HTML/DHTML, MS SQL Server 2008, T-SQL
• Strong communication skills – verbal and written
• Strong technical and analytical ability
• Ability to complete projects on a timely basis with an attention to detail with minimal supervision

Now, there’s a bunch of problems here. Firstly, their HR people clearly do not know what makes a good web developer. Listing DHTML is cause for concern – it’s a buzzword from the 90s, nothing more. Personally speaking I have great concern for people who have only been using Microsoft toolsets, particularly for websites, and I’ll go into this a bit more in a second. The other problem with the above list is that, BSc aside, if they were willing to overlook my lack of experience with things that didn’t exist back then, I’d have been qualified at age 15 or so. And then the last bit: “attention to detail with minimal supervision”. That’s very worrying.

It’s worrying because it implies that these underqualified, fresh-out-of-school/university undergraduates who have potentially never written a public-facing website are not going to have their work checked. That’s a clear, utterly obvious problem. That’s saying “You’re confident you’re good enough to not need a supervisor keeping track of your output” to someone who probably has a healthy ego on them but in reality could be utterly useless.

The all-Microsoft toolchain that CCP uses and requires people to know is all well and good, but security has never been in the forefront of Microsoft’s brain. Specifically, tools like Visual Studio and ASP.NET for web developers focus on making life easier. Reducing the amount of work you have to do and increasing the amount of bolting together existing bits and pieces. This hides the underlying reality somewhat, though, and can lead to people just not being aware of things like client-side cookie tampering or cross-site scripting. Malformed HTML just isn’t something most MS web devs really think about- not till they’ve had a few rounds of learning the hard way, at least.

This is just my personal experience, based on people I’ve worked with and projects I’ve had to work with in the past. If you hire people who did their first major websites using a text editor, you’re likely going to get people with more knowledge about why things work, and can better understand how people can attack their nice shiny working things. And that makes you a better programmer. It’s one of the reasons I prefer working with Django/Rails/Sinatra- none of these things hide code from you. They may let you generate code with helpers, but even that is merely convenience- you have to know what the helpers to do to be able to use them, at a code level. Dragging text boxes onto forms does not give you the same experience of interacting with the code directly, and abstracts a lot of important stuff away from you. The result? Worse code.

CCP was very proud at fanfest of the fact they’ve now got over 600 people in the organization. But what sort of people? And why so many? Sure, a goodly number of those people will be DUST and Incarna developers and artists and so on. And you’ve got admin people to go with that all. But CCP’s first website was done by far fewer people than are currently on the web cell. And say what you like about 10-year-old forums, they work, a point made loudly by many forum users. CCP could perhaps do better by hiring fewer people, but people with better real-world experience.

So, the real screw-up here was not a purely technical one; it’s much more about the people who are behind the tech. And CCP is going to find themselves in a bad place on that front in very short order, because nobody except fresh-faced undergraduates is stupid enough to want to work for CCP right now. And I hate myself a bit for saying that, but it’s true. CCP is becoming a liability to itself through its own actions. And like all bittervets, I only whine and complain and point out this sort of thing because I fundamentally love EVE. It remains the only MMORPG that transcends being an actual game and takes on a life of its own. But with CCP at the helm, the world is in perilous danger of being torn to shreds by the very entity that created it. Which would be a terrible shame.

I don’t like politics. I stay out of it when I can and when I get involved it’s because I actually think things are too important for me to just hide away. Student politics has a habit of turning nasty on a dime and I can do without the stress.

So why on earth am I bringing this mess to my humble home away from Facebook? Well, simple: I think it’s too important not to. Specifically a lot of people seem to misunderstand the issue and Facebook’s a crap place for good discussion and explanation. SURHUL tried to put up a page explaining both sides of the motion but I and several other people on both sides think it’s crap. So here’s my take on it.

A disclaimer, first: I support wholeheartedly UCU for deciding to take action. It’s sensible, pragmatic, and a reasonable approach given the evidence I’ve seen. I also voted against. However, I think that what’s needed here is a good, hard look at what happened to lead to a referendum. I’m not going to try and make your mind up for you. Let’s have a look at the motion itself and some surrounding stuff.

You can view the motion by clicking here, and the motion as it was originally proposed by clicking here (.doc, sorry – the SU doesn’t publish in PDF or TXT).

So why is this a contentious motion at all? Well, I was at the general meeting when this was proposed, and the motion was initially thrown out for being unconstitutional. That’s an interesting term in the context of the SU- it basically says “This motion goes against what SURHUL is meant to do, so we’re not allowed to discuss it because if it succeeded we’d be going against our own constitution”. Specifically, the motion went against point 2.2.c-stating that one of the functions of the SU was “To cater for the welfare and needs of all its members”. Now, this is where it gets complex.

Kieran Miles, who originally proposed the motion along with Dan Cooper, pointed out that in a previous GM the topic of the NUS Woman’s Rep was discussed despite being deemed unconstitutional, and the floor thus (after some to-ing and fro-ing) opened to debate. However, there were some amendments. A lot of this was because of this little line in the original motion:

[This Union Believes] 2. That whilst a strike may momentarily inconvenience students whose lectures will be interrupted, it is more important that lecturers are able to fight for their pensions and other working rights

This was the point that was most against the constitution as it specifically said that it would potentially inconvenience students and that students were less important in this regard than lecturers.

It’s worth noting at this point in the GM that the meeting hadn’t really started- the agenda hadn’t been approved. Overturning discussion of the motion was voted down (35 against 22, with 7 abstentions) and the agenda was subsequently approved, 40 to 12. And then a general meeting happened- the usual things, essentially, till we then got to the motion in support of UCU discussion. We started off with some for/against speeches as is protocol- this was an exchange between Kieran Miles, Beth Rowley, Dan Cooper and Emma Best. The speeches for focused mainly on the problems lecturers faced, and that the impact would be limited to 2 days of strike action this term. Dan Cooper also claimed the SU had a “pivotal role” in defending lecturers. Beth Rowley reiterated that the SU priority was the students.

Now, the people who were trying to put this through were arguing that it was “highly speculative” that there would be disruption to students. The SU’s president, Rachel Pearson, pointed out (quite rightly) that the problem with this particular point was not it’s political nature but the fact that it was not in the interest of students.

The SU is always about students, as defined constitutionally in 2.2, and so the point I mentioned above got slimmed down to this in an amendment from Dave Cobb:

[This Union Believes] 2. It is important that lecturers are able to fight for their pensions and other working rights

After this amendment was approved, Joe Rayment suggested that the motion should go to referendum as a GM was not particularly representative of the student body. Kieran Miles pointed out that there was advance notice on the SU website several days in advance. Victor Garcia (VPSA) and Stef Phillips both urged for a referendum, with Stef Phillips pointing out that a referendum was the fairest approach.

At this stage, Rustam Majainah claimed that it was clear that exams would not be affected and that a referendum might be a “misrepresentation”. I’m not quite sure what it would be a misrepresentation of, mind you… UCU reps were not present so some fact-checking was going on and so on. Eventually, a referendum vote was held and passed 34 to 18, with 8 abstentions. Rhiannon French came in with a last-minute amendment- and this is an interesting one. Here’s a chunk from the original proposed motion:

[This Union Mandates] 3. That SURHUL will support future UCU industrial and strike action in opposition to austerity measures by the current government.

Now, that’s mighty dangerous in my view. That’s a “whatever UCU does we’re cool with, forever or until someone gets enough support to change it” clause. When you’re talking about UCU being a body that is currently striking in a manner that has the potential to impact students, that’s hugely irresponsible to suggest. Rhiannon’s amendment eventually gave us the following:

[This Union Mandates] 3. That SURHUL will discuss its stance on future UCU action as the need arises.

Which is a much more sensible approach, if you ask me. That got through with 42 for, 2 against, and 5 abstentions. A vote held asking if we were going to put the amended amended version to referendum went through with 43 for, 3 against and 1 abstention.

Now, that’s how it came to be. And as I approach a thousand words I appreciate you’re probably asleep now. So let’s get to the exciting stuff!

Just kidding. There isn’t any. But there is the motion as it stands for referendum.

The important thing in my opinion is to be able to tell the difference between students supporting lecturers and mandating support from an organization to support lecturers.

And another important thing to note is how discussion of SURHUL’s stance on future UCU action might take place.

If the referendum goes through, and we support UCU this coming week as they strike, that’s all well and good perhaps. Though note that if your dissertation is late getting marked or you miss a class or anything like that, SURHUL can’t help you as much as they could. Which is a worry. But now consider this: While UCU hasn’t published any upcoming strike dates beyond next week (and why would they? Stopping people from planning in advance is a great way to help maximise the impact of a strike), several lecturers have explained in writing that they are going to strike for a longer period in the middle of exams next term if there’s no progress talking to management.

Now- Rhiannon French’s original proposal was to add the phrase subject to referendum in to the end of the original text. However, that was not the text we ended up with and future discussions would be likely to result in a decision being made by a General Meeting. This is dangerous because GMs are poorly attended and thus susceptible to popular groups getting all their friends to attend and vote for them. This may well not be in the interests of the students and could lead to a position where SURHUL is completely going against the majority of student’s wishes. This isn’t a good place to be in, clearly.

There’s another matter at play and that’s just simply asking why SURHUL needs to be supporting lecturers who have their own union. If a motion were proposed that said “SURHUL will support lecturers defending their own pay and pension etc interests so long as this does not conflict with the constitution” (obviously better worded, but you get the drift) I’d probably have voted for it and there wouldn’t be any problem with the suggestion from most people. If all we’re talking about is solidarity then the ACA and friends would be happy and everyone would just be able to be happy and get along. However, because this motion seeks to mandate SURHUL’s support in all situations, this is a much more damaging motion for students than it need be. It’s not a sensible or pragmatic motion, and it’s not doing anyone any favours on any side of any table, and that’s why I voted no. If you’ve not voted yet, go vote. Whichever way you want, obviously. But do your research before you vote.

Oh, and if you’re annoyed at how we vote, go have a read of my thoughts on improving how we vote at SURHUL.