Welcome to Pandora

We’ve successfully moved all sites, email, DNS, and everything else on our old server, Highpoint, to our brand new machine, Pandora.  This has entailed a lot more downtime than we’d anticipated; this has mostly been due to lack of preparation on my part, a glorious DNS cock-up and the added complexity of having Highpoint’s backhaul fail three times as we tried to move across all the data.

In total it was a fairly mammoth operation by our standards; we transferred in excess of 100 gigabytes of data between the servers over the course of 12 hours, shifted over 20 websites and 3 major webapps, and got everything up and running again in under a day once we’d moved it all to the new box. The downtime has been annoying and I’ve certainly learned some lessons for next time, but here’s the flipside…

We’re now running on a much, much roomier machine. We’ve not got the environment perfectly set up and we’ll no doubt spend the next week tuning everything, adjusting things till they’re just right and fixing bugs, as well as adjusting and rewriting chunks of applications to make use of the extended caching capabilities of our new environment. We’re already using this to great effect in the EVE Metrics APIs but we can make better use of caching throughout our apps.

Once we’ve gotten settled in, we should be performing much better and more reliably than previously. We’ve already seen huge performance gains on our database (we can process more than twice as many uploads per second, for example) and we hope to have things even faster soon.

Of course, to achieve this I have been running on more or less an empty tank as far as sleep is concerned and working things in around my life at university, which has been interesting. Still, we’re at the point now where it’s more or less stable and everything basically works, so now I’m going to grab a few hours of sleep before lectures tomorrow, before a long long lie-in on Saturday. Enjoy!

EVE Mail and training make an appearance

At last, we get an EVE mail API! It’s a bit rubbish as APIs go – no message bodies yet- but it’s a great step in the right direction. And of course we’ve got it implemented and polished already over at EVE Metrics.

All you need to do is head over to EVE Metrics, log in (or sign up if you’ve not got an account yet), add your API key(s) if you haven’t already, and then enable the EVE mail API method. And voila- EVE mails, in your browser, updated as often as CCP lets us.

The icing on the cake is that we’ve also provided a feed for RSS readers for your EVE mails. Google Reader/iGoogle or any other ATOM-compatible reader (which is basically all of them) can now monitor your ingame EVE mails at the click of a button.

We’ve also gotten around to doing skill training- you can see what you’re training (queue support of course is included) on all your accounts.

The next logical step from here is notification support- get an email or SMS whenever your characters can train a new skill, whenever you get a new EVE mail, whenever one of your market orders is outbid or fullfilled. You name it, I’d love to see it notifiable. We’re still in the early days with that, but that’s where we’d like for that to end up.

We’ll be improving on these and implementing other APIs in the coming days- we want to get notifications loading for all you corporate types, and we’re looking forward to bringing more skill monitoring/information into the UI. I’ve got a lot of ideas bubbling around- we’re getting to the point where we’ve got loads of little snippets of data that can all tie in with each other, creating something really fantastic for you guys and girls, the users. And that’s awesome.

Of course, we need your help to make all this run smoothly and perform well, which it has problems doing at the moment. We’re still asking for donations here, you can buy GTCs in support of us here, and we’ve just opened up advertising on the site through Project Wonderful. Any form of help is hugely appreciated.

Moondoggie & Market Browsing

OK. EVE Metrics is my big market browsing project. It’s very complex, it’s got a lot of data, but it all basically comes down to this: People browse the market with a program running on their computer, and when any market data is viewed, EVE Online writes it to a cache file, the program decodes that and fires it at the server. We collect all these reports and build a single picture of the market in EVE.

There’s the top-down view for you. We’ve never really not had enough data. We get good market coverage in most regions and we’re fairly up to date in the grand scheme of things. But compare the actual market of EVE to EVE Metrics and we’re still a long way off having a truly accurate picture. EVE moves quickly- in some markets, from minute to minute orders will be shuffling around and changing price and being bought out.

With Dominion we got a new browser. This means you can now use the full EVE Metrics website ingame, but also (through some Javascript client hook additions) lets us provide a fantastic new tool to help us get an even better picture of the market in EVE.

If you fire up the IGB and head over to the upload suggestions page, you’ll be given a list of 10 items, and a few options for automatic checking. Choosing this option will prompt EVE Metrics for a list of items to check, and will automatically go and look at those items. It’s slow, but it works. In the space of a few hours with one user, we can get data for an entire region across all the items on the market. This is utterly fantastic and we’re really looking forward to the larger volume of data this is bringing to the site.

So, if you’ve got a spare moment, or you need to go AFK for an hour, or you want to help out while you’re mining, or you’re just tired of clicking the next item in the list, install the uploader and visit the page ingame to get started. Every upload counts and helps us build the biggest, best picture of EVE’s market we can manage to produce. Uploads to EVE Metrics are also syndicated to other websites and tools, of course. Your uploads and contribution of time help hundreds of users who use the site, and tens of thousands more who rely on our pricing, history and order APIs for their applications.

Oh, and if you’re a developer, we now have a server status API with all the information you could possibly want on TQ, Sisi and the API servers. It can be found here (docs here). Enjoy!