Too long, more on EM2

Blimey. Twitter has distracted me from actually keeping this blog up to date. Still, better late than never, eh?

Since my last post a lot has happened. Temperatures have risen to record highs, reducing productivity as I persevered to build my own air conditioning on the cheap (which looks like a probable failure, pending a cheap pump). Charactr has managed to produce all sorts of interesting bugs in my inbox, delayed_job has broken in some new and interesting ways, and EVE Metrics 2 development has steamed ahead.

Let’s look at the last one quickly.

So far we’ve gotten most of the basic backend for EM2 done in such a way as to avoid scalability issues as much as possible. We’re using PostgreSQL instead of MySQL, we’re using table partitioning, we’re using an upload processor that does most of the log parsing work with a custom C extension we’ve put together (0.06 seconds as opposed to nearly 2 doing it in Ruby natively), and we’ve added more features and tools for statistics like inferred trade tracking and per-upload statistics.

So, a few things we’re pretty sure will be in EM2 at release:

• Inferred trade statistics and display – We can basically make guesses based on what we see about how much of an item is being bought at what price by observing changing or vanishing orders. It’s not perfect but mostly accurate.
• Performance. Pages will load quickly.
• Better APIs for developers – Including movement, historic price data (same stuff as you can see ingame on the graph tab) and other oft-requested APIs.
• Trade Finder. If you’ve got a hauler, some time, and want to make some ISK, this will tell you the optimal way to do so.
• API integration. You’ll be able to put in your API key to have your market orders autoupdated on the site, optionally fed into the main market display for more accurate data overall, and so on.

Stuff we’re looking to implement but we’re not sure if we’ll have it out in the first version includes Science and Industry integration, Location-based filtering, and so on. We’re also looking at a way to reward uploaders; after all, the site is made accurate by accurate and regularly updated data, and while uploading with EMU isn’t much of a chore we’d like to reward those uploaders who go and upload whole swathes of the market or bits that aren’t so regularly covered, and so on.

The team working on EM2 is more or less at it every day and with Makurid’s determination to make stuff fast, we’re getting scarily close to having a really, really nice framework on which to build some exceedingly powerful tools. So now we’re at the phase where we’re now working out the final feature list, and implementing them. Hopefully we’ll be looking at a release in a few weeks time, assuming nothing goes horribly wrong; watch this space!

Edit: This post was originally titled ‘Too long, new ideas’. Kinda drifted off from my original post and renamed it. Ho hum. New ideas can get talked about later.