Of OLAP and T3 (Plus more on projects)

Blimey, it’s been a while since my last post. I hasten to add that this delay comes only by virtue of the fact that I am exceptionall busy with various projects right now. I thought an update might be appropriate, in any case.

I’ve spent most of my time working on EVE Metrics. There’s some very cool, very powerful changes coming up soon; early Feb saw the introduction of much more accurate prices and indexes, with a newly improved algorithm for calculating the average prices of items. But even better is some of the new stuff coming in the next few weeks- notably the implementation of a fully-fledged OLAP warehouse for EVE Metrics, which will open up some very awesome possibilities in the long run.

Also under the scalpel this month has been the API system. EVE Metrics will support full and limited keys when it comes to the API. However, there will be a quirk; if you want to make use of other people’s API data, for example to see more detailed market analysis with transactions hooked in and so on, then you’ll have to share your data. This means if you want to benefit from other people’s data, you must reciprocate and share your data for the benefit of others. Your data will, in all cases, be used for global averages, but entirely anonymously; for example, the number of transactions per day on a given item may include data from your API, but nobody would know it. This system will hopefully encourage users to share data more often than hide it. I’m planning to make this an opt-out system, with the choice to opt-out given on the API key page as part of the form. It’ll be really hard to miss, and those who are paranoid or wish to hide their activity completely can check the box to opt out.

I released accVIEW a few days ago, and it’s had quite rapid takeup from corporations. It’s a service that lets you perform background checks on prospective new members to your corporation- the basic tool lets you view skills, characters on an account, and various bits of information like their corp details, CEO, and so on. The premium version (For the low cost of 150mISK) lets you see the applicant’s wallet journal (with tools to show suspcious transactions and filter the results), as well as their recent kills/losses.

EVE’s M10 expansion, Apocrypha, should be awesome. Tech 3 is going to be great- lots of people complain about the skill loss and the fact that it’ll make FCing a nightmare. Well, no. The skill loss makes sense, and provides an interesting new dynamic to EVE. FCing- well, those who complain about T3 making FCing impossible are evidently not up to scratch as FCs. It’ll give FCs a real change and challenge for the first time in years. The wormhole stuff will be an interesting thing to watch pan out- there’s lots ot potential there, and it could end up being a lot of fun…

Nexus is progressing slowly but surely; the large amount of data we’re gleaning via an API-scraping installation for Sc0rched Earth is helping no end with development, and we’re busy tidying things up behind the scenes and refining some of the interface to make more sense. Once I’ve gotten ActiveWarehouse’s ETL library working properly, my next step will be to break out Photoshop and my text editor- EVE Metrics, ISKsense, maybe accVIEW, and the new MMMetrics site (Which will be launched soon) are all going under the knife and getting a serious facelift. And then it’s on to even more awesome stuff for EVE Metrics!

Killboards Questionnaire Results

Firstly, an apology- I’ve not been updating as often as I should perhaps do. This was mostly down to the mild concussion I suffered fairly early on Wednesday morning, which limited my capacity for clear thinking for quite a while. Now, onwards- with a summary of what we got from the killboard questionnaire.

Most Wanted Features

  1. API import of losses and kills
  2. In-detail fitting information/fitting database
  3. Accurate price data
  4. Scoring/points
  5. Standings import

Most Wanted Fixes

  1. Needs to be quick
  2. Needs to be easily themed and styled
  3. Needs good documentation
  4. Needs a better admin panel than EDK
  5. Needs easier installation

Medals and ranks were a close runner up, as were better engagement summary tools.

So, let’s go into a little more detail and see how we’re covering each of these for Nexus.

Continue reading Killboards Questionnaire Results

Workflows in Rails Development

What’s your workflow like for Rails development? For Nexus we’ve got a fairly good workflow set up, though we’re only using it with 2 developers. When I’m working on standalone projects (ISKsense, EVE Metrics, and other sites where there’s just me working on the code) then I use pretty much the same workflow. Here’s how it works.

Git (hosted at github.com) stores the project source code. For a given problem, I fork, make changes, merge back in, and push. That results in the Git repository on Github having the most up-to-date version on the ‘master’ branch.

At this point, CruiseControl.rb (A great continuous integration tool) picks up the update, and tries to run the full RSpec test suite on the project. Emails are sent out to all developers when the tests pass or fail, with associated error information if it’s the latter, as well as a HTML report on our CCrb instance.

Assuming we’re ready to deploy, I fire up Capistrano (an excellent pure-Ruby tool for deployment) on my development box, and run ‘cap deploy’. Capistrano connects to the server, makes a Git clone of the most recent master copy, copies in keys/database configs (So they’re kept out of Git), migrates the database, and then makes a symlink to the new project so Apache (running Passenger) is updated as to where it should be looking. And voila, we’re deployed!

I’d be interested to hear what workflows other people use for Rails development. This works fine, and Git means it scales fairly easily to accept more developers, as well as providing incredibly flexible distributed control (So I can work on my laptop away from the internet, committing, branching and so on as needed). But I’d be at a loss to make this work on a more complex deployment, say with EC2 or other virtualisation software. A continuous integration tool that played better with RSpec would be awesome, too.