Posts by Rahoul Baruah

Queues and Callbacks 3 Nov 09

A major part of our work behind the scenes is about improving our internal processes and, whenever possible, automating tasks. To this end we have a number of systems that need to communicate with each other.

The Control Panel that you may be familiar with uses Delayed Job. This is a Rails-specific gem that uses the database as a queue, with a nicely packaged worker process that handles messages as they arrive. Because the Control Panel only ever talks to Rails from Rails, this worked extremely well.

However, our other systems were not homogenous – there are a number of different interfaces that needed to be instructed at various times and across various machines, and Delayed Job didn’t really fit the bill. In particular, there were some tasks that could only happen on certain servers – while Delayed Job let us have multiple worker processes on different boxes, it essentially managed a single queue, so it could not differentiate between messages for one worker and messages for another.
Read the rest of this entry »

Free software and Brightbox 9 Mar 09

At Brightbox we like free and open source software.

Every Brightbox runs Ubuntu, which is an operating system built on top of the GPL Linux kernel. Our infrastructure is built upon Xen, Apache, Nginx, MySQL, Nagios and many other open source software projects; not least of which are Ruby and Rails themselves.

But whilst we benefit from this software, without contribution, free software is nothing, so we contribute anything we can. The most obvious of these are the Brightbox deployment gem and its associated server-side tools. These are extensions to Capistrano that help you get your application onto your Brightbox as quickly and easily as possible.

We also have our apt repository where we repackage a number of free software projects to make configuring your Brightbox as easy as possible. More details on the repository are available on the wiki, but the most notable are our Passenger and Ruby Enterprise Edition packages.

However, nowadays, the real place for sharing your code is on Github. We have a number of projects available there, ranging from the tiny to the large.

These include:

• Flashing rails

  A rails plugin that makes it simple to display flash messages in your views in a consistent manner.

• Rujitsu

  A simple gem that collects together a number of convenience methods and various helpers.

• RSpec-rails extensions

  A gem that tidies up specifying your code with RSpec-Rails.

• Object Factory

  Brightbox’s very own answer to Factory Girl or Machinist that lets you build your test data with minimal configuration and no fixtures.

• Altered Beast and Redmine.

  We have taken our own forks of two popular Rails applications. David’s version of Altered Beast handles the Brightbox forums and Redmine handles our internal bug tracking and task lists.

• Warren and Bigwig

  Last, but by no means least, we have Warren and Bigwig. These are our wrappers to AMQP and RabbitMQ.

  We use RabbitMQ internally to deliver messages across our various infrastructure systems and needed a simple way to interface our ruby code to Rabbit (which is implemented in Erlang).

  This led to Warren, our wrapper over the AMQP protocol that make it simple to post messages onto the queue.

  In order to receive and act on those messages, we also built Bigwig (no prizes for spotting the rabbit references there), which takes those messages and responds. Bigwig matches each incoming message against a set of plugins, each plugin being small and focused on a particular task. Unrecognised messages are discarded, ensuring that rogue commands can’t wreak havoc upon our network.
  UPDATE: It turns out that Bigwig isn’t quite ready yet, as a big chunk has been rewritten. We’ll get it out there as soon as we can.

As these are all free software projects, please take a look inside and poke around. Any suggestions, improvements, patches or forks will be gratefully received. Also, stay tuned for an announcement on a major project we are looking to start in the next couple of weeks.

Update 2: We’ve also put the code for Isitruby19.com onto Github, under an MIT licence. Please go to the forum if you have any questions.

Announcing isitruby19.com: tracking gem compatibility for ruby 1.9 5 Feb 09

By now you will all have heard of the release of Ruby 1.9.1 – the first, stable, production-ready release of the next-generation Ruby interpreter.  This has a number of enhancements; RubyGems is bundled with the interpreter, native threads are used instead of green threads and we have a general performance boost all round (amongst many others).  

However, there is a significant barrier to Ruby 1.9 adoption; the compatibility of the gems that we have all come to depend on.  

Which is why we’ve launched isitruby19.com - a site that tracks gems and lists whether they are 1.9 compatible.  Each time you come across a gem that works for you, drop by and leave a comment, so we all get a feel for which gems need some work and which are ready today.

Using RSpec, Cucumber and User stories to build our internal systems 21 Nov 08

Here at Brightbox we are making heavy use of RSpec and Cucumber as we develop our next generation internal systems. These let us write specifications, in English and in code, for how the systems should behave. The specifications document the system for future reference and provide an automated test suite to prove that things are working as they should.

We chose RSpec because of its philosophy of “getting the words right”; code is often easier to write than it is read. As these specifications are also our internal documentation they must be easy to read as well.

However, as some of this Behaviour-Driven and Story-Driven development is pretty new, there isn’t much guidance on best practice, especially when it comes to the “User Stories” (which form the basis of the system’s acceptance tests). With that in mind, we thought we’d share our basic process we follow for each new feature.

(Download the original presentation here)

By the way, there’s a very subtle bug in the code; no prizes if you spot it!

Rails CSRF Security Vulnerability 19 Nov 08

Users of Rails 2.1 and 2.2 need to be aware of a vulnerability in Rails’ CSRF forgery protection.

For those that don’t know, Rails generates an authentication token within your forms and verifies this token when the form is submitted back to your application. This prevents attackers from crafting malicious requests whilst pretending to be your authenticated user.

However, for certain types of request (supposedly those that cannot be generated from a browser) this authentication token is ignored – in order to make it simpler for automated API access to your application (using JSON, XML or a few other data transport types). Unfortunately, text/plain is wrongly included as one of these types.

Luckily, the fix is simple. The long-term solution is to upgrade your application to Rails 2.1.3 or 2.2.2 (when they are released); the quick fix is even easier – tell Rails to verify text/plain requests by creating a file (called mime_type_csrf_fix.rb) in your config/initializers folder:


# temporary fix for http://www.rorsecurity.info/journal/2008/11/19/circumvent-rails-csrf-protection.html
Mime::Type.unverifiable_types.delete(:text)


“uninitialized constant REXML::VERSION” with Rails 2.1.1 and Rails 2.2 10 Sep 08

We’ve found a minor problem with Rails 2.1.1 (and Rails 2.2) on our Brightboxes.

Because we use Ubuntu Dapper, which ships with Ruby 1.8.4, it includes an earlier version of the REXML library (for XML processing). As you may know, there was a recent security vulnerability to do with the REXML library and these latest two versions of Rails include a fix for this.

Unfortunately, the library that ships with 1.8.4 is slightly incorrect. Ruby conventions state that a constant should be all capitals so, quite rightly, the Rails updates check for a constant called REXML::VERSION. However, in 1.8.4, the constant is actually called REXML::Version meaning that Rails falls over with an “uninitialized constant REXML::VERSION” error.

The quick fix is to manually edit /usr/lib/ruby/1.8/rexml/rexml.rb and add a new (correctly named constant). You will need to use sudo to edit the file, as it is owned by root; after editing the file should look something like:

module REXML
        Copyright = "Copyright © 2001, 2002, 2003, 2004 Sean Russell <ser@germane-software.com>"
        Date = "2005/224"
        Version = "3.1.3"
        VERSION = "3.1.3"
end

We are also looking at our options for a permanent fix for this issue.

RailsConf Europe 2008 Round-up 9 Sep 08

The time: Tuesday the 2nd September 2008, 9am CET.
The place: The Maritim proArte Hotel, Friedrichstrasse, Berlin.
The plan: Brightbox set up their booth and then have a leisurely stroll around Berlin, taking in the sights, sounds and Bratwurst, before a good night’s kip and a fantastic exhibition at RailsConf Europe.

That was the plan. Unfortunately KLM conspired against us. A two and a half hour journey rapidly expanded into a twenty three hour marathon, involving unidentifiable hotel food, a detour via Paris, lost luggage, hysterical, smelly geeks, truffle cake and “luxury” mojitos. We were also without Neil, who bravely volunteered to stay in England installing a load of new hardware in our new racks.

RailsConf Europe is pretty much the biggest Ruby on Rails event on our continent. We were told there were about 800 attendees here (although RailsConf US in Portland, earlier in the year, had twice as many). With that in mind Brightbox stepped up to become gold sponsors of the event, providing us with a booth between ELC Technologies, a global agile development shop, and Five Runs, the Rails profiling tool, who we know quite well.

We didn’t get the chance to see many sessions, but there was a pre-conference Q&A session on Tuesday evening with David Heinemeier Hansson (DHH), Jeremy Kemper and Michael Koziarski. It was interesting to hear DHH praising 37signals’ Xen virtualisation setup (this being the platform we also currently use), later on I chatted with him and explained how we’re working hard to get a production-ready Rails stack included with Ubuntu; which will help when setting up your servers, even if you don’t choose Brightbox.

DHH also opened the conference proper on Wednesday morning with his Keynote on dealing with legacy code. This was interesting to me as, like most developers, I am often overwhelmed by the urge to rewrite code that I wrote a few months ago. This was followed, later in the day, by Jeremy Kemper talking about performance improvements – in particular the new features in Rails that make use of the facilities that HTTP provides.

Overall, however, the consensus seemed to be that the talks were solid but uninspiring (apart from our very own Rob Lee with a talk on semantic markup, dressed in a Brightbox t-shirt).

At the booth things were mental – especially in the first break. All the free Brightbox t-shirts vanished in less than ten minutes, as did the “I love Ruby” stickers. However, our giveaway was nowhere near as good as ELCs – they had free beer!

We did meet some of our existing customers (a few for the first time in real life) and a lot of prospects. Interestingly, at least from our point of view, we saw a lot of interest in our Managed Cluster services (where we build and maintain a high availability cluster of boxes for you).

An evening out with a load of our friends from Yorkshire (the aforementioned Rob and Deb, Louisa, Paul and Charmagne) ended my involvement with RailsConf (I had to fly back early because my babby was starting school) but John and Jeremy stayed on till the very end.

Overall, the travel was horrible, the food was great (I had the best burger I’ve ever eaten at a hotel just off Friedrichstrasse) and the conference was good. Berlin is a magnificent city and we had a fantastic time with some old friends and made some new great new ones. So that’s our story – how was it for you?