Updated NGINX Passenger packages will follow shortly (they’ll be available via a separate ppa)

Updated NGINX Passenger packages will follow shortly (they’ll be available via a separate ppa)

Created by Wesley Beary and recently adopted into Engine Yard’s open source programme, Fog has a lot of momentum and is a great way to get started with provisioning resources across multiple cloud providers.

If you’ve not yet done so, request a Brightbox Cloud beta account to get started.

We’ve now got Ubuntu Hardy and Lucid packages available on our apt repository and we consider it ready for production use. Brightbox customers can upgrade from Passengr 2.x simply by running these commands on their Brightboxes:

sudo apt-get update
sudo apt-get install -qy libapache2-mod-passenger

If you’ve not got a Brightbox, you’ll need to add our apt repository key and config first. You can read more about it on our wiki.

We now have Passenger 3.0.0-pre3 packages available for Ubuntu 10.04 (Lucid) and Ubuntu 8.04 (Hardy). As they’re pre-release versions, we don’t recommend them in production just yet and have put them in their own repository to prevent any accidental upgrades.

If you’re not already a Brightbox customer, then you’ll need to set up base access to our apt repository first.

Otherwise, just add the new passenger-testing repository (switch “lucid” to “hardy” if you’re on Hardy):

sudo sh -c 'echo "deb http://apt.brightbox.net lucid passenger-testing" > /etc/apt/sources.list.d/brightbox-passenger-testing.list'

Then you can upgrade/install 3.0.0-1bbox1~pre3

apt-get update
apt-get install libapache2-mod-passenger

Passenger now has a native library, which depends on your version of ruby. For simplicity, these packages currently require the ruby1.8 packages to be installed. You can, of course, switch to ruby1.9 and passenger will auto-compile the necessary native support for you. We’ll be providing packages for 1.9 support soon, so you won’t need to rely on the auto-compiling.

Improved router syntax for Action Controller

The router syntax in Rails 3 has been completely revamped to build on the work from Rails 2 and provide a more elegant and flexible way to provide completely RESTful access for controllers. To get started see the new routing guide.

Brand new Action Mailer

Action Mailer was previously a bit of a hybrid, part controller, part model. Rails 3 sees Action Mailer completely rewritten purely as a controller, it now behaves much more like Action Controller. The new Action Mailer guide describes how to get going.

New query engine for Active Record

Active Record has adopted a new query engine to make complex queries more consistent and manageable. Execution of queries is now delayed until actually required and not when defined. For an introduction to the new query engine check out the new new Active Record guide.

Bundler

Traditionally, managing the dependencies for your Rails app deployment can be a bit of a nightmare. While Capistrano, Rake and other partial solutions make automating things easier, they’re often not elegant or simple. Bundler provides a complete solution to managing gems, libraries, frameworks and plugins that your app depends on. The latest release of the Brightbox deployment gem offers full support for Bundler.

Other improvements include built in XSS protection, an official plugins API, Agnosticism with plugins, Active Model callbacks & validations, better handling of character encoding and many more. For a more comprehensive list of changes see the release notes.

Rails 3 on your Brightboxes

Getting up and running with Rails 3 on your Brightboxes should be as simple as you’re used to with your existing Rails 2 apps.

Prerequisites

Rails 3 requires Ruby 1.8.7 or higher, Ruby 1.8.6 is no longer supported. If you’re using Hardy based Brightboxes you’ll need to upgrade the standard Ruby Enterprise Edition package, Lucid based Brightboxes include 1.8.7 by default. Ruby 1.9.2 is also supported, though we don’t have packages for it just yet.

Full support for Rails 3 deployment was added to the Brightbox deployment gem in version 2.3.7, so if you’re using an older version you’ll need to update. Don’t forget to update the brightbox-server-tools on your boxes too! The gem now has full bundler support for managing gem dependencies so if you’re using bundler you no longer need to define gems in your deployment recipe. Bundler is now our recommended solution for handling dependencies.

You’ll also need RubyGems 1.3.7 or newer. New Brightboxes include our 1.3.7 rubygems package for Ubuntu, if you’re running an older version you can upgrade like so

sudo apt-get update
sudo apt-get install rubygems

We’d also recommend running your app with the latest version of Phusion Passenger, especially if you plan on running Rails 2 and 3 side-by-side.

Installation

Installing Rails 3 is as simple as

gem install rails --version 3.0.0

Deployment

You can deploy your Rails 3 apps using the Brightbox gem just like you’re used to. Make sure you’re using the latest release of the gem which contains some new features to improve Rails 3 support.

Documentation

For more information on using Rails 3 see the newly updated official documentation. For help with running Rails 3 on your Brightbox, including information on running Rails 2 & 3 side-by-side see our Rails 3 wiki page.

For further information on using these packages see the release announcement for our Ruby EE 2010.01 packages.

Available from today, you can now choose either Ubuntu Hardy or Lucid as the base OS when building new Brightboxes.

Ubuntu Lucid is the latest Long Term Support (LTS) version of Ubuntu with security fixes provided until April 2015. It brings a whole bunch of upgrades such as Ruby 1.8.7, Monit 5, Apache 2.2.14 and new packages like CouchDB, Sphinx, Chef, RabbitMQ, MongoDB, ejabberd and many more.

We’ve treated it to the usual Brightbox Ruby deployment tune-up, including our Ruby Enterprise Edition 1.8.7-2010.01 packages. Updated Phusion Passenger packages are now available on our newly Lucid-enabled apt repository.

When buying a new Brightbox, you’ll see a combo box that you can use to select Lucid (Hardy is still currently the default). Upgrading from Hardy to Lucid isn’t really viable due to the way Hardy boxes handle kernels, so you’ll either need to request a re-image (which involves wiping your box, so make backups!) or buy a new box and move your apps to it.

As with our other Ruby EE packages, they upgrade (i.e replace) the standard 1.8 Ruby installation. This means all your gems stay the same, and everything on your system immediately starts using them (Phusion’s own Ubuntu packages do not work like this).

These packages are also the best way to get Ruby 1.8.7 on Hardy, which you’ll need if you’re playing with Rails 3.

If you’re on a Hardy based Brightbox, just create or edit /etc/apt/sources.list.d/brightbox-rubyee.list to contain the rubyee-testing component like so:

deb http://apt.brightbox.net/ hardy rubyee-testing

If you’re on one of our Lucid beta boxes provisioned before today, simply create /etc/apt/sources.list.d/brightbox-rubyee.list and add the rubyee component:

deb http://apt.brightbox.net/ lucid rubyee

Finally, update and upgrade libruby1.8:

sudo apt-get update
sudo apt-get install libruby1.8 irb1.8 libopenssl-ruby1.8 libreadline-ruby1.8 rdoc1.8 ruby1.8

If you’re not on a Brightbox, see the instructions on our wiki first. The wiki also documents how to revert back to the old packages.

ActiveMessaging is a great piece of code, but at first glance it appears to have a couple of wrinkles

• The standard configuration doesn’t support load balancing (so that you can talk to another broker if your normal one isn’t responding).
• The queues and topics you want to speak to are defined at class level – which is a problem if you want the queues to be dynamically defined based upon data values.

But thanks to Ruby, YAML and a bit of lateral thinking you can get around these issues.

ActiveMessaging uses the Stomp Gem under the hood, and that can do load balancing by itself. So all you have to do is get ActiveMessaging to pass the relevant incantations down to the Stomp Gem – intact. And you do that with a bit of fancy YAML. The trick is to add your host configurations to the login configuration entry in hash format like this:

 production:
    adapter: stomp
    login:
      :randomize: true
      :hosts:
      - :login: myloginid1
        :passcode: mypassword1
        :host: broker-host-1.somedomain.co.uk
      - :login: myloginid2
        :passcode: mypassword2
        :host: broker-host-2.somedomain.co.uk

The Stomp Connection class checks login to see if it is a hash and if it is uses that in preference to anything else. You can setup anything Stomp understands within that Hash. Check the class code for details.

The beauty of this approach is that it is entirely in the configuration. Your Rails code doesn’t need to know about it. The reconnection happens automatically in the background.

I’ve used an observer class within Rails to handle the message sending. It watches a model and sends messages when stuff is created and updated – nicely separating the concerns. But I wanted to talk to different queues depending upon what was in the model. So the standard ActiveMessaging::MessageSender approach didn’t seem appropriate.

After a bit of digging around it turns out that you can get hold of the broker connection directly rather than going via the Gateway.

client = ActiveMessaging::Gateway.connection

Then you can send to any queue you like, not just the ones defined up front in messaging.rb:

client.send("/queue/#{some_variable}", body, headers)

This is a lower level connection and you lose the filter chains and some error checking. However if you need it, it’s there.

This is generally considered to be best practice where ever a server is geographically located, however, it may  cause a problem if you are hosting a  geographically specific web application.

If your web application requires a different time-zone such as CET, BST or PST, or Daylight Saving Time is giving you a headache,  take a look at our Wiki page for a timely introduction to the subject of time-zones on Brightboxes.

We’ve built our Ubuntu Hardy packages for i386 and AMD64 architectures which are now available from the Brightbox apt repository.  We’ve also upgraded our librack-ruby packages to 1.1.0.

For example:

  require 'xapian-fu'
  include XapianFu
  db = XapianDb.new(:dir => 'example.db', :create => true,
                    :store => [:title, :year])
  db << { :title => 'Brokeback Mountain', :year => 2005 }
  db << { :title => 'Cold Mountain', :year => 2004 }
  db << { :title => 'Yes Man', :year => 2008 }
  db.flush
  db.search("mountain").each do |match|
    puts match.values[:title]
  end

The full announcement is here, github project here and rdoc here.  Hope you find it useful!

Today I’m pleased to announce a new revision of the Brightbox Deployment Gem, version 2.3.6. This release fixes some of the issues reported to us by our users, along with a few extra features.

The documentation on the wiki has been updated with the new fixes and features. If you have any tips or find any errors just let us know.

NEW FEATURES

Apt Package Dependencies

Just as you can currently define gems your application relies on and have them automatically installed onto your servers, you can now do the same with apt packages. Define them in your deploy.rb and they’ll be installed before installing your required rubygems, which means you can use it to install apt packages that gems depend on. As an example, the nokogiri gem depends on a couple of apt packages.

depend :remote, :apt, "libxml2-dev"
depend :remote, :apt, "libxslt1-dev"
depend :remote, :gem, "nokogiri", ">= 0"

Intermediate SSL Certificates

Support for Intermediate SSL Certificates in Apache has been added, meaning you no longer need to manually edit the Apache config files directly. Just add one line to your deploy.rb and the gem now takes care of updating the Apache config for you. An example SSL configuration:

set :ssl_certificate, "my_cert.pem"
set :ssl_key, "my_cert.key"
set :ssl_intermediate, "intermediate.crt"

See the wiki page for Adding SSL Support to Apache for more information.

Deploy.local.rb

A common issue we’ve seen with is setting the deploy password. If you put it in deploy.rb, then it inevitably ends up in your source control repository, which is a bad idea. A workaround we’ve suggested in the past is to have a deploy.local.rb file, which isn’t tracked by source control, and load that from within your deploy.rb file. This allows you to have the password (or any other sensitive settings) defined, but kept outside your source control.

As part of this release, the gem now has official support for a deploy.local.rb file. If it exists, the gem will load it in for you.

For more information see the Advanced Gem Settings page on the wiki.

Disable (re)generating webserver config

If you’ve made manual changes to your webserver (apache or nginx) configuration, then you don’t want an accidental deploy:setup to overwrite your configs. Up until now you just had to avoid running the command, but now there’s a setting to disable it for you.

set :generate_webserver_config, false

FIXES AND ENHANCEMENTS

• Max Age setting now accepts a number as well as a string
• deploy:initial reloads Apache so the new config is loaded

This package won’t actually support Rails 3 just yet though as we’ve yet to package and test the librack 1.1.0, which Rails 3 depends on.  We’re working on it now and once we’re happy with it we’ll add new packages to our repository as usual.

Brightbox is also sponsoring the meeting so there will be free pizza afterwards (free as in pizza, not as in speech).

More details on the NWRUG blog page.

Hope to see you there!

sudo apt-get update
sudo apt-get install libapache2-mod-xsendfile
sudo a2enmod xsendfile
sudo invoke-rc.d apache reload

As usual, details on installing the packages from our repository are available on our wiki.

If you’re using Passenger and it’s making you happy, please do consider supporting its development by donating money in the form of an “Enterprise License” direct from Phusion, the company behind it.

Expiring the cache is just a case of deleting the html file. But what if you want to expire an entire tree of cache files? Say you change something in a header or footer, so every single page needs expiring at once.

The usual way to do this is to just delete the entire page cache tree, with FileUtils.rm_rf.  This works pretty well, but with a big tree you’ll get strange behaviour under high load due to concurrent access.  Whilst your rm_rf process is deleting the tree, file by file, your webserver will still be looking in there for page cache files and Rails will still be trying to write them.

This is easily solvable.  On a POSIX compliant filesystem, like EXT3, the rename operation is atomic – it either happens or it doesn’t, there is no in-between state where it is half renamed or anything.  So, before running the rm_rf, you rename your highest-level cache directory to something temporary.  This means the cache expiry is instantaneous, even if you have 100 meg of page cache, and you won’t get Rails writing new page cache files into it whilst you delete it.

It’s a good idea to use a robust temporary filename format so two processes don’t end up renaming a cache directory to the same thing at the same time, especially if your page cache is on a shared filesystem.

An example snippet of code follows. It assumes you want to expire pages from the controller named entries.

require 'socket'
tmp_cache_dir = [Socket.gethostname, Process.pid, Time.now.to_i, rand(0xffff)].to_s
page_cache_tree = File.join(ApplicationController.page_cache_directory, 'entries')
FileUtils.mv(page_cache_tree, page_cache_tree + tmp_cache_dir)
FileUtils.rm_rf(page_cache_tree + '-' + tmp_cache_dir)

This renames RAILS_ROOT/public/entries to something like RAILS_ROOT/public/entries-hostname22351125837834752773 (which should be sufficiently unique across a number of nodes in a cluster as to avoid collisions) and then deletes it.

If you want to expire the entire page cache, you’ll need to change the default from RAILS_ROOT/public as you can’t rename and delete that (it has images and javascripts etc. too!). Change it to something like RAILS_ROOT/public/page_cache. You’ll need to update your web server config to consider this new path too.

Remember that rename is only atomic within the same filesystem, so if you symlink your page cache directory from your RAILS_ROOT onto a shared filesystem, then you need to do all your renames and deletes within this.

Also, this has the side effect of working around a bug with our shared filesystem, GlusterFS, which got upset with multiple concurrent directory tree deletes (this is now fixed though).

As with our other Ruby EE packages, they upgrade (i.e replace) the standard 1.8 Ruby installation. This means all your gems stay the same, and everything on your system immediately starts using them (Phusion’s own Ubuntu packages do not work like this).  We’ve tested it with the usual Railsy native gems, RMagick, Mongrel, fasthread etc. and have had no problems.

If you’re on a Brightbox, just edit /etc/apt/sources.list.d/brightbox-rubyee.list and change the rubyee component to rubyee-testing:

deb http://apt.brightbox.net/ hardy rubyee-testing

Then update and upgrade:

sudo apt-get update
sudo apt-get install libruby1.8

If you’re not on a Brightbox, see the instructions on our wiki first. The wiki also documents how to revert back to the old packages.

As said above, we now have 64bit packages available too (which was recently made easier by some Debian package dependency updates, also included in our repository).

Please let us know how they worked out for you on our forum. Thanks!

You’ll work closely both with our Development and Technical Teams to develop our customer control panel and other backend systems. We’re looking for someone who, in addition to being a great coder, will be an integral part of the team and an organised and enthusiastic project manager.

The role is full-time and you can work from home, from our office in Leeds, or a bit of both (the rest of us do a bit of both).

Send a hello, a CV and salary expectations to jobs at the Brightbox UK domain. CVs should be in an open format, preferably PDF or plain text. Closing date is Friday 11th September 2009.

As always, recruitment agents should e-mail our special recruitment company email address: root@localhost

Job Description

• Work under the Technical Director and with our other developers to:
• Maintain and extend our “cloud” management systems
• Maintain and extend our billing system
• Build the front end for our next generation of deployment systems
• Help with application consultancy (advising customers how to design and scale their apps)
• Work on side-projects that help the community and promote Brightbox (such as isitruby19.com)
• Extract code to be shared with the community as open source/free software
• Invent and develop new ideas for services and products
• Project manage several concurrent projects
• Work from home or our office in Leeds

Qualifications

• Great Ruby on Rails coding skills
• A track record of implementing live RoR sites
• Experience of test driven development using RSpec
• Experience of deploying Rails using tools such as Capistrano
• Experience of project management desirable
• Proactive developer able to work independently as well as part of the team
• Suited to working in an agile, flexible environment
• Good written and verbal communication skills
• Living within 2 hours of the UTC time zone

Passenger 2.2.4 actually is just a small bug fix release for a memory leak in 2.2.3, but obviously brings all the benefits of 2.2.3 too.  A huge number of bugs have been fixed, particularly the “Broken Pipe” errors some sites under heavy loads were experiencing.

As usual, details on installing the packages from our repository are available on our wiki.

If you’re using Passenger and it’s making you happy, please do consider supporting its development by donating money in the form of an “Enterprise License” direct from Phusion, the company behind it.

A denial of service (DoS) vulnerability was found on the BigDecimal standard library of Ruby. Conversion from BigDecimal objects into Float numbers had a problem which enables attackers to effectively cause segmentation faults.

ActiveRecord relies on this method, so most Rails applications are affected by this. Though this is not a Rails-specific issue.

We’re currently  building new Ruby packages for Brightbox customers with the relevant patches to fix this vulnerability. We’ll keep this post updated with the latest news.

UPDATE, 15:46 BST: New Ruby EE packages are now available in our Ruby Enterprise Ubuntu repository. We’re working on updates for the standard Ubuntu version of Ruby.

You can confirm that the update fixes the bug with the following command:

ruby -e 'require "bigdecimal";BigDecimal("E99999999").to_s("F");puts "OK"'

If your version of Ruby is vulnerable, you’ll get a “Segmentation fault” error message, otherwise it prints “OK”.

UPDATE: Official Ubuntu packages to fix this vulnerability are now available.  The Hardy package is libruby1.8 version 1.8.6.111-2ubuntu1.3 and the Dapper package is libruby1.8 version 1.8.4-1ubuntu1.7.  The packages will be available for install after a normal apt-get update.