You are currently browsing the Brightbox Blog weblog archives for November, 2009
Brightbox founders Jeremy and John will be attending Conferencia Rails in Madrid later this week.
Brightbox is sponsoring the event and John will be speaking on Thursday about the wonders of UNIX – how its often forgotten tools can help with Ruby on Rails development and deployment.
They’ll also have some of our highly coveted tshirts and stickers to give away so if you’re going, make sure you track them down and say hello.
Passenger 2.2.6 (quickly followed by 2.2.7) was released last week and we now have i386 and AMD64 Ubuntu Hardy packages available in our repository.
As usual, details on installing the packages from our repository are available on our wiki.
If you’re using Passenger and it’s making you happy, please do consider supporting its development by donating money in the form of an “Enterprise License” direct from Phusion, the company behind it.
As I’m sure you’re aware, Rails can be told to use ActiveRecord (and hence the MySQL database) to store session data. (New Rails apps use the cookie store by default—See the rails sessions guide for more info on both.)
However, this session data is never deleted, which means your session table continues to grow and grow forevermore. Your old sessions are left stored in the database, and although the table is indexed to help with finding sessions, it will eventually fill the disk up.
At this point you might be thinking the solution is obvious, just empty the sessions table after so long and let it fill up again. Rails even provides a rake task that does this for us, rake db:sessions:clear. The problem with taking this approach is that any active sessions get lost as well, which could be people with items in their baskets, currently logged in users, etc.
There is another solution, which is to only delete sessions that we consider to no longer be active. The updated_at column in the sessions table has an index on it, and thus looks designed for this type of query to be run. On the Brightbox control panel, we’ve decided this is session data that hasn’t been updated for over 24 hours. (A side effect of this is customers that haven’t visited the control panel in the last 24 hours are logged out. We’ve decided this is ok, but YMMV of course.)
We use the following rake task that clears out sessions 24 hours or older, which is run via cron at 3am every morning. You can change the threshold by editing "1 DAY" in the query, see the MySQL DATE_ADD() docs for valid values.
desc "Clear expired sessions"
task :clear_expired_sessions => :environment do
sql = 'DELETE FROM sessions WHERE updated_at < DATE_SUB(NOW(), INTERVAL 1 DAY);'
ActiveRecord::Base.connection.execute(sql)
end
As you have deleted quite a large number of rows, the sessions table may now be fragmented and affect performance. Fragmentation also means the table may hold a piece of the disk larger than it should for the number of rows it holds.
De-fragmenting the table does introduce locks, so we suggest running this when your site is quiet. Though, it is most likely the command below will only take 1 or 2 seconds to complete.
To de-fragment your table login to your Brightbox, and use the MySQL client to connect to your database. You can find out your database server, and password from the left hand menu item “MySQL” in your Control Panel.
mysql -h mysqlserver -u username -p
Select the database which holds the sessions table.
use database_name;
Then run the optimize command to de-fragment the table.
optimize table sessions;
You should now have a smaller, and faster sessions table.
As you’ll all know, Rails has page caching baked right in – the first time an action is hit, it writes a html file of the result to the filesystem. Subsequent hits are served direct from the html file at high speed by the web server without ever involving your Rails app.
Expiring the cache is just a case of deleting the html file. But what if you want to expire an entire tree of cache files? Say you change something in a header or footer, so every single page needs expiring at once.
The usual way to do this is to just delete the entire page cache tree, with FileUtils.rm_rf. This works pretty well, but with a big tree you’ll get strange behaviour under high load due to concurrent access. Whilst your rm_rf process is deleting the tree, file by file, your webserver will still be looking in there for page cache files and Rails will still be trying to write them.
Read the rest of this entry »
We’ve built 32bit and 64bit Ubuntu Hardy packages for Ruby Enterprise 1.8.7-2009.10. These packages are still in beta, and this is quite a major change from the default Hardy Ruby interpreter,which is 1.8.6, so we recommend you test thoroughly before putting it into production. We’ve been using them for a couple of days with no problems though.
As with our other Ruby EE packages, they upgrade (i.e replace) the standard 1.8 Ruby installation. This means all your gems stay the same, and everything on your system immediately starts using them (Phusion’s own Ubuntu packages do not work like this). We’ve tested it with the usual Railsy native gems, RMagick, Mongrel, fasthread etc. and have had no problems.
If you’re on a Brightbox, just edit /etc/apt/sources.list.d/brightbox-rubyee.list and change the rubyee component to rubyee-testing:
deb http://apt.brightbox.net/ hardy rubyee-testing
Then update and upgrade:
sudo apt-get update
sudo apt-get install libruby1.8
If you’re not on a Brightbox, see the instructions on our wiki first. The wiki also documents how to revert back to the old packages.
As said above, we now have 64bit packages available too (which was recently made easier by some Debian package dependency updates, also included in our repository).
Please let us know how they worked out for you on our forum. Thanks!
A major part of our work behind the scenes is about improving our internal processes and, whenever possible, automating tasks. To this end we have a number of systems that need to communicate with each other.
The Control Panel that you may be familiar with uses Delayed Job. This is a Rails-specific gem that uses the database as a queue, with a nicely packaged worker process that handles messages as they arrive. Because the Control Panel only ever talks to Rails from Rails, this worked extremely well.
However, our other systems were not homogenous – there are a number of different interfaces that needed to be instructed at various times and across various machines, and Delayed Job didn’t really fit the bill. In particular, there were some tasks that could only happen on certain servers – while Delayed Job let us have multiple worker processes on different boxes, it essentially managed a single queue, so it could not differentiate between messages for one worker and messages for another.
Read the rest of this entry »