Posts by John Leach

Beta testers required to trial new Content Delivery Network (CDN) service 5 Nov 08

We’re currently working on offering a Content Delivery Network (CDN) service for Brightbox customers.  This will accelerate the serving of your static assets, distributing them around the globe and serving from the closest server to the user.  We’re trialling a partnership with Panther Express to provide this service, who provide CDN services for some pretty huge sites, such as LiveJournal, The Guardian and Shopping.com.

It can be used with the built-in Rails asset host system too, so no heavy modifications or fancy plugins.  We’re looking for some beta-testers, so if you’re interested please drop us an email to hello at the brightbox address and provide us details of your current static asset bandwidth usages if possible. All customers are welcome!

New service status blog 20 Oct 08

We’re moving the service status feed from Twitter to a full blog over on Wordpress.com.  The address of the new service is http://status.brightbox.co.uk/

Please update your bookmarks and feed readers.

Happy Birthday Ubuntu! 20 Oct 08

Today is Ubuntu’s 4th birthday.  Ubuntu is the GNU/Linux distro that makes our Brightboxes rock.

Happy birthday Ubuntu!

Now with Ubuntu 8.04 LTS Hardy powers 2 Oct 08

We’ve now switched to Ubuntu Hardy (8.04 LTS) as our default Brightbox image.

We’ve been using Hardy for months now to power our managed products and clusters and it’s proved very stable.  Some of the highlights:

• Ruby 1.8.6 as standard
• Apache 2.2.8 as standard
• Imagemagick 6.3 as standard (roll on the new rmagick gems!)
• Git 1.5.4 as standard
• Better virtualisation support - we can now live upgrade ram on Brightboxes without a reboot

It still comes pre-configured for Rails deployment and we’re still providing improved and extra packages where necessary (such as nginx 6.31).

We’re also still supporting our Ubuntu Dapper machines and can still pre-install Dapper for customers who are already using it on their other boxes.

Upgrading

Due to the way Brightboxes are built, you can’t easily upgrade your Dapper box to Hardy.  Currently, the only supported way to upgrade is to file a support ticket with us and we’ll rebuild your Brightbox as Hardy.  This will require downtime and redeployment of your app (a good time to move to the new Brightbox deployment gem if you haven’t already)

Happy Birthday Debian! 17 Aug 08

The Debian project was officially founded by Ian Murdock on August 16th, 1993.  That makes it 15 years old yesterday.  Without Debian we wouldn’t have Ubuntu, the distro that powers our Brightboxes.  Happy birthday Debian!

Emergency Maintenance, 8th - 11th July 2008 7 Jul 08

We’ll be doing some emergency maintenance on the cluster this week. We’ll be be migrating Brightboxes around to avoid having to reboot any of them, but the migration currently causes 10 to 20 seconds network outage.  Everything should continue as normal once networking is restablished - apps will not have to be restarted.

This work will take place between 1am and 3am every morning this week. We will need to log in to all boxes to make some Xen specific config changes during the migration. We’ll be contacting customers with HA clusters to arrange node failovers.

Apologies for the short notice.

Ruby Security Vulnerabilities 25 Jun 08

Some of you will have noticed the kerfuffle regarding the recent Ruby security vulnerabilities.  Fixed version of Ruby were released over the weekend but they are causing crashes in applications.  Until working fixes are available we’re all a bit stuck.

Details of the bugs have been kept officially secret but people are figuring it out for themselves (thanks to Zed in particular).  This secrecy has just contributed to the fear, uncertainty and doubt surrounding the issues and hasn’t helped the situation at all.

We currently recommend sitting tight until proper fixes are available.  When this happens, distros will release new packages in the usual manner.  Brightboxes are based on the Ubuntu distro and their security team are aware of the problem and are working on it (see the bug status here).

For those of you using the standard Ruby from Dapper (most of you) you should be able to just upgrade  using aptitude as soon as Ubuntu release new packages.  For those of you using the backported Ruby 1.8.6 packages, you’ll need to wait for us to backport the fixes once they’re released.  We’ll obviously be doing this asap.

We’ll update the blog as we know more.

UPDATE: Ubuntu have fixed ruby1.8 packages available now. They have already appeared in the Ubuntu security repository and are available for install.  Preliminary testing of the Dapper packages has been successful (gems with native libraries too).  We’re re-backporting the Hardy 1.8.6 packages right now and they’ll be available soon.

UPDATE: We have the fixed Hardy packages (1.8.6-p111) backported to Dapper available on the Brightbox testing apt repository.  They’ve passed a lot of preliminary tests but have not been tested extensively in production yet.  Please report any problems  with them (segfaults etc.) to support@brightbox.co.uk.

Ubuntu Openssh vulnerability 14 May 08

A vulnerability in some versions of Openssh on Debian and Ubuntu Linux was announced yesterday which can result in attackers gaining ssh access to machines with weak keys.  Any versions of Openssh that can produce these weak keys needs to be upgraded, and any weak keys in use need to be regenerated.

One of our admin keys, used for accessing customer machines for support, was generated on a vulnerable version of Ubuntu.  This key is installed on Brightboxes by default though is limited to access from the private network only, mitigating the risk somewhat.

We’ve generated a new key and have now installed it on all affected Brightboxes and removed the weak one (you may have noticed some ssh connections from the private network to your box this morning as the user bbox-admin).

The Brightbox distribution is based on Ubuntu Dapper, which is not directly vulnerable to this bug, but if you are using an ssh key generated on one of the vulnerable versions then your Brightbox might be at risk.  If you use any of the vulnerable versions of Ubuntu yourself then please follow the instructions in the Ubuntu security notice.

Jobs at Brightbox - Join the team! 14 Apr 08

Brightbox is looking for some new techie people to join the team. Depending on your skills you’ll be:

• managing and developing our hosting systems
• developing our free software Rails stack, based on Ubuntu
• supporting our Rails/Merb hosting customers

There are full-time and part-time roles available and you can work remotely from anywhere in the world or at the office in Leeds (or a bit of both).  We’re still a small team so we need people with pretty diverse skills and, more importantly, an ability to learn new ones. Overall, the following special powers are necessary:

• enthusiasm
• problem solving
• communication
• great written and spoken English

Regarding particular skills, you’ll need experience with some of the following:

• Coding: Ruby, C.
• Ruby stuff: Rails, Merb, JRuby
• Typical Rails/Merb deployment stuff: Mongrel, Capsitrano, Monit, Memcached
• Web servers: NGINX, Lighttpd, Apache
• Databases: MySQL, PostgreSQL
• Clustering: RedHat Cluster Suite (CMAN, CLVM, GFS), Heartbeat
• Virtualisation: Xen
• Storage: Linux LVM, Linux RAID, AoE, Bacula.
• Network Filesystems: NFS, Samba, Glusterfs
• OS: RedHat Enterprise Server / Centos 5, Debian / Ubuntu
• RPM / Deb packaging
• Advanced Linux Networking: Routing, Bridging, Netfilter, QoS
• Security: Technical auditing (Nessus, NMAP), Policy development.
• Flame retardant killer robots

Send a hello, a CV and salary expectations to jobs@brightbox.co.uk. CVs should be in an open format, preferably plain text or PDF.

Recruitment agents should e-mail our special recruitment company email address: root@localhost

Closing date for applications is 16th May.

NGINX 0.6 for Ubuntu Dapper 11 Apr 08

We’ve backported NGINX 0.6.29 packages from Debian experimental to Ubuntu Dapper and included the fair proxy balancer module.

It’s in our testing repository at the moment so give it a whirl (it will of course install on any Ubuntu Dapper box, not just Brightboxes).  We have a page on the Brightbox wiki on how to configure NGINX for your Brightbox apps too (which can easily be adjusted to any NGINX install really).

If you’re playing with any of this beta stuff (like these packages or the Brightbox gem) and have feedback or need help, feel free to discuss it on the Brightbox-beta Google group that i just set up.