Synching Your Amazon S3 Asset Host using Capistrano

published Nov 06, 2007

Note: This article is out of date. The latest version of this article is on the new permanent page in the projects section.

So you’ve got multiple asset hosts running in your Rails application, and you’re using Amazon’s S3 to host your assets. Now you want to make sure that your assets are kept up to date. This plugin is a Capistrano recipe that keeps the asset hosts synchronized with the public directory in your subversion repository.

Usage

After you get everything setup and do your first deploy, just run cap deploy as normal and all changed files in RAILS_ROOT/public will be uploaded to all of your asset host buckets before the final deploy:symlink task.

The following tasks are also available:

  • cap s3_asset_host:get_s3_revision
  • cap s3_asset_host:find_changed
  • cap s3_asset_host:list_changed
  • cap s3_asset_host:find_all
  • cap s3_asset_host:upload_changed
  • cap s3_asset_host:upload_all
  • cap s3_asset_host:upload
  • cap s3_asset_host:reset_and_upload
  • cap s3_asset_host:setup
  • cap s3_asset_host:create_buckets
  • cap s3_asset_host:delete_all
  • cap s3_asset_host:connect

You can get documentation on these tasks by running cap -T

Requirements

This plug-in is a Capistrano extension. It requires Capistrano 2.0.0 or greater.

You will also require the aws-s3 gem

So far, this plug-in:

  • assumes that you are using the ‘checkout’ method of deployment.
  • only works with svn.

If you are using another version control system, I think all you’ll have to change is the two methods in lib/scm.rb. If you do get something other than svn working, please let me know.

If you want to use more than one asset host, then you have to either install the multiple asset hosts plugin
or upgrade to Rails 2.0 (see setting up multiple asset hosts in Rails)

Setup

To set-up, you need to do the following

  • Install the plug-in
  • Install the AWS-S3 gem.
  • Set up your Rails application to use asset hosts.
  • Set up your asset hosts.
  • Configure Capistrano.

Installing the plug-in

From RAILS_ROOT, run:

script/plugin install svn://svn.spattendesign.com/svn/plugins/synch_s3_asset_host

Installing the AWS-S3 gem

You need to do this on both your local computer and the computer that is defined as the asset_host_syncher (see Capistrano Configuration, below).

$> sudo gem install aws-s3

Setting up your Rails app to use asset hosts

Single asset host

For a single asset host, simply add the following line to RAILS_ROOT/config/environments/production.rb:

config.action_controller.asset_host = "http://assets.example.com"

Multiple asset hosts

Follow the instructions in setting up multiple asset hosts in Rails

Setting up your asset hosts

Set up a CNAME entry for each asset host pointing to s3.amazonaws.com. How you do this depends on your domain host. Here’s what it looks like on easydns

You may need to wait up to 24 hours for the DNS entries for these new hosts to propagate.

Configuring Capistrano

Capistrano installation

This plugin requires Capistrano 2.0.0 or greater.

To upgrade to the latest version (currently 2.1.0):

$> gem install capistrano

Once the plug-in is installed, make sure that the recipes are seen by Capistrano

$> cap -T | grep s3_asset_host

should return a bunch of tasks. If you don’t see anything listed, then you need to update your Capfile by doing the following (this is from Jamis Buck):

In Capistrano 2.1.0 or above:


$> cd RAILS_ROOT
$> rm Capfile
$> capify .

If you do not want to delete your Capify file, or if you are using Capistrano 2.0.0, add the following line to your Capify file:
Dir['vendor/plugins/*/recipes/*.rb'].each { |plugin| load(plugin) }

Capistrano configuration

Create a new file in RAILS_ROOT/config called synch_s3_asset_host.rb. Add the following lines to it, and edit to suit:

# =============================================================================
# S3 ASSET HOST OPTIONS
# =============================================================================
set :asset_host_name, "assets%d.example.com"
set :aws_access_key, "your Amazon AWS access key" # You can also set this in your environment as AMAZON_ACCESS_KEY_ID
set :amazon_secret_access_key, "your Amazon AWS secret" # You can also set this in your environment as AMAZON_SECRET_ACCESS_KEY
# set :dry_run, false# Set to true if you want to test the asset_host uploading without doing anything on Amazon S3
before "deploy:symlink", "s3_asset_host:upload_changed"

You have to do one more thing: in RAILS_ROOT/config/deploy.rb. Specify one of your web hosts as an “asset_host_syncher”, like this:

role :web, webserver1, :asset_host_syncher => true

The first deploy

Commit all changes to your rails application and do the initial bucket setup:

$> cap s3_asset_host:setup
$> svn commit -m "Adding synch_s3_asset_host plugin"
$> cap deploy

This will do the following:

  • Create your Amazon S3 AWS buckets
  • upload everything in RAILS_ROOT/public (in your svn repository) to each bucket
  • Set the revision in each bucket to the latest revision in your repository.

This could take a while if you have lots of images or other big files.

You’re done!

That should do it. Now, every time you run cap deploy, your asset hosts should be updated with any changes to files in RAILS_ROOT/public.

Let me know if you have any problems, suggestions or comments.

blog comments powered by Disqus