Hacking at 0300

I use Amazon S3 for static storage, since nearlyfreespeech.net charges for storage. I've written how I've set up my .htaccess and my PHP files to use it. But now my work has decided to block all access to S3 (as "personal storage sites"), so I can't use my bililite.s3.amazonaws.com sites any more.

But I'm smarter than that; I'll just use a CNAME DNS entry to point to the S3 site. They're not blocking cdn.bililite.com. Unfortunately, it's not that simple. Amazon uses the domain to determine the bucket, so cdn.bililite.com has to point to cdn.bililite.com.s3.amazonaws.com. (remember that external CNAME's have to have the extra period at the end!), meaning the bucket called cdn.bililite.com, and I have been using bililite as my bucket name.

You can't rename a bucket. So I have to create a new bucket called cdn.bililite.com, and now I have to copy all the files from the old one. That's also not so simple, since downloading from one bucket and uploading to another takes forever for thousands of small files. Luckily, Amazon has created a command line interface to S3 that lets you do in-the-cloud copying.

Download and install that, then go to Powershell. Enter

aws configure

and enter your AWS keys (you can enter a default region as well). Note that these are kept in plaintext in ~/.aws/credentials, so keep your computer safe!

Then, aws s3 ls will list all of your buckets, and aws s3api get-bucket-location --bucket bucketname will list the region. None as a region name means us-east-1, as far as I can tell.

To copy the entire bucket, do

aws s3 cp s3://source-bucket s3://target-bucket --recursive --region region-of-target-bucket --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers

(if the region is the default region you set up with aws configure, you can leave out the --region option. I wasn't smart enough to include the grants option, which makes the copied objects public (hence read=AllUsers), and I couldn't figure out how to change the permissions with the command line interface, so I had to go back to the online Amazon AWS console and set the permissions there (open the bucket, select all the folders, and select "Make Public"). That took more than an hour, but at least it was in the cloud, not running on my computer.

Now, in order to be able to load things like fonts which obey the cross origin restrictions (why fonts? I don't know. AJAX I can sort-of understand), in the AWS console, under Permissions select Edit CORS Configuration to:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
    </CORSRule>
</CORSConfiguration>

Now I have Amazon S3 access that is not blocked!