Using a CDN with S3

I’ve started shopping for a content delivery network for SlideShare. It’s a market with pretty opaque pricing: if you’re making the jump to using a CDN for the first time it’s not easy to get a real sense of what monthly costs will be.
Conceptually, integration between a CDN and Amazon S3 is pretty straightforward. Here’s the basic steps:
1) Dedicate a subdomain (say to serving up all the content you want to serve via the delivery network.
2) Make a cname entry in your DNS to tell traffic going to that subdomain to go to your CDN instead
3) Tell the CDN which bucket on amazon S3 you’re saving your static content on.
The CDN receives the request for content at a geographically local server (so Europeans hit a node in Europe, Asians hit a node in Asia, etc). The node will first look in it’s own (in-memory) cache. If it doesn’t find the content that is requested, it will fetch it from S3 and save it so that will have it cached for next time. How long they cache it for is typically configurable, and APIs are typically provided that allow you to flush the cache.
In my investigations so far the following companies have turned up as potential vendors:
Akamai (the biggest company in the space)
Limelight is another big contender (famously used by Youtube and other Web 2.0 video companies)
Panther express is smaller contender. I’ve had the most conversations with these guys.
Level 3 is interesting in that they’ve recently announced that they’ll be selling CDN bandwidth at normal bandwidth rates. I haven’t talked to them yet, but I probably should. ;->
If anyone has any other recommendations for vendors I should check out, feel free to reply on this post! Frankly I really wish Amazon would just provide this as a service on top of S3: that way we wouldn’t have to change any of our code at all! Unfortunately, it doesn’t seem like this going to happen in the near future.

4 thoughts on “Using a CDN with S3

  1. Frank Sinton December 1, 2007 / 11:11 am

    Hi Jonathan,
    I am looking for a CDN too, but will probably use PEER1 since that is where we are hosting. I wish Amazon would offer this too, as we are using S3 for images…
    BTW, is there any way you could include an enclosure or media:content with a .swf file? YouTube and others do this very effectively – it makes it easier to syndicate MediaRSS and still pushes out your brand.
    Loving SlideShare – thanks for the great service!

  2. Woody Ferrell December 3, 2007 / 2:02 pm

    We manage our own IP Network and run BGP with Savvis, Sprint, MCI, and ATT. What does that mean? Low-latency and high performance. We also manage our own datacenters and we currently have 17. What is the advantage? We do not have to negotiate space for our CDN Nodes, they are all in Peer 1 datacenter. We also do not cache your content until it is requested initially. The advantage is you are not paying for transfer until it’s requested at least once. I’m available to discuss and I hope this helps.

  3. Arun Vijayan December 26, 2007 / 11:42 pm

    I am really confused. Why can’t anyone setups a simple CDN using three freeBSD boxes loaded with Varnish which morrors S3. use your imagination to do the DNS trick.
    3 boxes preferably in NY, Amsterdam and Bangalore/singapore.
    When you are really getting loaded, put one BIG5 in front of these boxes and scale horizontally. And many hosting guys will do it happily for you. I really hate Akamai. But L3 looks promising if they keepup their word.

  4. Arun Vijayan December 26, 2007 / 11:45 pm

    BTW, I know somebody (big) from india who uses limelight for video streaming and they are pretty happy with it. Send me a mail if you want to have a word with them.

Comments are closed.