The buzz is building around Amazon Web Services as an application platform. Don MacAskill has been using AWS’ S3 storage service for SmugMug, and according to Jeremy Zawodny, will be talking about it at this year’s ETech conference. Jeremy and others have been experimenting with S3 as storage backup for desktop and laptop systems. Even Dave Winer is experimenting with S3.
When designing GigaVox Audio Lite (currently in by-invitation-only alpha test) I decided to use not just S3 (storage) but essentially all of AWS’ services for our infrastructure. This graphic should give you some idea of what we’re doing:
In addition to S3 (storage) I’m using the Elastic Compute Cloud (EC2) which allows us to instantiate servers on-demand. If we need a dozen extra servers for two hours of transcoding, we just fire them up for as long as we need them (perhaps a few hours) then shut them down. And that can be automated based on the load on the current servers. Dynamic computing power on demand. The dream has been realized.
To communicate among our servers and with the external systems we also make heavy use of the Simple Queue Service (SQS). There are actually many more queues than shown in the diagram which is somewhat out of date.
In asking around, it appears we have one of the most complex and sophisticated systems built to date using all three AWS services. It is infinitely scalable, extremely reliable and costs very little until we need it. And our system administrator doesn’t have to carry a pager. All those servers are someone else’s problem. No hardware or software to buy, maintain, upgrade, etc. It has been a lot of fun to actually make use of some of the asynchronous message-based architecture ideas in my book, Loosely Coupled. Hey…this stuff actually works! Sweet. Congratulations to Amazon for brilliant vision and execution.
If you want to see it in action for the production and publishing of your podcast, send me an email, tell me about your podcast, and I’ll send you back an invitation to the alpha test.
“all three AWS services…”
Amazon offers seven services, not three. You left out Amazon E-commerce Service, Amazon Mechanical Turk, Alexa, and Amazon Historical Pricing service..
I’m myself using S3 for quite some time — other than a stray slow response time once or twice, the service has been amazing. And even if the resp time is slow, Amazon folks are very quick to rectify it.
the amazon WS is just brilliant and a life-saver when it comes to startups who want to focus on execution and business rather than the headache of dealing with storage, backups and stuff like that.
slideshare.net also uses amazon s3 to host the flash based ppts.
GNC-2007-02-05 #240
Mixed bag of tech talk tonight along with some links to some really cool sites. The amount of listener has been amazing thanks for your continued support. Sponsors: [Save 10% off on any order at GoDaddy.com!] Use Code Todd [Try…
GNC-2007-02-05 #240
Mixed bag of tech talk tonight along with some links to some really cool sites. The amount of listener has been amazing thanks for your continued support. Sponsors: [Save 10% off on any order at GoDaddy.com!] Use Code Todd [Try…
Awesome setup, Doug. You’re one of the few out there really taking full advantage of Amazon’s offerings. For the curious, Basejumpr.com is another startup using S3, SQS, and EC2 in conjuction. I can’t wait to see what else gets built on top of Amazon’s infrastructure.
A while back I had built this: http://www.francisshanahan.com/robocast
with thoughts of perhaps developing it into a service. It looks like there are some parallels there but you have taken it to the next level with Gigavox.
Nice job.
GigaVox Audio Lite: Web Services in the Wild
Technometria is fast becoming one of my favorite podcasts every week. I like to say it’s “like The Gillmor Gang, but actually good”. This week’s episode is one of the best yet. It’s a conversation with Doug Kaye of IT…
I use S3 too for streaming videos over HTTP. you can check out the servcie at KONKAN TV. Apart from the ocassional failed SOAP / REST file push requests and ocassional CPAINT errors and/or streaming speeds, it’s been an awesome ride so far…
mduaoffrosmaofgoerosmonvosmogosnolvoslovnms
ncvosnrpos
Do you know where can I find the Amazon architecture? I mean, servers, and Databases.
Understanding Amazon EC2
Amazon’s new web services have been gathering a lot of momentum recently, in particular the Simple Storage Service (S3). I can wrap my head around S3 pretty easily and there’s no shortage of the canonical example of “remote backups…
Hi Doug, i was listening to your video on Amazon Web Services. Thanks for sharing the info.
I wanted to follow up on your decision to use the external Database server. Can you provide some detail on why you did not choose to use one of the images as a database server.
regards
The way that you break down the structure makes it much easier to understand. Great job.
Rick Marnon, Howell
http://www.oaklandlivingston.com/howell.html
what standards or lack of, were there when you were beginning to use Amazon’s web services?
Doug – where’s the graphic of the architecture?
Hi, I have a question about the real world usabitity of the Amazon Web Services. Could I run windows servers in windows vm clusters on the Amazon Elastic Compute Cloud (Amazon EC2) Windows Server running SQL, Office Middle Ware, VOIP… like a real hosted utility grid?
Can’t find any data on people using windows server os’s with it?
Thanks
Hi there, the graphic doesn’t show up any more
Any way I can have a glance at it? I’m really interested in how it works with Amazon s3!
amazons architecture is totally different from others there way of building in entirely different and i coulnt follow the block diagram above
The architectural style of the amazons are totlly different to the rest of the world and the blog diagram provided is very informative thanks.
police officers to the immigration service and recruiting a further 360 new staff to boost efforts against illegal immigration.
Yea,i agree with Larry
[...] provides infrastructure on demand, which can be anything ranging from operating systems to storage servers. Choosing this particular [...]
[...] et PHPAWS n’est pas une société rachetée par Amazon, contrairement à de nombreuses autresRetour d’expérience sur l’adoption d’Amazon Web Services Suivez l’actualité AWS en suivant l’expert Jeff BarrCommentairesPartager sur: Mixx [...]
Great post. Will read on…