One interesting item of trivia here: The per-request pricing has remained fixed since it was first introduced, on June 1st, 2007. When S3 launched, it billed based on storage and bandwidth usage alone, at $0.15/GB-month of storage and $0.20/GB of bandwidth.
In what must be one of the greatest "oh crap" moments of AWS history, what Amazon designed as an object store very quickly started to be used as backing storage for virtual disks when EC2 launched in August 2006: Since EC2 at that point had only ephemeral disks (EBS arrived two years later) people created virtual disks with each block -- often 4 kB sectors -- being stored as an S3 object. To make matters worse, since bandwidth between EC2 and S3 was free, you could do as much I/O as your network could handle, yet pay only for the storage.
Before Amazon changed their pricing to introduce the per-request fees, I knew people who were making upwards of 50 million S3 requests per day -- mostly PUTs -- while paying only a few dollars a month. The introduction of per-request fees quickly extinguished that usage case, by pushing the cost upwards by a factor of a thousand... a fact which I'm sure Amazon was very happy about, given that they had presumably been losing lots of money trying to service all the tiny operations effectively for free.
Reminds me of an anecdote of a large tech company that used internal cloud-like storage, each team got a certain "budget". They "charged" people in pseduo-money. They had a charge per GB, obviously. But one team figured out that there was no charge based on filenames. So they just stored all the data in the filenames. Lots of 4K long filenames of 0 bytes meant they had free storage. :)
I did a similar thing for a virtual machine service. Inventory was counted at 2AM on the 15th of the month. We would have a "planned maintenance event" from 1-3AM on the 15th of the month for our 12x5 supported apps. Took two years to get busted ;)
What I don't get is now Rackspace Cloud files (with the Akamai CDN) is pretty much the same price as s3, and you don't have to pay for GET, PUT etc. requests at all.
I think Akamai's CDN is still faster then Amazon's (especially here in New Zealand), and is one flat rate rather then Amazon's rate which increases outside of US/Europe.
Also Rackspace's control panel is much easier to use then Amazon's (In general I don't like the AWS panel, it's takes a while to get the hang of it, especially teaching a newbie is a nightmare) I can't say much about the API, but I'm guessing Amazon win's that as it is the standard.
Rackspace Cloud has regions in the US and UK only. AWS has regions globally including 3 in the US , 1 in Europe, 2 in Asia, 1 in Australia and 1 in South America. Object storage content is often sensitive in nature and not distributed publicly by CDN. Additionally, AWS provides a larger and more comprehensive suite of complementary cloud services including a more robust compute platform.
Akamai's NetStorage is a pain in the ass to work with vs S3. Akamai's CDN replicates much slower than Cloudfront (we've seen distribution/caching complete in seconds, vs upwards of hours for Akamai).
Yes, the AWS GUI is......difficult sometimes. We don't pay them because of their GUI. We pay them because their replication of our content is quick.
is www.rackspacecloud.com used for anything? I ask because the certificate is for * .rackspacecloud.com and expired on 4/1.. but https://manage.rackspacecloud.com has a * .rackspacecloud.com cert that expires in July (issued in 2011).
Also www.rackspacecloud.com just redirects to www.rackspace.com
Recently started hosting my Jekyll blog on S3 with CloudFront and it's already costing me more or less nothing. I love how they're constantly lowering the prices of their services.
I like that Amazon is sharing the benefits of their massive scale with their customers. Though, by keeping their margins slim, they'll make it increasingly harder for smaller vendors to compete.
"If you have bigger lungs than your competitor, all things being equal, force them to compete in a contest where oxygen is the crucial limiter. If your opponent can't swim, you make them compete in water. If they dislike the cold, set the contest in the winter, on a tundra. You can romanticize all of this by quoting Sun Tzu, but it's just common sense."
I think this is going to come into play quite heavily with Amazon Cloud Drive vs. Dropbox, given that Dropbox uses S3. Disk space will always be cheaper for Amazon.
Cloud storage companies reduce their costs and storage needs by analyzing stored content to find duplicate content (and I would think partially duplicate content) so that they can store only single copies.
I think BitCasa offers unlimited storage. They had stated some time ago that they can offer unlimited storage because their algorithms for content matching were very effective at limiting what they actually had to store. Size becomes a competitive advantage.
I agree, but anything Dropbox can do, Amazon can do too, including dedupe. And if Amazon owns the disks, they get to benefit from Dropbox's usage in terms of economies of scale, but without having to pay an outside vendor for the privilege.
I wonder if this will mean cloudfront pricing will come down as well. Suddenly it's a lot cheaper to host your assets on S3, without putting cloudfront in front of it.
S3 costs are internal costs within their datacenter; Cloudfront costs start to assume costs possibly outside of their control (i.e. vendors they have to pay for their edge locations).
One could only hope. However, I don't see this changing the S3/CloudFront decision that much. I'm not sure if we're special, but on our bill, requests cover about 10% of the CloudFront total, compared to the transfer price, while requests cover just 2% of the S3 total price.
I guess it quite depends on the type of assets. On our bill in the largest region, requests are about 77% of the total price. Motivates me to look into combining assets at some point :). (base64 encoding images in CSS etc).
In what must be one of the greatest "oh crap" moments of AWS history, what Amazon designed as an object store very quickly started to be used as backing storage for virtual disks when EC2 launched in August 2006: Since EC2 at that point had only ephemeral disks (EBS arrived two years later) people created virtual disks with each block -- often 4 kB sectors -- being stored as an S3 object. To make matters worse, since bandwidth between EC2 and S3 was free, you could do as much I/O as your network could handle, yet pay only for the storage.
Before Amazon changed their pricing to introduce the per-request fees, I knew people who were making upwards of 50 million S3 requests per day -- mostly PUTs -- while paying only a few dollars a month. The introduction of per-request fees quickly extinguished that usage case, by pushing the cost upwards by a factor of a thousand... a fact which I'm sure Amazon was very happy about, given that they had presumably been losing lots of money trying to service all the tiny operations effectively for free.