1) Redundancy and backups are someone else's problem (Linode charges you extra for that... although you'd be crazy to rely on Amazon as your only backup, it does cuts down on the tedium -- if you're a small startup, you probably want to be writing code, not playing sysadmin).
2) Scaling is someone else's problem. For static objects, S3 can handle traffic spikes orders of magnitude beyond what would bring your Linode to its knees (and there's their CDN option on top of that). Spinning up a new server on AWS (or a hundred servers, or a thousand) to handle a sudden burst of traffic takes a minute or less, and you can turn them off in an hour if the traffic dies down, paying only for that hour. Spinning up new servers on Linode may or may not be possible at any given time, and you have to pay for them for a whole month.
Running your own or VPS is cheaper for predictable, steady loads, AWS can be much cheaper and more reliable for unpredictable loads.
It's worth having AWS in your toolkit in case you get that front-page post on HN, TechCrunch, Slashdot....
Having a cloud provider in your toolkit to handle traffic peaks, sure. That's very different from handling your base load with AWS, which a lot of people seem to end up doing.
Handling your base load with AWS is ludicrously expensive compared to the alternatives.
And the upside of having your system set up to be able to make use of a cloud provider with instance instance spinup in your toolkit is that it increases the cost gap:
Whereas if you go dedicated only, you need to be able to handle reasonable spikes on what you pay for on an ongoing basis, if you can spin up AWS or other cloud instances as needed, you can push your dedicated boxes far closer to max utilization than what you otherwise would.
Handling your base load with AWS is ludicrously expensive compared to the alternatives.
Yes but the cost is tiny compared to paying some engineers to do all the hard work replicating what AWS offers on top of some alternative like Linode or whatever.
If your hosting costs are huge part of your costs, you're doing it wrong (or are very very successful, Google/Facebook scale).
2) Scaling is someone else's problem. For static objects, S3 can handle traffic spikes orders of magnitude beyond what would bring your Linode to its knees (and there's their CDN option on top of that). Spinning up a new server on AWS (or a hundred servers, or a thousand) to handle a sudden burst of traffic takes a minute or less, and you can turn them off in an hour if the traffic dies down, paying only for that hour. Spinning up new servers on Linode may or may not be possible at any given time, and you have to pay for them for a whole month.
Running your own or VPS is cheaper for predictable, steady loads, AWS can be much cheaper and more reliable for unpredictable loads.
It's worth having AWS in your toolkit in case you get that front-page post on HN, TechCrunch, Slashdot....