what happens if 1 shard contains a whole slew of large companies (macy's, gap, etc), or over time users on one shard grow to the point that they cannot be contained within one physical machine? Are they just assuming that this will be very unlikely?
No not at all. We already have this problem. Right now we are able to take the DB hit but we will eventually attack this problem from cache perspective. We will have replicated caches for this scenario. In worst case if a shard is overloaded we stop creating new users in it and put it on a dedicated physical machine. I dont think we will let the situation get there.
