I am wondering if the future of multi-cloud lies in a third party entity managing real-time auctions for cloud services.
1. The developer sets up some requirements: number of instances, X amount of CPU power, Y amount of memory, Z amount of storage, B amount of bandwidth, or other characteristics. These requirements are sent to the third party.
2. Cloud vendors (the "bidders") receive the characteristics needed by the developer. Each of them bid by providing a price per hour for the given characteristics.
3. The bidder with the lowest price wins the auction, sends back credentials to start deployment
4. The developer can start the deploy with the infrastructure provided by the lowest bidder!
All this may happen in less than one second, similarly to the auctions that happen when advertisers bid for displaying an ad when someone visits a webpage. This would need a massive standardization across cloud vendors, a system of penalties when the infrastructure provided by the bidder does not satisfy the characteristics, etc.
This does not too far fetched to me. It also make it easier for new cloud providers to start selling, and lower prices for developers.
IaaS providers differentiate themselves on performance and price, but also reliability, networking details, auxillary services, tooling, API intelligibility, and support.
Cloud agnosticism sounds like a worthy goal but 9/10 it's the wrong move as it reduces you to the lowest common denominator.
How about moving gigabytes of data that underlie the application? The egress cost is not that low. The access latency is utterly important. Raw computing power on small amounts of data has a limited applicability.
It's also a reason putting things on the cloud is a fools errand. If you can be held hostage by a provider, you made a business mistake by going there in the first place.
Egress is not that expensive. You use it to serve the customers, in the end.
But it's (still) too expensive to allow migration between cloud providers e.g. on hourly, or even daily, basis. That is, it prevents the consumers from realizing the benefits of short-lived spot prices.
While only solving the storage part of the equation, checkout projects like Sia Coin[0] and Storj[0]. Both of them aim to provide hyper-competition for storage space, through the use of blockchain technology. I think a future, where multiple blockchain projects provide access to computational resources, isn't too farfetched.
Deutsche Börse created such a company a couple of years ago. Didn't go well, was shut down last year I think. I don't know how thorough the implementation was though.
This is not multi-cloud as it is commonly defined: Simultaneously handle workloads with multiple cloud vendors, to prevent lock-in to a single vendor. Rather, this is passing workloads between AWS and Google to leverage the advantages of each. A useful strategy, but the title is a bit misleading as it goes against the colloquial definition of multi-cloud.
I know you can use multi-cloud to prevent lock in, but I've never heard the term used to strictly mean that. Almost everyone I work with uses the term multi-cloud as "workloads on multiple cloud vendors."
this doesn't solve the biggest problem with multi-cloud which is data egress though. Even if latency is acceptable you would get killed on data transfer for any significant amount of data.
I think you're right: the headline on the article is bad/awkward in terms of how most people use multi-cloud. I get the impression that Google uses this term internally very differently. I also think you're right that they are leaving out the not insignificant detail about egress pricing.
In the context of this article, I think what they are pitching is the idea that, if you are already running services on AWS, you can still leverage some of the value-added service provided by GCP (e.g., their Vision API) by fronting them with Google Cloud Functions.
So, for example, let's say you built, and are running, an ecommerce platform on AWS. Now you'd like to leverage some of Google's Vision services to make "visual recommendations of related products". You already let customers upload product images via S3. Now you can have a Lambda function forward that image to a GCP CloudFunction, have it get processed by your visual recommender system, and spit back out a list of SKUs that have a visually similar image.
I think the goal is to convince companies that you don't have to put all your eggs in one basket in terms of capabilities, rather than convince them to have availability/capacity between multiple clouds (which how I think most people would interpret the headline).
Exactly. Even moving data between services in AWS or Azure can become costly if you aren't managing it properly. Moving that data to another cloud provider (images and video) would be a tough sell considering that both AWS and Azure have similar services already.
Really depends on the business model. I modeled different deployment scenarios for a product for a small company, and egress pricing broke the economics for the ones involving cloud providers. (It was principally concerned with hires images.) That was some time back and I know pricing has come down, so that may be different now.
That sounds kinda like the CDN example in my case :) I guess you could amend it to say "any kind of large media files"
But even if your business does do a lot with huge media files... no reason you can't do 95% of your business on AWS or GCE, and then just offload the giant file download piece to some unlimited bandwidth server you rent.
As Google seem to be getting more serious about attracting aws converts and dual cloud deployments (eg in built aws vpc peering) I wonder if they will offer some kind of cut price data transfer to aws networks. That would be an interesting move.
A (potentially) better approach to going multi-cloud is to use a CD tool like Spinnaker (spinnaker.io), which Google and Microsoft both support - and Netflix supports the integrations to AWS.
(full disclosure: my startup, Armory.io, builds enterprise features on top of Spinnaker)
You’re right, however this article is not talking about deployment tools. It's about setting up a serverless/lambda runtime pipeline between AWS and Google Cloud. As far as I know, Spinnaker does not do anything for your application runtime.
why would Google write a blog post about AWS lambda while they have cloud functions in beta?[1] why would i use google endpoint with AWS lambda instead of AWS API gateway?
Going "multi cloud" using AWS lambda with AWS API gateway and Google cloud functions with endpoints is a blog post I'm interested to read.
I don't know about google cloud endpoints, but AWS gateway is an absolute nightmare. Configuration is awful and communicating errors from backend to client is a travesty. The lowest of the low hanging fruit to compete with AWS on. If GEP is remotely coherent it will be a clear win.
Yeah, API gateway was written as some hack it seems (or maybe they used some third party ESB or a soa framework and slapped cloudfront on it). All the swagger extensions to configure it, these 200 status codes for 500 errors - its all barely usable.
Just take your own rest backend and put cloudfront on it and it you'll have more functionality and as much or more scalability and security, for an "api".
API Gateway PM here - Sorry to hear that. Would love to understand more details around the trouble you are having with API configurations. Please do reach out to us on our forums (https://forums.aws.amazon.com/forum.jspa?forumID=199) so we can assist further.
I suppose. The downsides seem large enough that I would try and find alternative implementations such that I could do it in one cloud. Or ar least something that highly minimized cross provider dependencies and expensive data xfer.
You're fighting the purpose built intention of the cloud to make it costly and clunky to move data out.
Seeing all those services hooked together gives me the willies. It sounds like a right PITA to debug, and chaining them together in serial for a single request multiplies the chance of an error even if each individual node is pretty stable.
I guess it's okay for a demo though. They're probably not setting an example so much as trying to throw as many services together to show off the technique.
Ya the only way I could ever see this being maintainable is if it's declarative. It takes a tremendous amount of boilerplate and configuration to perform what comes down to a one liner in any functional language or a formula in any spreadsheet. I think I would be more interested in a transform that converts a series of goroutines to lambda functions or something.
You know, architecturally, at application level no one wants to deal with this but devs orchestrate nightmarish dependencies for deploys and hit 'endpoints' with aplomb
in service architectures with only a playbook or data blueprint to follow. What's the point of fighting this?
Sort of. "Spread Critical Workloads" is somewhat silly to me. The uptime will be lower than the worse of the two clouds, complexity higher, and cost higher due to egress for cloud to cloud.
I do get the use case for actual multi-cloud, where there aren't cross cloud dependencies.
This setup seems unwise though. Maybe as a short term migration pathway, or similar.
1. The developer sets up some requirements: number of instances, X amount of CPU power, Y amount of memory, Z amount of storage, B amount of bandwidth, or other characteristics. These requirements are sent to the third party.
2. Cloud vendors (the "bidders") receive the characteristics needed by the developer. Each of them bid by providing a price per hour for the given characteristics.
3. The bidder with the lowest price wins the auction, sends back credentials to start deployment
4. The developer can start the deploy with the infrastructure provided by the lowest bidder!
All this may happen in less than one second, similarly to the auctions that happen when advertisers bid for displaying an ad when someone visits a webpage. This would need a massive standardization across cloud vendors, a system of penalties when the infrastructure provided by the bidder does not satisfy the characteristics, etc.
This does not too far fetched to me. It also make it easier for new cloud providers to start selling, and lower prices for developers.