We don't call it an ESB in our non-Enterprise environment. It's just called "service bus" or "message bus". Which is really just a fancy application-level API around services, queues, and messages (see http://particular.net/NServiceBus). It's incredibly useful, dare I say essential, for abstracting away those infrastructure level concerns when developing complex distributed applications.

We use it pretty extensively in our SaaS product to achieve loose coupling, yet high cohesion, between our various software components.

We use SOA a lot, we have a RabbitMQ server/cluster in the middle who all services communicate via. We have applications written in Ruby, Go, Clojure and Node.js, all publishing and subscribing JSON messages.

Benefits over monolithic: New languages/techniques can be introduced at any stage, and used for what's best for. Applications can be isolated, and only do one thing, making the system much more reliable. Your web front doesn't have to down because you have a bug in the mailer app.

Benefits over HTTP: Not everything is web facing, workers etc doesn't need a full web stack. Services can be offline, without interrupting other services. You get load balancing and work queuing for free. Performance, 5000msgs/s with disk persistence is no problem for a low end RabbitMQ server. Great inspection capabilities with RabbitMQ's management interface.

Tip: Get away from a "RPC" mindset, think "events" instead. Let other services subscribe to events they should take action to. Decide as much as possible up front, in the application that publishes events, so that information doesn't have to be returned.

Disclosure: I'm the owner of CloudAMQP (www.cloudamqp.com), RabbitMQ as a Service. But have used SOA and RabbitMQ a lot longer, both for small and large projects.

I've used Kafka, ActiveMQ, MSMQ and Tibco on long running projects.

An ESB can be incredibly powerful when you give a small developer team access to it and wait to see what they dig out of the data. We pulled realtime data about sales that no one had access to and it was enlightening for everyone.

When non-programmers use something like tibco, the results are abysmal. Slow overly complicated highly coupled mess that only one person will understand, the creator. When most people say ESB, this is what they mean and it is a tax on the company.

How will you manage communication between services in a SOA? If you exclusively use point-to-point welcome to spaghetti code.

Our philosophy is to use point-to-point when you absolutely need another service's input to complete your task. Otherwise put a message into the bus. See the Jay Kreps blog post I linked in below.

It's not spaghetti as long as your interfaces are well-defined, is it?

I can see the value in a service locator/registry, but that doesn't seem like a substantial piece of the ESB pitch.

When do you have a fire-and-forget case like you seem to be talking about? I can guess a few secondary concerns like logging, but most of the time if you're calling another service it's because you need a response from that service. At least for the apps I'm used to writing.

I would say "spaghettiness" is pretty much topology - not whether those interfaces are well defined or not, which is pretty much an orthogonal property.

i.e. You can have a poorly defined ESB and a big pile of spaghetti with well defined interfaces. Unfortunately I have seen both!

..."in a non-Enterprise environment" - yes...well..expanding into an enterprise; using BizTalk server (Microsoft).

I design and create SOA solutions through services exposed by/through BizTalk and build an ESB solution to orchestrate the services which acts as a substrate for 'composable' applications (again, using BizTalk and apps that use the services are web-based, or whatever client apps the organization wants to use; many apps are integrations between systems like line of business using various approaches that include EDI, etc).