This has been tried a dozen times. The chicken and egg problem is especially hard in real time. And in general, chat is dead. It's boring. The web has more to offer than it did when chat rooms were popular.
Chat is NOT dead. See campfire for example which even is a quite new chat tool. In fact, entire companies (including my own) are using chat for all their internal communication and were doing this for years. Maybe the anonym chatroom for teenager in the 90s is dead, but that is for good reason. The Chat as a tool itself is still really powerful and effective. (and widely used)
The implementation of rabblr also is really cool. But i think the part with the anonymous users might be a problem. I just logged in for a few minutes and it was full of trolling. Also that you can't carry the chat over multiple pages is anoying, but there might be a way around that.
Plus i really don't want to talk to random people on webpages. I probably want to talk to someone who can give me an information that i want, which would be a site adminstrator or company staff or whatever. Like olark.
Or i want to talk to my friends or people i know, but i can do that on any IM that i got open anyway.
So to sum it up. Nicely done, but i also doubt the usefulness of this tool.
Hence the "in general". It still has its place for internal company tools. I often use IRC for technical help. Still, chat as we knew it, as some people still think of it, is dead.