JamKazam allows musicians to play with other musicians over the internet in real-time.
There are a number of challenges we've faced so far, but the biggest two are latency and audio gear setup. Latency doesn't only come from the internet; your audio gear and OS can also add quite a bit of delay as well. Our approach with latency is to measure well, and help users reduce latency where they can (and of course optimize our own software to add as little latency as possible).
And we continue to refine how you hook up your audio gear to the JamKazam PC/Mac application. Some gear works great. Some... gives us quite a bit of trouble.
Anyway, the product is usable, although we have much, much more to improve on.
If curious, we are using a RoR w/ Postgresql hosted mostly on Linode. We are using Websockets + RabbitMQ to help route messages between clients and browsers. We communicate over websockets for control messages when initially establishing media between two parties, then ultimately going directly P2P with UDP audio to help with latency; hair-pinning audio through the server is not your best bet.
This is realtime, right? I still can't believe it. How did you manage to reduce latency enough to be manageable? Even the slightest latency kills performing for me.
I always thought you could go A -> B -> C, but A only hears herself, B hears A and B, etc. Do all musicians listen to each other?
Does this need particularly good internet connections? What is the acceptable latency between peers?
If you look at university research papers that have studied this domain, and if you look at first-hand experience of watching musicians play with different amounts of latency, they can tolerate just a little latency. From research and experience, we'd say 0 to 20ms of one-way latency between musicians yields a tight session we'd label as "green". This is like playing in the same room. In the 20ms to about 35ms range, you start really feeling/hearing the latency more, but it's still manageable and can yield a fun and productive session. Somewhere up in the 35-40ms range and greater, it gets increasingly sloppy until it's just not fun or worth it . These are general rules of thumb. It also depends on the tempo and the level of precision required. And some musicians are more or less sensitive to it. But if you consider the speed of sound, playing with 20ms of latency is like playing while sitting 20 feet away from someone on a stage or in a room.
JamKazam streams all musicians audio to each other and mixes the streams from other musicians with each musician's own stream back into his/her headphones. So you hear everyone, including yourself, mixed together. As long as latency is reasonably low, it feels like you're playing with others in the same room with your eyes closed. If you check the videos of various bands on the JamKazam YouTube channel, that's what they were all doing.
You don't need a lot of bandwidth, but you do need a solid connection. We recommend uplink bandwidth of 1Mbps to support a session with 4 musicians. You definitely want to connect via an Ethernet cable to your home router, as WiFi routers tend to bog down due to packet rate, not bandwidth. And cable and fiber broadband works best. DSL typically has too much packet loss, and satellite too much latency.
A cool service built by some great folks out of Austin. It's pretty amazing how they've been able to shrink the latency to the point where many musicians don't notice the difference. They did a jam with several bands at SXSW this year. You can hear the recordings here: http://www.jamkazam.com/events/virtual_jam_fest_2014.
I know these guys personally but am not involved with JK and am very impressed with what they've built. I always thought physics would play a cruel joke on them, but so far they've been able to make it happen!
There are a number of challenges we've faced so far, but the biggest two are latency and audio gear setup. Latency doesn't only come from the internet; your audio gear and OS can also add quite a bit of delay as well. Our approach with latency is to measure well, and help users reduce latency where they can (and of course optimize our own software to add as little latency as possible).
And we continue to refine how you hook up your audio gear to the JamKazam PC/Mac application. Some gear works great. Some... gives us quite a bit of trouble.
Anyway, the product is usable, although we have much, much more to improve on.
If curious, we are using a RoR w/ Postgresql hosted mostly on Linode. We are using Websockets + RabbitMQ to help route messages between clients and browsers. We communicate over websockets for control messages when initially establishing media between two parties, then ultimately going directly P2P with UDP audio to help with latency; hair-pinning audio through the server is not your best bet.
You can check out some videos...
Musicians jamming: https://www.youtube.com/watch?v=I2reeNKtRjg
Overview: https://www.youtube.com/watch?v=ylYcvTY9CVo
Getting Started: https://www.youtube.com/watch?v=DBo--aj_P1w