Because the time wasted by all the threads trying to accept() or read() on the ready socket introduces latency for all other sockets. And since throughput is bounded by the number of threads divided by inverse of latency, increasing the latency makes you lose in scalability.

In addition, because a single "readiness event" has to wake up many threads, it can introduce lock contention and cacheline bouncing in the kernel's data structures.