“Scientists start out doing work that's perfect, in the sense that they're just trying to reproduce work someone else has already done for them. Eventually, they get to the point where they can do original work. Whereas hackers, from the start, are doing original work; it's just very bad. So hackers start original, and get good, and scientists start good, and get original.” - Paul Graham in Hackers and Painters
BTW: While information theory is everywhere, I have to yet see where measure theory makes a practical impact on practical deep learning. The importance of pure math for practical machine learning is highly overrated (and I speak as someone who did study that).
Which, well, I use as an opening quote to my intro to deep learning, https://github.com/stared/thinking-in-tensors-writing-in-pyt....
BTW: While information theory is everywhere, I have to yet see where measure theory makes a practical impact on practical deep learning. The importance of pure math for practical machine learning is highly overrated (and I speak as someone who did study that).