I found it helpful to read through Think Stats and Think Bayes before tackling a machine learning book.

[1] Think Stats: http://www.greenteapress.com/thinkstats/thinkstats.pdf

[2] Think Bayes: http://www.greenteapress.com/thinkbayes/thinkbayes.pdf

As someone who has zero calc training nor linear algebra (some discrete mathematics was all I took in University), what are some recommended start point to most quickly be up to speed to digest the resources posted both in the OP and by other commenters in this thread? Just a bit of background about where I am at math-wise: I tried taking Andrew Ng's ML course, and quickly fell behind starting with the second programming assignment (it was implementing a linear regression algo, I believe).

Andrew Ng's Machine Learning course at Coursera, week 1 contains about 1 hour Linear Algebra review: lectures on vectors, matrices, their multiplication, transpose and inverse.

So do you think these lectures are not enough to bring one up to speed in applying these concepts in linear regression?

Of course, a formally educated person has taken a full semester of Linear Algebra, and solved dozens of homeworks of "transpose this", "invert that" etc. so it's difficult to guess how much homework of the boring kind would be needed before one is able to apply these concepts in problem solving.

The first few chapters of

ET Jaynes: 'Probability Theory: The Logic of Science': http://bayes.wustl.edu/etj/prob/book.pdf

Are great (and free) as a thorough introduction to bayesian reasoning.

Actual book is here (warning 13 MB pdf):

http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...

Was delighted to see a notation list as the second page in the book.

MacKay's Information Theory, Inference and Learning Algorithms: http://www.inference.phy.cam.ac.uk/mackay/itila/

Elementals of Statistical Learning: http://www-stat.stanford.edu/~tibs/ElemStatLearn/index.html

The best advice i can give is to go through this video, it was fun and really helped me a lot. http://pyvideo.org/video/608/bayesian-statistics-made-as-sim...

Are there books on practical machine learning? The math is fine in these books, but does not address the practical side: data analysis, pre-processing, on-line pattern recognition, etc.

Not a book, but Andrew Ng's coursera course (or Stanford class video lectures) are great and have lots of practical tips.

Try Segaran's Programming Collective Intelligence.

[For the really lazy]

PDF download link:

http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/270212.pdf