I agree that it's not easy for some. I watched a group of intelligent people who'd forgotten all their high school math try and learn programming for 9 weeks and it was unproductive. Math builds abstraction upon abstraction, and programming does too.

There was a study two years ago showing that learning maths literally rewires your brain (noticeable on scans)... so it's not just a matter of adding some more knowledge - it's a different way of thinking.