Crunching numbers is most definitely abstract. What is a number? Why does it matter? What does it represent? What is zero? Why is zero important? Think about how long it took for humans to actually start doing math. And then think about how difficult it used to be for people to grasp negative numbers. Sure, schools teach things with M&Ms and counting other concrete things, but you still have to think abstractly to apply the same concepts to other concrete things.