Is anyone here able to offer an explanation for why our brains are able to do really complex tasks without using much energy, at least compared to AI systems?
The brain relies on discrete, sparse events in space and time to handle computation.
Most of the computation and learning that occurs is attributable to the relative timing of spiking events. A lot of information can be encoded in the delay between 2 spikes. The advantage of biology is that there is no explicit quantization of the time domain that must occur. Biology gets to do a lot of things "for free". Simulating causality in a computer in a similar way requires a priority queue and runs like ass by comparison.
The way neurons and synapses work you spend a lot of energy keeping them ready to fire. How often they actually fire is a smaller cost compared to maintaining them in ready state.
We end up using 100W (2000kcal/day) for the whole body, or about 20W for the nervous system alone (though a nervous system alone wouldn't be able to survive). That's comparable to what a modern laptop uses. Sure, that laptop can't run a large LLM at any reasonable speed, but it can do basic math far better than my brain. By a comically large margin. Just a consequence of the very different architectures chosen
Current AI systems aren't biomimicry; they run a simulation of something vaguely similar to neurons. This is rather like "why does it take more processing power to emulate a PS2 than the original PS2 had".
Why would they not be? A brain and a computer are completely different things. They don’t do the same thing and they don’t work the same way at all.
"Artificial neuron" was a useful metaphor at the beginning, but they really are a very simplified model based on what some people understood of neurology back then. They are not that useful to get insights into how actual neurons work.
Because computers use digital circuits which are not allowed to make mistakes, i.e., they amplify each signal during every step as it passes through the system.
My thinking was that its sort of like an engine spinning at idle vs when the gear's engaged, in either case the engine is still spinning and using fuel, just more so when its engaged, as opposed to an electric motor.
I know the metaphor isn't exact, it's just how i thought of it.
If I'm not in flow state focusing on some programming problem, my brain is still going a million miles a minute pontificating about 10 different threads of nonsense at once. So I could see where focusing on one task doesn't actually burn any more energy, it just pulls in all those other workers and puts them to work on one thing.
