> I don't think anyone would even go as far as to include all deep neural networks which are indeed a large scale collection of neural networks as being "100% deterministic"

Any other position is pure magical thinking. Adding more linear algebra never gets you out of 100% determinism. It gets you more complexity, and potentially makes systems chaotic, but chaotic systems are still deterministic, just highly sensitive to small input variations.