To me that's a complete circle back to lessons learned from neural networks - I long suspected we'd be heading that way again with transformers being a computational step rather than the whole thing.