> Maybe there's something weaker than transformers, but good enough for practical applications, while being more resource-efficient?

I wish! Gboard is terrible at predicting the correct form of a word.

It generally knows the best word, but it doesn't get from context that (e.g.) a gerund form of the verb is the most appropriate next