It's also dramatically more efficient and safe to have a quickly readable visual symbol. Maybe that's just my opinion but it seems reasonable to assume it factored into the design.
There's also the fact that it's understandable when you're looking at it the wrong way. It's useful to gauge who should stop and who has right of way.
For example, in France (and most of Europe, I think), the default is for traffic coming from the right to have right of way. So if you're coming from the left and see the back of the sign, you know you don't have to yield.