I agree with you. A good example of this is what we experience when driving. If someone cuts me off, my instinctive reaction is to think, "What a moron!" Which is unfair, of course: it may have been a one-time stupid mistake.
Even if you think you see a pattern, someone who seems stupid at something may just be outside of their comfort zone/talent area. In another field, they may be quite smart.
Another example would be when a developer who build a framework at an organisation has left the company.
90% of the time, when people look at the code, you will hear comments like 'who wrote this garbage??'. In reality, the system was generally built to the best knowledge / requirements that were available at the time.
(I'm sure many people here have thought 'who wrote this crap' and then viewed the author to find out it was something they wrote many years ago, but have just gotten much better over time)
Even if you think you see a pattern, someone who seems stupid at something may just be outside of their comfort zone/talent area. In another field, they may be quite smart.