According to Elon Musk the biggest single difference is the economic incentives of fixed priced bids (SpaceX) versus cost+ bids (traditional aerospace).

With traditional NASA bidding procedures there are very few to no incentives to contains costs. Military procurement has the same issue, combined with stringent requirements that preclude off the shelf for virtually anything. (Off the shelf is not, of course, an option for SpaceX. But if it was, they would likely consider it!)

NASA's definitely hampered both by organizational and political issues that SpaceX isn't, but as other posters have pointed out NASA has also managed to do some amazing things on a comparative shoestring budget in recent years and -- unlike SpaceX -- these are things no-one has ever done before with relatively few failures.

But then one has to wonder why NASA's technology has not improved during the 37 years that came after 1975 despite a budget of over 10 billion dollars a year.

I guess its really trivial to point this out which is why it isn't being said, but they did send rovers to the Mars, and they kept the ISS manned with the shuttle program, all the while under orders to move away from manned missions.

This isn't a comprehensive list by any means.

~~Also - the cold war ended, the symbols are now eroding.~~

NASA does not get 10 billion a year just to explore the moon. They had and have other priorities. Why would they develop a new-and-improved shuttle when space exploration was basically canceled?

Because NASA has no incentive to become more efficient. When you are a national organization the incentive is to keep your budget as high as possible and spend as much as you can to prove that you need the money, every year. That is precisely why such businesses are ripe for disruption, because they were never built for achieving a lot while spending less. If we ever see Space Tourism develop, it will not be through NASA operations, that is for sure.

NASA's mission is not to efficiently commercialize space technology. That's the sort of thing best left to the private sector.

That doesn't explain why other companies still have much more expensive rockets, though. As far as I know, SpaceX has the cheapest ones by far.

The other companies are in general legacy NASA contractors or partnerships of the same. They are probably stuck in there ways they are essentially offering the same technology that NASA paid them to develop all these years.