It's possible that this is generally true, but that's definitely not my perception. Growing up I remember hearing about stories like this [0] where people camped out in a church for over a decade to try to stop it from being closed. As someone who isn't religious, I still can't understand the level of attachment to the building itself, and it's hard to imagine that this would happen from people not ascribing some sort of religious significance to the building itself. That said, I know that there are some notable differences between Catholics and other Christians in terms of veneration, so maybe this perception is just due to my growing up in an area where other types of Christianity weren't as common, and elsewhere in the country where Catholics aren't as common, the generalization would hold more true.

[0]: https://www.cbsnews.com/news/massachusetts-church-protest-11...