I don't understand sinhala but if it is classified as Indo Aryan language and not Dravidian, then that is the reason.

There will be always be some influence and loanwords from neighbouring languages but comparative linguistics can tell origin / family of the language pretty accurately.