If I'm not mistaken, the difference is that in the case of JWT, your app manipulates the secret directly, so it must show up in clear form somewhere, from the app's perspective.

So, if the app host is compromised, the attacker shouldn't have too hard a time to extract the JWT and use it from somewhere else.

In contrast, with an HSM, the attacker would need to have the HSM sign any new connection attempt, which should be a bit more involved if it happens on a different machine.