Downvotes turned on for me a little after 500. I'd imagine it's a flag in your profile that's flipped in a batch operation once to a few times a day. Either way, you're not missing much - like another commenter here I wish I could just turn them off.