I would say, that surely flags of an article to count (how much, I really am not sure). At least I would do that.
And I believe, that comments would also count a fair amount. An up-vote is a strong signal, but a comment is an even stronger signal.
I would (without telling anybody) even take the length of comments into account when weighting. I would at least try this and do an A/B-test, to see if this heightens the quality of articles on the front-page.
I would also use clicks on the link as an indicator, but one would have to be careful, as this would push link-bait articles to the top. So here one should not put too much weight into clicks.
Actually, the recent articles on HN's ranking algorithm suggest that PG considers comments to be evidence against an article being a quality article. Presumably that's why he penalizes articles with lots of comments.
He believes that the number of comments correlates with how controversial something is and wants to select against controversial topics. Anything with greater than 40 comments gets an automatic penalty.
I think a better way of doing it would be to get a large number of articles which are known to be controversial or uncontroversial and then use features from them (not just the number of comments) and try to develop a good predictor of how controversial something is. It would probably be a lot more effective than just arbitrary penalties for articles with 40 comments.
Though I kind of object to this stuff being done automatically at all. There aren't that many articles on HN that human moderation is impossible. As cool as automation is.
> Anything with greater than 40 comments gets an automatic penalty.
That's not quite right. Anything with >= 40 comments AND more comments than upvotes gets the controversy penalty. Both conditions need to be true. An article with tons of comments will have NO controversy penalty as long as there are more votes than comments.
For most articles, there are more votes than comments, so commenting will not harm the article's rank. (I wrote the recent article analyzing HN ranking.)
It appears he wants to correlate the number of comments with their contextual value, though.
A better indicator of how trollish a thread is and how much that's getting in the way of the conversation (because those are two different things - users may be willing to put up with vehement discussion in one instance but not another) would be the ratio of downvoted comments to comments. I can even see measuring the length of comments and the depth of a thread, as a deep thread with lots of tiny comments might be an indication of sniping.
I realize i'm probably being incredibly naive and that the whole thing is complex with multiple factors interrelating but I don't see how the ratio of upvotes to comments tells you anything useful, if it doesn't take into account that people may comment without upvoting, and people might upvote bad comments. I make certain to upvote the article now with every comment because i'm aware that not doing so, regardless of the content of my comment, is an implicit downvote.
I think it's also a mistake to correlate "controversial" with "unwanted" but that might just be a matter of semantics.
Agreed, the stories sunk include ones like the Anandtech review of the Surface Pro, and the site is quite reputed for being neutral, however the excessive number of comments combined with not too many upvotes can easily sink articles which are otherwise just news and this contributes to the filter bubble(funnily there's a anti-filter bubble post right near this article on the front page).
No, no. More comments than votes counts against the article. Articles which are punished by that are submissions that are more controversial than interesting.
Well that is quite an interesting signal. I hadn't thought of this. Looking at the ratio between upvotes and comments really is an interesting take on the matter.
I would believe, that PG had analyzed a lot of articles, to find a good ratio, that does penalize these kind of controversial topics, that he does not want to gain too much traction.
What I do not understand though is the fact, that controversial topics are penalized in the first place. But that is just me, one who loves to argue, to find optimal solutions (or to see, that there are really only two believe-systems at work).
Apparently it penalizes articles if number of comments > number of upvotes, so I doubt much analysis was involved - it's unlikely that it would result in such a simple ratio.
Yes, that came out in a thread where people were complaining that many positive/neutral posts about Microsoft/Nokia were being downranked and disappeared from the front page, while negative stories didn't usually have that problem leading to a chilling effect on the stories submitted and commented on. People seem to have got riled up over it because they thought it was being flagged, but PG said it was because of the flamewar detector that penalizes articles with lots of comments but fewer upvotes, atleast in that particular case.
The flamewar detector does its intended job with things like the gender discrimination stories but it looks like it throws out the baby with the bathwater and contributes to the echo chamber effect.
It's really hard to tell(except if you're pg or HN staff) whether a story disappeared because of the flamewar detector, or flagging or the domain greylist or something else. I guess this is intentionally so.
And I believe, that comments would also count a fair amount. An up-vote is a strong signal, but a comment is an even stronger signal.
I would (without telling anybody) even take the length of comments into account when weighting. I would at least try this and do an A/B-test, to see if this heightens the quality of articles on the front-page.
I would also use clicks on the link as an indicator, but one would have to be careful, as this would push link-bait articles to the top. So here one should not put too much weight into clicks.