> In the two examples above: When you upvote a comment, the computer thinks that the other comments you have read (the blue ones) but did not upvote, are not terribly interesting.
If you upvote a comment as interesting and the not the ones leading up to it, then you are punishing that comment because you're punishing its predecessors in the threaded chain, so it will be less-seen because you upvoted it.
Any other assumptions of which comments are read will probably not end well. Some people may skip to comments to get someone's personal opinion, other people may only read shorter comments.
~~~
It's worth noting that HN has an interesting algorithm that does not exactly sort by comment score. I have a very high average karma, and when I post on a topic, even if the topic already have 60 comments, my comment begins at the top and stays there even if I'm not upvoted. I suspect that the sorting algorithm is doing something like implicitly adding average karma to the karma of each post when determining order. Therefore it doesn't sort by "best", but by "well regarded".
This is good and bad. It's good because if a person who usually makes high quality comments comes along late to the game, their voice will be heard. It's bad because it creates its own rich-get-richer problem.
I think a better solution to the problem is to simply:
1) Hide upvote numbers like HN is already doing[1]
2) Dither the comments between "new" and "well regarded". First comment is the newest, second is the most well regarded, third comment is the next newest, etc. Perhaps randomize which category comes first.
~~~
[1] Of course hiding upvote numbers has its own major problem because sometimes the hard number of "I agree" upvotes is important. When you want to "Ask HN", say, what particular frameworks people can vouch for, you have no real way of evaluating the comment responses in terms of people that agree. Since they aren't even sorted from best to worst, the information you have is somewhat dim. This could be remedied by giving submitters the ability to turn on comment scores for only their thread. (Or the ability for mods to do it)
I think the issues you mention can be addressed. I've outlined how, below:
1: > """If you upvote a comment as interesting and the not the ones leading up to it, then you are punishing that comment because you're punishing its predecessors in the threaded chain, so it will be less-seen because you upvoted it."""
That's a good point. I'm not sure if it is an issue, however. Not upvoting a comment would have very little impact — it wouldn't weight as much as a downvote, for example. So I think the effects wouldn't be that bad. (A comments score could be: `(upvotes - downvotes) / number_of_people_who_read_but_didn't_upvote`
and adding +1 to `number_of_people_who_read_but_didn't_upvote` will have only a tiny tiny effect.)
Anyway I think the issue can be addressed like so:
If there is a thread with very interesting comments somewhere in it, prioritize that thread a little bit — so it'll get a score that is a little bit better than the very first comment in it (the comment that starts the thread).
I think SlashDot does something reminiscent of this? — Sometimes, [a mediocre comment that starts a new thread] is collapsed, but [comments deeper inside the thread but with many upvotes] are shown in full.
Alternatively: Modify the algorithm: Don't punish ancestors, only siblings.
2: > """Any other assumptions of which comments are read will probably not end well. Some people may skip to comments to get someone's personal opinion, other people may only read shorter comments."""
I think it's okay that the algorithm makes mistakes sometimes. It only needs to work well on the whole, given data about many visitors. — On mobile phones, however, where only 1 comment is shown at a time, it'd be easier to know what the reader is reading.
Also keep in mind that the examples in the article were intended as examples — if there are issues, the algorithms can perhaps be modified to take them into account. For example, your example visitor that reads only short comments, would be more likely to upvote only short comments. So the algorithm could then "realize" that "oh, this visitor only reads short comments. I'll take that into account" :-)
> This is good and bad. It's good because if a person who usually makes high quality comments comes along late to the game, their voice will be heard. It's bad because it creates its own rich-get-richer problem.
This is also bad because, if true, it creates a strong disincentive to comment on less popular stories because, regardless of the quality of your comments, doing so would lower your average karma.
Perhaps normalize by dividing each of your comment scores by the median score of other replies to the same parent? Then average these normalized scores.
FYI I think all comments begin at the top. My comments also start at the top and then fairly quickly move down into upvote order, my karma is well below the top of the leaderboard.
Yes, I believe HN comment sorting uses the same algorithm as news items, where the score of an item is based on both votes and time, decaying over time.
I've ruled this out as a possibility. For instance here I was the 4th comment (there are now 16), I posted 28 minutes ago and I'm still at the top comment slot at only 1 point.
It must be more than just a function of time and score, otherwise any of the new 1-point comments would have superseded mine, but they didn't.
Your average karma is 24, so your comments are weighted much, much more than others (it's one of metrics HN uses to rank comments). Also, your comment is long, which is also a factor (though, not a big one).
If you take comment length + karma per post / comments on post you should end up with reasonably decent comments making it to the top. Ish. It's still never going to be able to fully deal with all the corner cases on a site as large as HN but it's often Good Enough.
It would make sense if your comments were weighted higher having a higher karma, but I think all of our comments start at the top to make sure that at least someone sees them and that they can get upvotes if they are insightful/interesting.
You wouldn't create a top level comment if you weren't hoping to contribute and for it to be read. For it to be more likely to be read, it's better off at the top. So yes, I'd say that people care.
Why comment at the top level in this sort of discussion then?
The topic isn't so much poll position but giving every comment a chance at being noticed so that the better options rise naturally to the top and create a better conversation.
I don't think this is the "I don't care about silly internet points" discussion you think it is.
Well, I personally like to see the most interesting comments at the top. But that creates the inherit problem - the first interesting comment gets stuck at the top and prevents all the newer ones from being seen.
For HN specifically, collapsible comments would make a huge difference. A top-level comment in position one can have dozens of comments under it, making the next comment (which is perhaps only 1 upvote lower) appear far, far down the page.
If you could collapse the top level comments, this problem would evaporate.
This is exactly why HN needs collapsible comments. Not so I can collapse them for my convenience, so that stuff doesn't get buried so easily. Sometimes the top thread is all there is on the first page ... for example, there was this great blog post by raganwald a while. Here's the new version [1] since I assume the posterous one won't be around much longer. And here's the HN discussion of the original [2] in which, as he noted, a comment about IQ testing (completely tangential to the post) took the whole first page.
In that case, you would collapse the top comment chain and be shown nothing underneath, since HN only shows X amount of total comments before requiring you to click More.
I think HN would have to increase the total number of comments per page before allowing collapsable chains.
That's true, once it got that big, but I think the hope is that it wouldn't necessarily remain the top comment under those circumstances. Allowing people to collapse the thread, before it took a whole page, would allow them to see and upvote the other comments if they wished.
HN and reddit should randomize comment placement based on the poster's karma, the ranking of the comment with some notion of number of readers and/or time to normalize the expected value. Ideally, the algorithm would also keep track of how well comments due in their temporarily elevated positions.
> In the two examples above: When you upvote a comment, the computer thinks that the other comments you have read (the blue ones) but did not upvote, are not terribly interesting.
If you upvote a comment as interesting and the not the ones leading up to it, then you are punishing that comment because you're punishing its predecessors in the threaded chain, so it will be less-seen because you upvoted it.
Any other assumptions of which comments are read will probably not end well. Some people may skip to comments to get someone's personal opinion, other people may only read shorter comments.
~~~
It's worth noting that HN has an interesting algorithm that does not exactly sort by comment score. I have a very high average karma, and when I post on a topic, even if the topic already have 60 comments, my comment begins at the top and stays there even if I'm not upvoted. I suspect that the sorting algorithm is doing something like implicitly adding average karma to the karma of each post when determining order. Therefore it doesn't sort by "best", but by "well regarded".
This is good and bad. It's good because if a person who usually makes high quality comments comes along late to the game, their voice will be heard. It's bad because it creates its own rich-get-richer problem.
I think a better solution to the problem is to simply:
1) Hide upvote numbers like HN is already doing[1]
2) Dither the comments between "new" and "well regarded". First comment is the newest, second is the most well regarded, third comment is the next newest, etc. Perhaps randomize which category comes first.
~~~
[1] Of course hiding upvote numbers has its own major problem because sometimes the hard number of "I agree" upvotes is important. When you want to "Ask HN", say, what particular frameworks people can vouch for, you have no real way of evaluating the comment responses in terms of people that agree. Since they aren't even sorted from best to worst, the information you have is somewhat dim. This could be remedied by giving submitters the ability to turn on comment scores for only their thread. (Or the ability for mods to do it)