6

This is a two-part question.

  1. As an UX person and a data nerd I am missing an information regarding users performance in UXSE: a ratio between answers and upvotes the answers get.

For the sake of this question, we could call it a "user's upvotes index". For instance, my UUI is 2.6 (2.6 upvotes per answer in average).

I would like to see the UUIs in the little stats tooltip that appear on hover because UUI says so much more than just users' score — it shows who delivers adequate and helpful answers and who is gaining score by delivering as much answers as possible (which is a valid strategy in it's own right, I'm not criticizing, I just think the difference matters).

  1. However, most importantly, I'd like to see UUI's data because I am interested in the dynamics of all UXSE users average UUI — I'd like to validate my idea (more like a feeling) of UXSE community reacting to answers differently than, say, a year ago — the infamous "decline of the upvotes" we have discussed earlier.

So, in short, I suggest to conduct an UXSE user research ;-) Let me know what you think.

4
  • 1
    I wouldn't consider only upvotes per answer but score including downvotes. UX is not like, for example, SO but there is still some discrepancy (imo)
    – Adriano Repetti
    Dec 29, 2015 at 8:36
  • Good point. My hypothesis is about people not doing any actions regarding answers, including upvotes, downvotes and accepting, so maybe all indices should be counted separately and then calculated by several formulas.
    – Zoe K
    Dec 29, 2015 at 8:44
  • Hm, most of my activity on the entire network is upvoting and, if I have enough karma on the site, downvoting, with the occasional comment; rarely is something already entirely unsaid. This is kinda bad, it happens to others to that are experts in a field but don’t even get to 1k karma… though I suppose that, when they answer, they don’t often get upvoted, because it’s not popular?
    – mirabilos
    Dec 29, 2015 at 16:58
  • 1
    Related feature request on the main meta: Show average vote score per question and answer
    – Graham Herrli
    Jan 13, 2016 at 17:53

3 Answers 3

Reset to default
3

An interesting idea, but I believe it would not be as accurate as we want to. At first, this community is constantly growing which makes the number of users increase at a steady pace. The effect is that great answers four years ago have less up-votes than great questions posted today. If you take a look in the archives, you'll see what I mean.

Second the number of up-votes doesn’t reflect the quality of a post as we hope to. It measures the popularity of a post. Being a member for almost five years this is what I’ve come to learn. I’ve written some really great posts (bragging I know) and some not so great posts. My latest, not so great post, have 227 up-votes (and 1 down-vote) just because I was the first to answer a question that later on turned out to be a hot network question. The post has two sentences from me and the rest is citation from another website. Not so great.

One of my greatest post have 9 up-votes. It’s from 2012 addressing UX and IA competence differences. I have a bachelor degree in IA which makes me more than qualified to answer that question. Still only 9 up-votes. Other users here have the same experience in terms of up-votes.

Now when we know that votes are based on popularity rather than quality, this has another unwanted effect. Complex, well written questions and answers that isn’t popular and often on a topic unknown to the majority have less chance of getting a lot of up-votes. UX-users specialized in these fields don’t get the rep they deserve. Just because their posts aren’t popular.

Adding a ratio wouldn’t add a quality stamp, but it would add a popular stamp to users. And I think we already have that in rep count by week/month/quarter/year and all.

Pro Tip: To gain reputation fast, make sure your posts are popular, that everyone can relate to and that divide the audience into two different mainstream views. Try to get the post on the hot network question list and maintain the post smart. Activity pays off.

Disclaimer: I have posted a lot of low quality questions and answers too :-)

4
  • 1
    Great answer which was pretty entertaining to read (I take it, you DO know how to write popular posts ;-). As for the dynamics — yep, sounds legit, apart from the time stamp data issue. Thanks for the advice, I'm not sure I can pull the fame off, but I'll sure keep it in mind ;-). I guess any strategy is good if it fits your personality. Cheers!
    – Zoe K
    Jan 6, 2016 at 9:53
  • Yes, the post you said is best is really good. Why do you think it's best — is it the rarity of the expertize, or the post itself? I sort of like your popular post about the sound being annoying, because it boils down what pretty much is an universal opinion that so many people share, unlike IA problems. If expressed in terms of popular neuroscience, your popular answer lies in the lizards brain area, while your IA answer is quite cerebral. And there are more species that share the lizard brain than those who sport a cerebellum.
    – Zoe K
    Jan 6, 2016 at 10:00
  • 1
    @ZoeK It took a lot of more effort to produce that post than the sound-post. Thanks for getting the essence of my answer :-)
    – Benny Skogberg
    Jan 6, 2016 at 11:12
  • Well, it was well put :-). Guys on top got lots of upvotes, so I'm going to check your answer, although it's not really a kind of question where the right answer could have been possible.
    – Zoe K
    Jan 6, 2016 at 12:51
7

Here you go, here's a list of all the top UX.SE users by average answer score (it's a rough and very slow query cobbled together from others already in the data explorer, but hopefully it serves its purpose).

As you can see the data is very skewed by single answers with very high point totals (the median score which I've also listed in the query seems to be a more interesting number, frankly).

6
  • Wee! Nice New Years present, thanks Kit!
    – Zoe K
    Dec 30, 2015 at 13:09
  • I will need a different kind of data comparing two samples for two periods of time, but thanks nonetheless, I didn't know about the Data Explorer!
    – Zoe K
    Dec 30, 2015 at 13:14
  • 1
    You can change the query yourself to filter by post date etc. but I don't think the data exists to see what a post's score was at a certain date (so older posts will naturally do better because they've had more time to accumulate votes)
    – Kit Grose
    Dec 30, 2015 at 13:16
  • Hm, you got a point.
    – Zoe K
    Dec 30, 2015 at 13:18
  • 2
    And here's a list of all SE networks, in descending order of votes per post (warning: it's somewhat expensive, because each site is stored in its own database). You can see that UX is slightly above average in terms of both its VoteRatio and UpVoteRatio, but its DownVoteRatio is fairly low, putting it in the top 20% by Up/DownRatio. This may not help evaluate performance over time, but if there's been an "upvote decline" it just means that UX used to be even further from the mean than it is now.
    – Dan Henderson
    Dec 31, 2015 at 0:42
  • 1
    Great work, kit! I have my suspicions you're correct. I don't see anything particularly unusual in terms of voting myself, some peaks and troughs but nothing I'd call a persistent decline. Biggest change we've made RE: upvotes is probably being less accepting of voting-happy list/joke/etc questions.
    – Ben Brocka
    Jan 1, 2016 at 19:51
7

Before reading my rather negative reaction to this suggestion, please bear in mind:

So, in short, I suggest to conduct an UXSE user research

But I think that your concrete suggestion of a user tooltip is not constructive, and overlooks some serious negative side effects of such a feature.

I would not like to see this data in the little tooltip. Data is great, but it should be placed where it has a purpose. Moreover, data shouldn't be placed where it leads to undesirable side effects.

People have a weird relation to KPIs (I'm using the business consulting term here on purpose). They read into them much more information than they actually contain. They imbue them with a lot of meaning. They start making moral judgments based on them ("What should I care about that idiot, he only has a UUII of 0.8, I can safely ignore him and it would even be OK to ridicule him"). They also change their behavior to optimize the KPIs into a direction they find desirable, regardless of how this differs from the behavior which would actually be optimal for the system's functioning.

If you wave the UUII in people's faces all the time, you'll get people doing whatever they can to make sure theirs is higher than that of the neighbour's. They will stop giving answers unless they think "this is sure to gather tons of votes". They will still have something to say though, so the lamentable tendency of "answers in comments" will grow, undermining the system and placing more burden on moderators. They will start going through old answers which did not get much attention, and deleting them. There will be tons of good answers which are entirely worth keeping, but got a score of 0 or one because the question is too obscure, because of the "fastest gun in the west" phenomenon, or whatever else. And this is just the most obvious consequence. I'm certain that hundreds of users with the time and motivation to game the system will come up with new and creative ways to increase this number without increasing the quality of their answers.

In short, such a feature will undermine the community aspect of the site and have negative impact on its main goal of providing a large quantity of high-quality answers to questions relevant for its users.

If all you want is to play around with the data and see if it tells you something interesting, that's cool. Stack Exchange gives us the tools to do so, and Kit Grose's answer produced a data set which seems to be good for this purpose. If you end up with interesting findings, you're welcome to share them with us.

In short: research is a great thing, please do it! You just have to keep in mind that some research methods are unsuitable for use because they interfere with the routine functioning of your subject. Choose a method without such side effects, and we will love seeing your results and be grateful for your contribution.

1
  • Actually, I wanted a discussion, so I appreciate everybody's opinion. If I will be able to scrape the data, I'll play with it, although Kit just coined a point that it may be impossible to look up the upvote post stamps since they may not be captured. As for your UUI point — it does make sense, I didn't think of it but certainly I wouldn't want to create a bias on the base of it. It's good I didn't add the "feature request" tag. :-)
    – Zoe K
    Dec 30, 2015 at 14:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .