Totally agree. A misleading indicator doesn't really provide a benefit in my opinion. In fact I commented such against the linked meta stackexchange post, and it was responded to:
Notify users of possible reviews on toolbar
This is just confusing me. If the purpose of the indicator is to say 'stuff needs reviewing' then just have an indicator / icon or something. Don't add a meaningless number into it. We don't have a number inside the mod-alert indicator, or in the Inbox indicator, they just have a status - 'things in here need reviewing' or 'nothing'. – JonW ↵ May 7 at 13:55
@JonW At least the number changes and gives you an idea of the magnitude and scope of the review backlog (if any). An indicator that's effectively always on is worse than a number in the scenario where a large backlog exists. – Anna Lear♦ ↵ May 7 at 15:55
However rather than just pointing out whether it is or is not good UX it's probably useful to suggest improvements. So this is what I think:
- The purpose of the indicator is to show that there are reviews in the queue
- It has a number in there is to indicate the overall severity of the review queue
- The numbers are cached and not personalised so are never accurate.
With these things in mind I suggest an improvement could be:
Have the indicator take the form of a coloured shape, a different shape for each colour status. Kind of like a traffic light, but with different shapes to help with accessibility.
What the thresholds are would possibly have to be configurable, as different sites (StackOverflow in particular) will have constant RED otherwise, but the general idea being - red status means there are lots in here, amber means things are OK but need looking at and green means the review queue is pretty well under control.
The aim would obviously be to keep the queue in the green.
Alternatively, if this is a bit too 'in your face' we could just highlight the 'Review' item differently depending on how severe the queue is. Something like this perhaps:
This is a bit more subtle true, and a bit trickier to implement, but might be a nicer implementation.
/EDIT - I've now posted this as a suggestion on the main Stack Exchange meta to see what the general community think: Can / Should the review count indicator be changed to be colour-coded status instead of numerical?