11
\$\begingroup\$

Folks

I’d like to bring to your attention the growing size of our close review queue.

enter image description here

This is a large number, considering that the size of the review queue was 10-20 questions as recently as last September. For the last 6 moths or so, the close-review queue has been flaring up and down. Let’s look, however, at the last 2 years. In this time, the number of new questions per week went up by 79% (and so did the number of answers). The number of questions that get closed per week went up by 58%. The overall number of 3k+ members have increased. The number of people who participate in reviewing on any given week, however, remained roughly the same.

This week, 8 of our members have done close vote reviews, and some of us have put quite an effort into reviewing. Thank you!

We are doing a pretty good job at identifying poor questions and flagging them. But we currently don't have enough participation in the downstream review process.

There are about 20 of the 3k+ members that post questions and answers each week.
If each one of us does 3-4 close vote review tasks and checks the review queue each week, this issue will be gone, and remain gone.

\$\endgroup\$
  • \$\begingroup\$ I've just hit the Review button, and is says 0 questions VTC. Does that mean I'm looking in the wrong place, the queue happens to be 0 at the moment, or don't have the rep to see that queue? \$\endgroup\$ – Neil_UK Feb 15 '16 at 12:00
  • \$\begingroup\$ @Neil_UK I suspect that you were looking at the review queue for the meta.EE.SE, rather than the main site. \$\endgroup\$ – Nick Alexeev Feb 17 '16 at 18:13
  • 1
    \$\begingroup\$ I like to thank you for taking a screenshot showing I'm at least putting some effort in :-D \$\endgroup\$ – Asmyldof Feb 18 '16 at 13:53
7
\$\begingroup\$

You are only looking at part of the picture. One failing of the review queue stats is that they only record actions take as a result of going thru the review queue. I just checked, and it looks like I voted to close about 22 questions over the last week, but none of that shows up in the review queue. And of course it doesn't show the majority of questions that I looked at and didn't vote to close, but still therefore "reviewed".

Those of us that are fairly active here end up looking at many of the questions, and thereby "review" them when encountered. Personally, I feel like I already spend too much time on crappy questions by writing comments, downvoting, and voting to close them.

The review queue can also be awkward at times. I don't remember the details since I'm not in there often, but if I remember right things are grayed out making them difficult to read, you don't get to see the whole page, have to click on things to see what others have said, etc. As a result, reviewing questions by going thru the review queue not only feels like busy work, but its also more tedious than it needs to be.

\$\endgroup\$
  • 1
    \$\begingroup\$ Completely agreed, although I don't have enough points to review on this site yet. However, the problem that a long review queue reflects is that close votes age and are eventually cancelled if not seconded by enough people. Reviewing outside of the queue is absolutely fine and does reduce the queue length as well (even if you will not be credited in this case), but if the queue is getting long, it suggests that reviews aren't being done in a timely enough manner for the votes to consistently result in actual closure. \$\endgroup\$ – Oleksandr R. Feb 14 '16 at 17:57
  • 1
    \$\begingroup\$ Your VTC put the questions in the review queue... \$\endgroup\$ – Passerby Feb 14 '16 at 22:16
  • \$\begingroup\$ ... for all the right reasons, @Passerby . \$\endgroup\$ – Nick Alexeev Feb 14 '16 at 22:58
  • 1
    \$\begingroup\$ I have a few rep still to go before I'll ahve access to the VTC queue, but I most definitely agree on the review queues being awkward when there is information that you have no access to in the review queues. I hate working through the 'new posts'/'late answers' queues & having to go over and open the question/answer I'm supposed to be reviewing in another tab, so I can see other answers & comments, to get a "bigger picture view" on what I'm supposed to be reviewing in order to "process the queue." \$\endgroup\$ – Robherc KV5ROB Feb 14 '16 at 23:52
6
\$\begingroup\$

Voted to close my 20. My biggest close issue is that there are tons of duplicates that I don't want to chase down.

\$\endgroup\$
  • 4
    \$\begingroup\$ Unless you know the exact title, duplicates are almost impossible to find. I've got to the point of favoriting some of the most common for easy access. \$\endgroup\$ – Matt Young Feb 14 '16 at 14:51
  • \$\begingroup\$ I think I brought up the idea of better community wikis to cover some of them, and basically got "go ahead" as an answer. \$\endgroup\$ – Scott Seidman Feb 14 '16 at 14:56
  • 2
    \$\begingroup\$ Scott, I agree with you that finding duplicates has become difficult because of the increasing number of questions on EE.SE . Here are some of the tricks that sometimes help find the duplicates. (1) Check tag wiki. Canonical questions may be listed there. See if the sought dupe is among the canonical questions. (1.1) If you think that you have found a question that should be canonical, you could add it to the list in the tag wiki. \$\endgroup\$ – Nick Alexeev Feb 14 '16 at 21:10
  • 2
    \$\begingroup\$ (2) Throw the question into our EE.SE chat with a comment that there should be a dupe, and you are having a difficult time finding it. Somebody might remember it. \$\endgroup\$ – Nick Alexeev Feb 14 '16 at 21:10
  • 7
    \$\begingroup\$ @NickAlexeev: Combined with the usual attitude towards people writing duplicates (although maybe it's just my imagination), these comments sound quite comical, so I'll have to point it out: I hope you're not giving too hard of a time to the people writing those duplicates, since if the old question is so oddly worded that even EE.SE regulars are having a hard time finding it with their experience and when presented with the new question and having presumably seen the old question, can you really expect the EE.SE novice writing the duplicate to find it? :-) \$\endgroup\$ – Aleksi Torhamo Feb 15 '16 at 14:14
  • 2
    \$\begingroup\$ @AleksiTorhamo Closing because of a duplicate is nothing personal. It just says that the archive does not require a new question to be complete. The poster can find the original post, because the close message points them right to it. That says, the new poster can and should spend a whole lot more time looking for the answer to their question before they post it than a reviewer can be expected to take up looking for a dup. \$\endgroup\$ – Scott Seidman Feb 15 '16 at 15:37
  • 2
    \$\begingroup\$ @ScottSeidman: To clarify, I know that marking duplicates is a win for everybody and nothing personal; I was referring to those cases where somebody leaves colourful comments [ridiculing the poster] instead of/in addition to flagging as duplicate. (Even though sometimes, as illustrated here, finding the duplicates can be damn hard even for an experienced person. Search engines can't read minds and don't (often) recognize synonyms) \$\endgroup\$ – Aleksi Torhamo Feb 15 '16 at 15:48
  • 2
    \$\begingroup\$ @Aleksi Posting a duplicate question is an honest mistake, if duplicity is the only problem. (That's how I see it.) \$\endgroup\$ – Nick Alexeev Feb 19 '16 at 22:46
3
\$\begingroup\$

I stopped going through the close vote review queue because I got tired of reviewing questions twice. I already voted to leave it open when I didn't vote to close it, why do I need to vote to leave it open again? The remainder, I skipped because the question was out of my core competence, and didn't feel comfortable deciding whether it should stay open or be closed. This is a broken system, but I don't have any good suggestions on how to fix it.

\$\endgroup\$
  • \$\begingroup\$ In what way is it broken? Are you saying that closing questions is bad? \$\endgroup\$ – PlasmaHH Feb 14 '16 at 8:11
  • 3
    \$\begingroup\$ @PlasmaHH Not sure how you got closing questions is bad from that... I'm probably one of the most heavy handed closers on this stack. What's bad is forcing people to review the same question multiple times. It's pointless to go through a queue just to hit leave open or skip on almost everything in the queue. \$\endgroup\$ – Matt Young Feb 14 '16 at 14:53
  • 2
    \$\begingroup\$ on SO i never had a question in the queue that i already voted for. I always assumed they filter it. If not, that is a bug and should be properly reported imho. Also noone forces anyone to do the queues. I stopped doing them on SO \$\endgroup\$ – PlasmaHH Feb 14 '16 at 17:54
  • \$\begingroup\$ Voting to leave open ages close votes more quickly. In the review queue it is a positive action rather than a passive decision. Skipping questions is not a problem if you don't know or can't decide, but the queue helps to solicit second opinions on questions you otherwise might not see. I don't like the UI of the reviewing system, but reviewing is a useful action. \$\endgroup\$ – Oleksandr R. Feb 14 '16 at 18:01
  • \$\begingroup\$ @OleksandrR. That's my point. With our question volume, I see pretty much everything I would vote on. I don't need to go into the queue and see it again. \$\endgroup\$ – Matt Young Feb 14 '16 at 18:03
  • \$\begingroup\$ Well, you don't have to go through the queue if you feel you've done enough already, of course. But as I said under Olin's post, the long queue suggests that, at the moment, anyone who does vote to close or actively chooses not to is to some extent wasting their time, because on the average not enough people are seconding their votes before they time out. Keeping the queue small is important to the proper functioning of the site, whether you do that through the queue or otherwise. If you visit a site where people habitually don't review, you'll see the corrosive effect it has. \$\endgroup\$ – Oleksandr R. Feb 14 '16 at 18:16
  • \$\begingroup\$ @Matt In other words, the review queue has sequential access, but doesn't have random access. \$\endgroup\$ – Nick Alexeev Feb 14 '16 at 23:01
  • \$\begingroup\$ @NickAlexeev Could we add a link in each question's view screen for questions pending any form of review (from someone at the rep level of whomever is viewing), so that we can review a question/answer as we're viewing it? Rather than having to leave the question, go into the review queue, wait 'til we sequentially get to it, then open it again in another tab to make sure we see everything that has (maybe) changed since we navigated into the review queue before we can properly handle the review? \$\endgroup\$ – Robherc KV5ROB Feb 14 '16 at 23:59
  • \$\begingroup\$ @Robherc If a question is in the process of close or reopen review, there is an indicator for that (you'll see it when you have 3k reputation). The first post, late posts, low quality don't have an indicator. Perhaps that's because they don't reside in the review queues for long. \$\endgroup\$ – Nick Alexeev Feb 15 '16 at 0:11
0
\$\begingroup\$

For reference, Stack Overflow has a close-vote review queue of 9600 questions. SO gave up on the close-vote review queue years ago.

This is because the close vote system in itself is poorly designed. The equation is fundamentally wrong: there is far too many crap questions in relation to the amount of active user moderators.

There is not 5 active user moderators per crap question posted.

Rather, there are probably 100 crap questions per active user moderator, or something like that. I doubt it even adds up if every active user moderator exhaust their number of close votes every day.

There is simply too much reliance on manual work and user moderation. Particularly, it is far too much work to find canonical duplicates. Which in turn is caused by another design flaw, which is the lack of a properly working FAQ system.

Doing more manual work is no long-term solution; this is all caused by fundamental design problems in the Stack Exchange site concept. Instead, the problem should be brought to the main http://meta.stackexchange.com site.

\$\endgroup\$
  • 3
    \$\begingroup\$ Basic embedded systems design: if your system is not fast enough to process all incoming data, then no amount of queues and buffers will save it. Queues are used to handle peaks of incoming traffic, that your system cannot process straight away. But if a queue is never exhausted, the system design is fundamentally broken. Same applies here. \$\endgroup\$ – Lundin Feb 18 '16 at 9:33
  • 1
    \$\begingroup\$ By now we have beaten back the backlog of the close-votes. This was done by member mods (not diamond mods). It took less than a week. We don't have the close vote problems on the scale of StackOverflow. EE.SE is a fairly small stack. It will take us years until we get to the size comparable to StackOverflow. I'll also add that our audience is different. \$\endgroup\$ – Nick Alexeev Feb 20 '16 at 3:17
  • \$\begingroup\$ @NickAlexeev I think you fail to see my point. As the number of users grow, the number of crap posts, as well as the number of user moderators grow too. There's a "crap per mod" constant for every site. So the size of the site doesn't matter, more so its age. SO is the oldest site so it should have most user moderators. Yet it has the largest problems, which in turn is likely caused by user moderators getting depressed by the amount of crap and going inactive. It is a broken system design which will ultimately fail in the long term. \$\endgroup\$ – Lundin Feb 22 '16 at 7:17
  • \$\begingroup\$ My previous comments deals with empirical present of EE.SE . Your point deals with a hypothetical future. I do get your point, though. Have you got ideas about a long term overhaul of the SE framework? \$\endgroup\$ – Nick Alexeev Feb 22 '16 at 7:49
  • \$\begingroup\$ @NickAlexeev I think the only way to save SE in the long term is to continue to focus on quality. Which means focusing on stopping the crap before it hits the site. This could be done in many ways: smarter scripts, mandatory user review before 1st time posts hit the site, or splitting sites into different tiers of experience (have a beginner questions' site which you have to gain rep on before you are allowed on the main site) etc etc. Suggestions like these are never well-received though: there's always elitist concerns and there's the site owners who think crap -> traffic -> cash. \$\endgroup\$ – Lundin Feb 22 '16 at 7:57
-1
\$\begingroup\$

Why don't we drop the close criteria to 3 votes? Is there something magical about 5? If it isn't working for us, let's fix it

\$\endgroup\$
  • 2
    \$\begingroup\$ If we can't get five people to agree that it's bad, then its not bad. Lowering the threshold just means a less people get a bigger say. And we already have one person abusing their close and migrate powers as is. \$\endgroup\$ – Passerby Feb 18 '16 at 23:00
  • \$\begingroup\$ Five votes may be alright (at lest for now). We've beaten the review queue back in less than a week. The number of members who have participated in the review queue have tripled, compared to previous week. \$\endgroup\$ – Nick Alexeev Feb 19 '16 at 1:50
  • 3
    \$\begingroup\$ Recently, Programmers.SE ran an experiment. For a period of time, the close threshold was lowered to 3 votes. Results and discussion here. \$\endgroup\$ – Nick Alexeev Feb 19 '16 at 1:55
  • \$\begingroup\$ @NickAlexeev -- interesting story. \$\endgroup\$ – Scott Seidman Feb 19 '16 at 13:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .