I think closed questions are important. We should strive to have as few closed questions as possible, but realize that we will never reach 0%. But we don't want a site with 5 closed questions for every good question, so closed rate is important.
However, different close reasons point to different problems.
Different close reasons
(If you're familiar with the closing system you can skip to to sum up.)
- Duplicates are just mistakes by people who didn't search carefully enough, but we shouldn't bother too much about them and there's not much we can do about them.
- Off-topic votes come in different flavours:
- Bluntly off-topic: question on recommendations for products or places to purchase them as well as questions on the use of electronic devices are mostly cases where people didn't bother to read the help center. It's annoying and if we get a lot of this we could look into giving the help center a more prominent place in the UI for new users.
- Repair questions could possibly be reworded to become on topic but normally also this is end of story.
- Wrong network site doesn't happen a lot as far as I know, but again we should look into emphasizing the help center if this becomes a problem.
- Other reasons: lately many questions have been closed with a comment such as 'what part of the datasheet are you having trouble with?'. I hope moderators have a tool to monitor these other reasons to see if reasons should be added to the list of off topic reasons.
- Unclear what you're asking are very important. Often, these questions could be rephrased to make it OK. We should therefore monitor how many closes of this type are followed by a substantial edit (whatever that may be) or hopefully a reopening. If this amount is low we should look into ways to show to users that a closed question doesn't have to be the final station.
- Too broad: similar to the previous one, but less to worry about in my opinion, since less questions that are closed as too broad can easily be rephrased to fit into the SE framework.
- Primarily opinion-based questions would often be welcome in chat if I'm not mistaken. When I'm voting to close a question as such I always consider adding that as a comment. We could monitor how many users with a question that was closed as primarily opinion-based come to the chat afterwards (good luck writing a data query for that). But anyway I don't see many questions closed for this reason at the moment, so it doesn't seem real important.
To sum up, there are some close reasons we don't need to care about: duplicates and most of the off-topic reasons. Mistakes like that happen and questions closed as such normally can't be rephrased to fit into the framework. Questions closed as unclear what you're asking and too broad could often be rephrased. We should therefore be monitoring in how many cases the OP (or someone else) edits the post after it was closed, and if it was reopened in the end. If this amount is low, we have a problem.
By looking how often users attempt to edit their question, we can see how much they're encouraged to improve this EE.SE's quality, which should be one of our cornerstone values.
This raises another point:
Substantial edits and upvoted comments
Part of what SE is good at is answering questions of specific people in specific situations. However, the network is also intended to provide some kind of more universal information source. To this end, questions but more importantly answers should be edited to provide more information, explain better, etc.
We could therefore monitor the amount of substantial edits (again, whatever that may be) related to either the amount of views (how many visitors decide to improve something) or to the amount of posts (how many posts are getting improved).
Often, information is added not by editing the answer, but by leaving a comment. Typically, comments that provide good information are voted up (we can forget about the upvoted jokes for the moment). We could monitor the average amount of comments with some minimum score per post (possibly, comments on answers could be more important than comments on questions).
Other points
I've mentioned two things that weren't mentioned before, but don't want to claim these are the only things we should monitor, or even that we definitely should monitor these. I just wanted to open up the discussion a bit.
I like the idea from Adam Haun's answer of using the statistics used for beta sites, but I'm not sure if they address the kind of problem we have here (which is, mainly, bad questions).
I also like Olin Lathrop's idea of separating statistics on reputation to see how we can help new users get to know the site better.
As a last point, I would say that when many questions of the type 'do we have a problem?' are raised, this may definitely be a pointer to us having a problem.