7
\$\begingroup\$

I want to find out about maximising the rate of entropy collection using an analogue to digital converter fed by a noise source. It involves Shannon entropy calculations, log normal distributions, ADCs and signal amplification /scaling. It's a bit of statistics, electronics and mathematics, perhaps even a derivative.

The problem is the keenness of some to immediately vote the question off topic because it crosses forum boundaries. With the ubiquity of micro controllers and system on a chip chips, this must be a common issue.

Where is the best place to ask this?

(I tried asking my actual question at stats.SE but no one could answer).

\$\endgroup\$
5
  • 3
    \$\begingroup\$ surely you can break down your research into digestable chunks that all are more or less of one topic and then ask these as seperate questions. \$\endgroup\$ – PlasmaHH Feb 28 '17 at 16:30
  • 2
    \$\begingroup\$ It sounds like a DSP.SE question, at least to start with. Come back here if you run into specific EE issues with the implementation. \$\endgroup\$ – Dave Tweed Feb 28 '17 at 17:57
  • 1
    \$\begingroup\$ StackExchange chats - starting with our very own EE.SE chat - are good places where others can help you pre-digest a large question. [By the way, this isn't all that multidisciplinary, as multidisciplinary comes. No disrespect, of course.] \$\endgroup\$ – Nick Alexeev Feb 28 '17 at 20:59
  • \$\begingroup\$ @NickAlexeev I guess its not. It should be just up your street actually. I'll have a go again. \$\endgroup\$ – Paul Uszak Feb 28 '17 at 22:12
  • 2
    \$\begingroup\$ Just want to add that it is a shame that the "electrical engineering" SE is not the right place for an electrical engineering question. :-/ \$\endgroup\$ – Mad Jack Mar 12 '17 at 1:32
7
\$\begingroup\$

I find this interesting because I have had questions that involve more than one discipline and can't be pinned to any one forum. And then I read some of the advice that suggests you stick to one component of the problem and concentrate on that. That made me smile, because the end result of keeping it specific is going to be demands for more details. Ask about the A/D converter, and they demand to know what you're feeding into it. And then they want to know what you're doing with your sampled data. And so on.

Maybe the closest you can get it to ask each piece in each forum, and link the questions so that the curious can still get ahold of the other details. Otherwise, I'd say, just ask the questions that will get you the most information, and take the resulting abuse--it's okay as long as you get answers. You're never going to make everyone happy. Some will complain, and some will give answers. If you don't make a habit of it, it will just be another war story for your blog.

\$\endgroup\$
1
  • 3
    \$\begingroup\$ You're ABSOLUTELY right. I initially mentioned that it was a random number generator, they said use a pseudo one instead. I said it's from an ADC, they said get a better one. I didn't say what the source was and they said they needed to know the source! I put it down to two things. 1 - SE is a bragging site so most gush about what they know. 2 - it's a toyish social media site so most people can't be bothered to read the question, they just latch onto their hot button topic and spew /vote. 2 1/2 - defacto voting rings don't help. I've taken the abuse, but it wears thin... \$\endgroup\$ – Paul Uszak Mar 14 '17 at 13:00
5
\$\begingroup\$

If you are asking about the whole thing, then you haven't done your homework and the question is too broad, probably on any site. Ask about the specific problems you are stuck on. These should be separate questions, each asked where they fit best.

Actually, if you have a lot of questions to ask, then again you haven't done your homework and are essentially asking us to design your project for you. It's your project, and you should be doing most of the design and figuring out the details. We here to help on this site with electrical engineering specifics you might get stuck on.

\$\endgroup\$
3
  • 1
    \$\begingroup\$ Thanks for that encouragement - it's what I've come to expect from SE. However, you've misunderstood my issue. It's one question not a series. It will have a singular numerical answer probably ranging from 1 - 50. This is the reason I asked as everyone has just latched onto a few key words and immediately voted it too broad (as before). Perhaps I can dumb it down and take a punt somewhere... \$\endgroup\$ – Paul Uszak Feb 28 '17 at 22:10
  • 1
    \$\begingroup\$ See? I took a punt and so far it's not going well. I asked electronics.stackexchange.com/questions/289533/… and the expected answer is a singular voltage (one number). And I'm getting fobbed off because either people haven't properly read the question, made up their own question or can't do maths. A voltage is pretty much the electrical engineering specific that I'm stuck on. \$\endgroup\$ – Paul Uszak Mar 1 '17 at 13:50
  • 3
    \$\begingroup\$ @Paul: I looked at that question but skipped it. I think I understand what you're looking for, but the question was too dense for me to be sure after a quick read. I got scared off by talk of "entropy", "as you can clearly see" when I couldn't, and vague mentions of Shannon theory. I'm not going to take a chance answering that question and looking stupid when I can answer many other questions and look smart. I think what you want could be simply explained, but you didn't. The math for the right solution is probably fairly involved though. \$\endgroup\$ – Olin Lathrop Mar 1 '17 at 14:42
4
\$\begingroup\$

I think that was a perfectly legitimate question for EE.SE, since, for what I understand, involves the relationship between the (nominal/average?) voltage across the reference and the mathematical form of the lognormal distribution of samples from that reference (and then the math to obtain an entropy expression and maximizing that).

Disclaimer: although I got a faint glimpse of what the solution involves, this is not among my fields of expertise.

Sadly, I think most people answering or commenting probably didn't fully understand the problem and tried to convince you that you didn't understand your problem.

BTW, you didn't get downvotes, so it appears that most people deemed the question legitimate or even interesting. The lack of answer may be because it appears (and mathematically probably is) a difficult problem.

Maybe try to add some detail about what you already know, e.g.: some pre-digested math expressions adapted to your case; links to (freely available) references to the theory involved, additional info about the final goal of the project (if you can disclose them).

All this could (1) show that you really did all your homework (so people won't think you should know better) and (2) induce people to give a stab at the problem without the need to do all the preliminary research on the web.

Good Luck!

P.S.: if you ever find a solution by yourself, please come back and post it as an answer to your question. I'd really like to know!

\$\endgroup\$
4
  • \$\begingroup\$ I don't think it's quite as simple as all that. There were a few things in the original question that left other things open to question. I skipped it because I didn't feel like going back and forth with a OP that writes like a research paper instead of making accessibility a priority. I also got the impression that asking about some things would only lead to argument instead of clarification. I could be wrong, but I don't have unlimited time here and therefore cherry-pick what to take the time answering. \$\endgroup\$ – Olin Lathrop Mar 12 '17 at 14:49
  • \$\begingroup\$ @OlinLathrop Ok, so you are not convinced the OP has made everything clear and so you don't want to waste your time trying to force him to reveal all the details. Fair enough, really. As I also said above, more "accessible" information (to quote your words) would have been welcome.... \$\endgroup\$ – Lorenzo Donati -- Codidact.com Mar 21 '17 at 20:46
  • \$\begingroup\$ @OlinLathrop ...Anyway, what would be wrong with the OP writing a research paper? I've read your other, longer answer in this thread, and I share some of your concerns, but overall the question is fairly good (IMO), so if you don't feel like answering it or commenting on it (for whatever reason), that's ok. For the rest, it's up to the OP to make his answer the most "palatable" for the knowledgeable users of this site. \$\endgroup\$ – Lorenzo Donati -- Codidact.com Mar 21 '17 at 20:47
  • 3
    \$\begingroup\$ The issue is that the question is written in the style of a research paper. That means it is dense, assumes quite a bit of knowledge you have to follow a chain of references to get. It also often seems to be a game of trying appear superior by deliberately making things difficult to follow as long at it can be shown later that all the information was actually in there after much diligence by the reader. Note that the OP took time to insult the messenger, but not to actually address or clarify any of the things I mentioned. \$\endgroup\$ – Olin Lathrop Mar 21 '17 at 21:52
4
\$\begingroup\$

I would ask on math.se, as it's an optimization problem. You should do more work on your own before asking, though, as it should be pretty vanilla. Estimate an equation for your distribution. Express it in such a way that it reads "entropy = ...", and take the derivative with respect to Vref. That's how optimization is done. If you can't do the "entropy =..." step, then your real question is "how do I do the entropy = step?".

Your limit of 1023 puts an unnecessary constraint on your problem, and makes the math harder. If your issue is your saturation-- roll up your sleeves, do the problem without the saturation, and ask how to extend your solution with the saturation in place.

Despite the fact that you've dismissed Olin's response as "what you expect of EE.SE", he has a valid point. The world doesn't owe you an answer, and a bit of grunt work on your part will make your question much better on whichever stack you choose to ask it on.

\$\endgroup\$
2
  • 1
    \$\begingroup\$ Obviously the world doesn't owe me an answer. But if people like me don't ask questions like this, you'll have nothing to do, SE will not exist and Mr Spolsky won't get his riches. This is symbiosis. And I take (minor) exception to your characterisation of my original graphs, diagrams, formulae and 1000 words on the question across several SE sites as not even "a bit of grunt work". Civilisation evolves through the shared accumulation of knowledge. Even electricians. Do you not believe in that, rather preferring people to repetitively stick their hands into fire to see if it's hot? \$\endgroup\$ – Paul Uszak Mar 13 '17 at 15:08
  • 1
    \$\begingroup\$ I didn't say "not even a bit", as evidenced by the correct placement of your quotes. You made some fine pictures, and communicated the problem nicely, but you didn't make an attempt at solving your own problem. When there's a lot of work being done, but it's not in the right area, you're spinning your wheels. My assumption is that you know what needs to be done, as it's first-semester calc. I'd almost guarantee that if you posed it as an optimization problem with some complications, and posted it to math with your solution effort, you will get answers, if you haven't solved it on your own. \$\endgroup\$ – Scott Seidman Mar 13 '17 at 15:26
4
\$\begingroup\$

Seeing this topic is still getting attention after two weeks, I'll be more specific about what is wrong with the question you posted. Here is my thought process as I read your question:

I have a circuit that generates random noise which I've measured with a 10 bit ADC. The following is the sample distribution:

OK so far. The graph looks plausible for this kind of noise source.

You will notice that it's a log normal distribution

No, I wouldn't. Maybe some time in the dark past a particular probability distribution was given the name "log normal" in some class, but if so, that has been long forgotten. The OP has now alienated all those who haven't just taken the class where this was mentioned, are working in the theoretical parts of EE, or happen to have bumped into this recently enough to remember what the implications of "log normal" are. Surely some people will know, probably the more academic types, but that will be quite a small subset of users here.

Since he didn't define it, link to a reference (probably wouldn't have followed it anyway since it's his job to put the salient points here), or explained whatever part of it being log normal is relevant here, he just lost most of the audience.

The writing style also gives a hint of the "research paper" attitude. That's being technically right and rigorous, but difficult to follow unless one already knows what is being talked about. Mabye it isn't, but it often seems this is done deliberatly: "I'm smarter than you are because my math it more impenetrable than yours."

I'll just let this slide for now because it isn't clear how important this distinction is, and the relevant aspects might become clear from context later.

You can see that approximately 14,000 samples were at a value of 1023 or above

No, I can't. For reference, here is the diagram being referred to:

I can see that the frequency of occurance at 1000 is well below 5000, probably less than 1000 from eyeballing the graph. I do see a box with a top that could be at 14,000, but that box is not mentioned anywhere, and in any case is below 1023, not above it. What the ...?

Clearly I'm not getting something here. Probably best not to touch this question because I'll just look stupid. If this was written more accessibly and less arrogantly, maybe we could work it out, but not with this question.

they are effectively a source of random entropy

I can see the randomness, and the plot showing the relative probabilities of any particular value occurring, but what's the relevance of "entropy"? From context, this seems like some specifically defined way of measuring something about the randomness of the stream of samples, but like "log normal", whatever definition I might have understood from that has been lost in time.

This guy either has no clue about his audience, or is deliberately being academically arrogant to feel superior.

Shannon entropy rate per sample

It's definitely been too long since this was covered in a class (if it ever was) for me to remember what that means. Again, it seems like some kind of randomness measure.

The next paragraph explains that the point of the question is to maximize the randomness of the resulting samples by deciding what section of the overall probability distribution histogram to pick off. Why couldn't he just say that in the first place? He mentions scaling (which is really what changing the reference voltage does), but has ignored shifting the window around (adding a offset). That seems a obvious thing to do, but why isn't it considered? It is easy enough to do electrically. I would engage and ask about this if I didn't think I'd just get more Shannon entropy log blah blah babble in return.

Also, it's not really clear why not scaling and offsetting to pick off a narrow area around the hump of the curve isn't good enough. That will cause the A/D to clip to 0 and 1023 more often. These are really "no data" and therefore don't count as useful readings? What about the usual approach of using only the low bit of the A/D? If the window was chosen carefully so that the probability of clipping at 0 and at 1023 were equal, then the low bit should still be random and usable from the clipped samples too.

However, engaging with this guy feels like it's going to be painful, so I'll go find a bunch of simpler questions to answer in the time it would take going back and forth with this ivory tower weenie.

Also, if this guy has a equation for the probability distribution (I'm assuming that's what "log normal" refers to), and knows how to evaluate this Shannon entropy stuff he's on about, why doesn't he just write the equation of Shannon entropy as a function of the scaling factor and maximize it? How is that not obvious, especially considering the high-falutin theoretical stance he's taken? Something doesn't add up. It feels like penny-wise and pound-foolish. But, I don't feel like spending a lot of time and then just looking foolish myself as a result, so screw this.

\$\endgroup\$
4
  • 1
    \$\begingroup\$ @Paul: Or, you could actually fix the problems pointed out instead of insulting the messenger. \$\endgroup\$ – Olin Lathrop Mar 15 '17 at 12:15
  • 3
    \$\begingroup\$ This is a rant. \$\endgroup\$ – Passerby Mar 15 '17 at 19:10
  • \$\begingroup\$ Of course it is a rant but the question is valid. Where is the best place to ask this? and some lame Moderator deleted my answer with an invalid comment This area is reserved for answers to the question at the top of the page. My point is that EE's have to be multi-discipline to succeed at the hardest problems so this site ought to be the place to ask this question. yet with such incompetence, it is impossible to moderate such questions here. He ought to stick with Ohm's Law , the path of least conductance \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 Jun 7 '17 at 5:17
  • \$\begingroup\$ But then if someone is skilled enough to ask a multi-disciplinary question, the assumptions better be clear and terms of the question well defined otherwise it may end up in endless discussions. My answer was a meta type answer with an example to support my rationale why EE.SE should but never could support this. Only the Physics forum has the bandwidth. \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 Jun 7 '17 at 5:28
2
\$\begingroup\$

You should pick a community which could potentially generate the most knowledge relevant to your question that you will be able to absorb and be willing to apply. For example, if you're an EE expert, chances are that you got the EE part of your problem just right, and people on EE.SE won't tell you much of that you don't know already. Conversely, if you're a complete EE dummy, it will be hard for you even to formulate the question correctly, and you may get answers which are either irrelevant or too generic. Or people will just tell you to go read a book. Or people tell you to use a particular schematic which you're unable to understand, so the answer is of no use.

In your case, if question is purely about a particular property (sample entropy) of a given distribution, you should really consider asking it on stats.SE. You don't even have to talk about the voltage there: just say that your distribution has a parameter you can control, and show how it factors in the distribution.

However, if you want to know what could be done electrically to improve your generator, you should ask here. I wrote an answer to your question and I hope it will be helpful to you and improves the generator beyond what would be possible with only optimizing Vref. But if you tell me that you're not going to change your circuit no matter what, I will be left wondering why did you pick this site to ask.

\$\endgroup\$
2
  • 2
    \$\begingroup\$ That would seem like a good idea apart from stats.stackexchange.com/questions/240470/…. No answers. No comments. Question too hard. Do you see what I mean? \$\endgroup\$ – Paul Uszak Mar 13 '17 at 12:42
  • 1
    \$\begingroup\$ @PaulUszak Well, I have little experience with stats.SE, maybe they mostly do homework and this question is too hard for them. Or maybe they didn't like your question in particular. There are limits in what you can expect from a Q&A site, and many good questions go unanswered. I wonder if offering a bounty would help (in my experience, it often doesn't but I'd still try). \$\endgroup\$ – Dmitry Grigoryev Mar 14 '17 at 10:04
0
\$\begingroup\$

Stack Exchange operates on a simple rule. A question is welcomed at any page that the question is on-topic for, even if it may be more on-topic somewhere else. Migrations should only happen if the question is off-topic at the initial stack and on-topic at the other (and quality). Close votes should not be abused just because someone thinks its better off at another stack when its perfectly valid on the original one. IT'S OP'S CHOICE. Overlapping sites does not force migrations.

\$\endgroup\$
2
  • 2
    \$\begingroup\$ This is a rant, not a answer. \$\endgroup\$ – Olin Lathrop Mar 14 '17 at 16:41
  • 4
    \$\begingroup\$ No downvote, but the question was "where should stuff like this go", and the implicit question continues "...to get the best answer?" Of course it's on topic here, and I don't see suggestions otherwise, but that doesn't make this the best place to post it. \$\endgroup\$ – Scott Seidman Mar 14 '17 at 21:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .