III. HOME WORK PROBLEMS
8. GOAL SOFTENING (3)
It is well known that the accuracy of a sampling scheme depends only on
the size of the sample and NOT on the size of the underlying population
which we shall assume to be infinite in this problem. Suppose the distribution
of performances J of a system is normal N(0,sJ)
when plotted against a design variable q. We randomly sample N
designs, q1, q2, ..., qN
in an experiment and observe the system performance, J(qi)
in additive noise N(0,snoise), i.e., J(qi)observed
= J(qi) + noise. We now ask the question what
is
Prob{max[J(qi)observed, i
= 1, ..., N] in top-5% of J(q)} = p = ?
assuming we are interested in maximizing performance.
-
Purely as a test of your probabilistic intuition, what do you think is
the likely value of p for the case sJ = snoise
and N = 100?
A. p <= 0.4
B. 0.4 < p < 0.75
C. p > 0.75
Choose one alternative and then calculate the answer to see if you are
correct.
-
Whatever the value of p in the above, now consider doing m
independent experiments of N samples each and ask
Prob{at least 1 of max[J(qi)observed,
i = 1, ..., N] of the m experiments in top-5% of J(q)}
= p(m) = ?
-
Calculate p(m) as a function of m, p. What
did you learn from this calculation?
(Try out a couple of your guesses in the previous question for different
values of m).
-
Suppose instead of getting m we did ONE experiment with mN
samples and asks
Prob{max[J(qi)observed, i
= 1, ..., mN] in top-5% of J(q)} = p*
= ?
-
Is p* > p(m) or p(m) > p*?
Can you relate this to the ideas of Ordinal Optimization?