For sigma2 being finite, P will be problem
dependent.
To Next Slide
/
To Previous Slide
/
Table of Content
Let us give a simple example. Suppose we use the crudest selection method,
namely we blindly pick |S| members of the N samples and ask the
probability that among these |S| selected designs there are at least
k designs that will rank in the top-g of the performance order
(recall the engineering design example in the earlier transparency). Blind
Pick (BP) is equivalent to a selection method where the error variance of
selection is infinity. It turns out this probability can be calculated in
closed form. There are gCi ways of picking i
elements out of the top-g designs and N-gCg-i ways
picking the remaining (g-i) designs out the N-g
samples; this is divided by the total number of ways of selecting g
designs out of N samples NCg to arrive at the
alignment probability. The curves shown are for N = 1000. As we
can see for k=1, P ~ 1 for size of |G|=g=50, i.e.,
without any knowledge we can effect a 20:1 reduction and still be sure that
at least one of the top-50 design is in the blindly picked subset.
For sigma2, we no longer have a closed form answer. And we need a
model to calculate the alignment probability. For example, let us assume that
which is known as the Thurstone Model. We shall take the observed
top-|S| performances as the selected subset S and compute
experimentally via simple simulation using Eq.(*) the alignment probability.
However, it is also clear that this probability will be problem-dependent. We
need to introduce some other concepts before we can characterizing this
probability.