## III. HOME WORK PROBLEMS

### 3. AN WRONG CLASSIFICATION PROBLEM

Intuitively, we all know that the longer we run the simulation, the smaller the uncertainty associated with an estimate of, say, the performance J of the system. Suppose you were told that the uncertainty associated with J is uniformly distributed about the estimate with half width w(t) = c/t where c is a constant associated with the simulation. Now consider two parametrically different but structurally similar simulation running side-by-side. Their uncertainty constants are c1 and c2 respectively. Let the estimates for J be respectively and at time t. I would like to stop the simulation at a time when I observe, say, > , I can be 90% sure that the actual order is also J1 > J2 and not J2 > J1. Similarly if it is the other way around.
1. To simplify matters, let us pose an easier problem.
Suppose at some time we observe = 1 and = 2 and from statistical analysis we also know that w1 = w2 = c1/t = c2/t = 1.0. What is the probability that the actual order is J1 > J2, i.e., the probability of a wrong classification?

2. Now suppose that you know c1 and c2 you observe and , describe a procedure (as the function of time) by which you can be 90% sure that the observe order is in fact the correct order? (describe only the main idea, you are not required to work out the details).
[Hint: consider the probability of wrong classification and the below diagram.]

Fig. 1 Possibilities of wrong classification

3. Is the above figure the only possibility you should consider for part (ii)? If so or if not, explain your answer.
4. Suppose at some time we observe = 1 and = 2.5 and from statistical analysis we also know that w1 = c1/t = 5.0 and w2 = c2/t = 1.0. What is the probability that the actual order is J1 > J2, i.e., the probability of a wrong classification?
[Hint: consider the answer to part (iii)]