Instructions For Do-It-Yourself Ordinal Optimization (OO) Demonstration
by
Yu-Chi Ho
Harvard University
I assume you are using Microsoft Excel v.4 on a MaCintosh. Otherwise make
appropriate changes for PC and Lotus 1-2-3.
-
Open a new work sheet.
-
Enter the value of "1" in cell A1.
-
Enter the formula "=A1+1" in cell A2.
-
Copy cell A2.
-
Paste into cells A3:A200.
This will yield for column A 1 through 200 the values
1, 2, . . . , 200 which represents an Ordered Performance Curve (OPC) of
a complex system. Note the OPC must be monotonic and one dimensional regardless
of the complexity of the system. The best performance is "1", the second
best "2", . . . , and so on. You can change the values in the cell later
on if you want.
-
Enter the formula =rand()*100" into cell B1.
-
Copy B1 and paste into cells B2 through B200.
This enter a column of random numbers uniformly distributed
between 0 and 100. in column B Note the range of the noise is half as large
as the range of the system performance. You can of course change the value
defining the range which is 100" currently.
-
Enter the formula "=A1+B1" in cell C1.
-
Copy cell C1 and paste into C2 through C200.
This operations enters the estimated (or noise corrupted)
system performance of the 200 design in column C
-
Now copy the three-column region from A1 to C200.
-
Define another three column region from D1 to F200.
-
Use "paste special" command to "value-paste" the contents (not the formula)
of A1 - C200 to D1 - F200.
Because of the design of Excel we cannot just paste
A1 - C200 to D1 - F200. You will note that at the end of step 12, the content
of D1-F200 is what was in A1-C200 before the execution of the paste special
command. The content of A1-C200 is a NEW set of noises and estimated system
performance.
-
Define the region F1-D200, and choose the sort command from the "data"
menu. Now sort the F column in ascending order by rows.
-
Now look at column D, and see how many numbers from 1 through n appear
in the top n rows. This correspond to one sample realization of the alignment
between the actual top-n design with the estimated top-n design.
If you did the above steps correctly, you should see
on the average 4 of the true top-12 in the first twelve rows of column
D for the given value of the parameters, i.e.,
N = total # of designs considered and estimated =
200
W = range of estimation noise = 100
n = range of good enough subset, the top-n designs
= 12
-
Now you are ready to repeat steps 12-14 again for another sample realization.
-
You are also free to experiment with different noise ranges, W (other than
"100" try W=10,000 and see what happens), the # of designs, N (other than
"200"), and the values in the column A (other than linearly increasing).
-
It can not be emphasize too strongly that these results are generally applicable
to any systems, particularly in the case when W is large relative to the
range of the performance value. As W->_, we have the equivalent situation
of infinite noise or blind choice, the alignment probability can be explicitly
calculated and serves as a lower bound to alignment.
-
For demonstration purposes, you can make this spread sheet more elegant
and impressive using Excel macros. What I have stated here is the easiest
but not the most presentable demo.
You can also download the demo by clicking here.
Here are some instructions for downloading:
-
Save the file as "oodemo.exe" to your hard disk.
-
Change the extention from ".exe" to ".xls".
-
Open "oodem.xls" with Microsoft Excel 5.0 or up.
-
Run macro "Alignment" and take note of the number of
true top-12 design in column C.
-
Run macro "Blindpick" to see what if we have infinite
noise.