Model Selection Under Sampling Uncertainty Using Fuzzy Numbers

Model selection is one of the fundamental tasks of scientific inquiry. The most widely used methods such as ROC analysis do not take sampling uncertainty into account. To improve the robustness of model selection, the author developed a model selection method capable to incorporate sampling uncertainty. She captured the sampling uncertainty by using the bootstrap technique, and quantified the sampling uncertainty by introducing fuzzy numbers. In the book, the author applied the model selection system to a variety of real-world databases with respect to binary classifications. Among the tested datasets, the method performs in line with the traditional ROC analysis, whereas it provides the fuzzy presentation of ROC curves based on which not only the predictive accuracy but also the degree of sampling uncertainty can be addressed. In addition, the author developed a computer tool implementing the system, which eases the tedious procedures in model selection.