creators_name: He, Xiaofeng creators_name: Duan, Lei creators_name: Zhou, Yiping creators_name: Dom, Byron type: conference_item datestamp: 2009-04-06 19:12:19 lastmod: 2009-04-07 14:02:50 metadata_visibility: show title: Threshold Selection for Web-Page Classification with Highly Skewed Class Distribution ispublished: pub full_text_status: public pres_type: poster abstract: We propose a novel cost-efficient approach to threshold selection for binary web-page classification problems with imbalanced class distributions. In many binary-classification tasks the distribution of classes is highly skewed. In such problems, using uniform random sampling in constructing sample sets for threshold setting requires large sample sizes in order to include a statistically sufficient number of examples of the minority class. On the other hand, manually labeling examples is expensive and budgetary considerations require that the size of sample sets be limited. These conflicting requirements make threshold selection a challenging problem. Our method of sample-set construction is a novel approach based on stratified sampling, in which manually labeled examples are expanded to reflect the true class distribution of the web-page population. Our experimental results show that using false positive rate as the criterion for threshold setting results in lower-variance threshold estimates than using other widely used accuracy measures such as F1 and precision. date: 2009-04 pagerange: 1081-1081 event_title: 18th International World Wide Web Conference event_location: Madrid, Spain event_dates: April 20th-24th, 2009 event_type: conference refereed: TRUE citation: He, Xiaofeng and Duan, Lei and Zhou, Yiping and Dom, Byron (2009) Threshold Selection for Web-Page Classification with Highly Skewed Class Distribution. In: 18th International World Wide Web Conference, April 20th-24th, 2009, Madrid, Spain. document_url: http://www2009.eprints.org/121/1/p1081.pdf