http://www2009.eprints.org/10/
Latent Space Domain Transfer between High Dimensional Overlapping Distributions
Transferring knowledge from one domain to another is challenging due to a number of reasons. Since both conditional and marginal distribution of the training data and test data are non-identical, model trained in one domain, when directly applied to a different domain, is usually low in accuracy. For many applications with large feature sets, such as text document, sequence data, medical data, image data of different resolutions, etc. two domains usually do not contain exactly the same features, thus introducing large numbers of “missing values” when considered over the union of features from both domains. In other words, its marginal distributions are at most overlapping. In the same time, these problems are usually high dimensional, such as, several thousands of features. Thus, the combination of high dimensionality and missing values make the relationship in conditional probabilities between two domains hard to measure and model. To address these challenges, we propose a framework that ﬁrst brings the marginal distributions of two domains closer by “ﬁlling up” those missing values of disjoint features. Afterwards, it looks for those comparable sub-structures in the “latent-space” as mapped from the expanded feature vector, where both marginal and conditional distribution are similar. With these sub-structures in latent space, the proposed approach then ﬁnd common concepts that are transferable across domains with high probability. During prediction, unlabeled instances are treated as “queries”, the mostly related labeled instances from outdomain are retrieved, and the classiﬁcation is made by weighted voting using retrieved out-domain examples. We formally show that importing feature values across domains and latentsemantic index can jointly make the distributions of two related domains easier to measure than in original feature space, the nearest neighbor method employed to retrieve related out domain examples is bounded in error when predicting in-domain examples. Software and datasets are available for download.
Xie, Sihong
Fan, Wei
Peng, Jing
Verscheure, Olivier
Ren, Jiangtao
Latent Space Domain Transfer between High Dimensional Overlapping Distributions
2009-04-06T19:08:44Z
Xie, Sihong <http://www2009.eprints.org/view/author/Xie=3ASihong=3A=3A.html> and Fan, Wei <http://www2009.eprints.org/view/author/Fan=3AWei=3A=3A.html> and Peng, Jing <http://www2009.eprints.org/view/author/Peng=3AJing=3A=3A.html> and Verscheure, Olivier <http://www2009.eprints.org/view/author/Verscheure=3AOlivier=3A=3A.html> and Ren, Jiangtao <http://www2009.eprints.org/view/author/Ren=3AJiangtao=3A=3A.html> (2009) Latent Space Domain Transfer between High Dimensional Overlapping Distributions. In: 18th International World Wide Web Conference, April 20th-24th, 2009, Madrid, Spain.
application/vnd.ms-powerpoint
application/pdf