Cogprints

Combining independent modules to solve multiple-choice synonym and analogy problems

Turney, Peter and Littman, Michael and Bigham, Jeffrey and Shnayder, Victor (2003) Combining independent modules to solve multiple-choice synonym and analogy problems. [Conference Paper]

Full text available as:

[img]
Preview
 PDF
218Kb

Abstract

Existing statistical approaches to natural language problems are very coarse approximations to the true complexity of language processing. As such, no single technique will be best for all problem instances. Many researchers are examining ensemble methods that combine the output of successful, separately developed modules to create more accurate solutions. This paper examines three merging rules for combining probability distributions: the well known mixture rule, the logarithmic rule, and a novel product rule. These rules were applied with state-of-the-art results to two problems commonly used to assess human mastery of lexical semantics -- synonym questions and analogy questions. All three merging rules result in ensembles that are more accurate than any of their component modules. The differences among the three rules are not statistically significant, but it is suggestive that the popular mixture rule is not the best rule for either of the two problems.

Item Type:Conference Paper
Subjects:Computer Science > Statistical Models
Computer Science > Language
Linguistics > Computational Linguistics
Linguistics > Semantics
Computer Science > Machine Learning
ID Code:3163
Deposited By: Turney, Peter
Deposited On:19 Sep 2003
Last Modified:11 Mar 2011 08:55

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

Eric Brill and Jun Wu. Classifier combination for improved

lexical disambiguation. In Proceedings of the Annual

Meeting of the Association for Computational Linguistics,

volume 1, pages 191-195, 1998.

David J. Chalmers, Robert M. French, and Douglas R.

Hofstadter. High-level perception, representation, and

analogy: a critique of artificial intelligence methodology.

Journal of Experimental and Theoretical Artificial

Intelligence, 4:185-211, 1992.

Cathy Claman. 10 Real SATs. College Entrance Examination

Board, 2000.

Christiane Fellbaum. WordNet: An Electronic Lexical

Database. The MIT Press, 1998.

Radu Florian and David Yarowsky. Modeling consensus:

Classifier combination for word sense disambiguation. In

Proceedings of the 2002 Conference on Empirical Methods in

Natural Language Processing (EMNLP 2002),

pages 25-32, 2002.

Robert M. French. The computational modeling of

analogy-making. Trends in Cognitive Sciences, 6(5):

200-205, 2002.

Tom Heskes. Selecting weighting factors in logarithmic

opinion pools. In Advances in Neural Information Processing

Systems, 10, pages 266-272, 1998.

Geoffrey E. Hinton. Products of experts. In Proceedings of

the Ninth International Conference on Artificial Neural

Networks (ICANN 99), volume 1, pages 1-6, 1999.

Robert A. Jacobs. Methods for combining experts' probability

assessments. Neural Computation, 7(5):867-888,

1995.

Robert A. Jacobs, Michael I. Jordan, Steve J. Nowlan, and

Geoffrey E. Hinton. Adaptive mixtures of experts. Neural

Computation, 3:79-87, 1991.

Mario Jarmasz and Stan Szpakowicz. Roget's thesaurus

and semantic similarity. In Proceedings of the International

Conference on Recent Advances in Natural Language

Processing (RANLP-03), in press, 2003.

George Lakoff and Mark Johnson. Metaphors We Live By.

University of Chicago Press, 1980.

Thomas K. Landauer and Susan T. Dumais. A solution to

Plato's problem: The latent semantic analysis theory of

acquisition, induction and representation of knowledge.

Psychological Review, 104(2):211-240, 1997.

Michael L. Littman, Greg A. Keim, and Noam Shazeer.

A probabilistic approach to solving crossword puzzles.

Artificial Intelligence, 134(1-2):23-55, 2002.

Robert E. Schapire. A brief introduction to boosting. In

Proceedings of the Sixteenth International Joint Conference

on Artificial Intelligence, pages 1401{1406, 1999.

Egidio Terra and C. L. A. Clarke. Frequency estimates

for statistical word similarity measures. In Proceedings

of the Human Language Technology and North American

Chapter of Association of Computational Linguistics

Conference 2003 (HLT/NAACL 2003), pages 244-251, 2003.

Peter D. Turney. Mining the web for synonyms: PMI-IR

versus LSA on TOEFL. In Proceedings of the Twelfth

European Conference on Machine Learning (ECML-2001),

pages 491-502, 2001.

Peter D. Turney and Michael L. Littman. Learning analogies

and semantic relations. Technical Report ERB-1103,

National Research Council, Institute for Information

Technology, 2003.

Peter D. Turney and Michael L. Littman. Measuring praise

and criticism: Inference of semantic orientation from association.

ACM Transactions on Information Systems,

in press, 2003.

Lei Xu, Adam Krzyzak, and Ching Y. Suen. Methods of

combining multiple classifiers and their applications to

handwriting recognition. IEEE Transactions on Systems,

Man and Cybernetics, 22(3):418-435, 1992.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.