--- abstract: |- Existing statistical approaches to natural language problems are very coarse approximations to the true complexity of language processing. As such, no single technique will be best for all problem instances. Many researchers are examining ensemble methods that combine the output of multiple modules to create more accurate solutions. This paper examines three merging rules for combining probability distributions: the familiar mixture rule, the logarithmic rule, and a novel product rule. These rules were applied with state-of-the-art results to two problems used to assess human mastery of lexical semantics -- synonym questions and analogy questions. All three merging rules result in ensembles that are more accurate than any of their component modules. The differences among the three rules are not statistically significant, but it is suggestive that the popular mixture rule is not the best rule for either of the two problems. altloc: [] chapter: ~ commentary: ~ commref: ~ confdates: ~ conference: ~ confloc: ~ contact_email: ~ creators_id: - 2175 - '' - '' - '' creators_name: - family: Turney given: Peter D. honourific: '' lineage: '' - family: Littman given: Michael L. honourific: '' lineage: '' - family: Bigham given: Jeffrey honourific: '' lineage: '' - family: Shnayder given: Victor honourific: '' lineage: '' date: 2004 date_type: published datestamp: 2005-01-10 department: ~ dir: disk0/00/00/40/27 edit_lock_since: ~ edit_lock_until: ~ edit_lock_user: ~ editors_id: [] editors_name: - family: Nicolov given: Nicolas honourific: '' lineage: '' - family: Botcheva given: Kalina honourific: '' lineage: '' - family: Angelova given: Galia honourific: '' lineage: '' - family: Mitkov given: Ruslan honourific: '' lineage: '' eprint_status: archive eprintid: 4027 fileinfo: /style/images/fileicons/application_pdf.png;/4027/1/turney.pdf full_text_status: public importid: ~ institution: ~ isbn: ~ ispublished: pub issn: ~ item_issues_comment: [] item_issues_count: 0 item_issues_description: [] item_issues_id: [] item_issues_reported_by: [] item_issues_resolved_by: [] item_issues_status: [] item_issues_timestamp: [] item_issues_type: [] keywords: ~ lastmod: 2011-03-11 08:55:49 latitude: ~ longitude: ~ metadata_visibility: show note: ~ number: ~ pagerange: 101-110 pubdom: FALSE publication: 'Recent Advances in Natural Language Processing III: Selected Papers from RANLP 2003' publisher: John Benjamins refereed: TRUE referencetext: | Brill, Eric & Jun Wu. 1998. "Classifier Combination for Improved Lexical Disambiguation". Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’98), vol. 1, 191-195. Montreal, Canada. Chalmers, David J., Robert M. French, & Douglas R. Hofstadter. 1992. "High-level Perception, Representation and Analogy: A Critique of Artificial Intelligence Methodology". Journal of Experimental and Theoretical Artificial Intelligence 4:185-211. Fellbaum, Christiane. 1998. WordNet: An Electronic Lexical Database. Cambridge, Mass.: MIT Press. Florian, Radu & David Yarowsky. 2002. "Modeling Consensus: Classifier Combination for Word Sense Disambiguation". Proceedings of the Conference on Empirical Methods in Natural Language Processing, 25-32. Philadelphia. Heskes, Tom. 1998. "Selecting Weighting Factors in Logarithmic Opinion Pools". Advances in Neural Information Processing Systems 10:266-272. Hinton, Geoffrey E. 1999. "Products of Experts". Proceedings of the 9th International Conference on Artificial Neural Networks (ICANN 99), 1:1-6. Edinburgh, Scotland. Jacobs, Robert A. 1995. "Methods for Combining Experts’ Probability Assessments". Neural Computation 7:5.867-888. Jacobs, Robert A., Michael I. Jordan, Steve J. Nowlan & Geoffrey E. Hinton. 1991. "Adaptive Mixtures of Experts". Neural Computation 3:79-87. Jarmasz, Mario & Stan Szpakowicz. 2003. "Roget’s Thesaurus and Semantic Similarity". Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-03), 212-219. Borovets, Bulgaria. Lakoff, George & Mark Johnson. 1980. Metaphors We Live By. Chicago: University of Chicago Press. Landauer, Thomas K. & Susan T. Dumais. 1997. "A Solution to Plato’s Problem: The Latent Semantic Analysis Theory of Acquisition, Induction and Representation of Knowledge". Psychological Review 104:2.211-240. Littman, Michael L., Greg A. Keim & Noam Shazeer. 2002. "A Probabilistic Approach to Solving Crossword Puzzles". Artificial Intelligence 134:23-55. Schapire, Robert E. 1999. "A Brief Introduction to Boosting". Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI-99), 1401-1406. Stockholm, Sweden. Terra, Egidio & C. L.A. Clarke. 2003. "Frequency Estimates for Statistical Word Similarity Measures". Proceedings of the Human Language Technology and North American Chapter of Association of Computational Linguistics Conference 2003 (HLT/NAACL 2003), 244-251. Edmonton, Alberta, Canada. Turney, Peter D. 2001. "Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL". Proceedings of the 12th European Conference on Machine Learning (ECML-2001), 491-502. Freiburg, Germany. Turney, Peter D. & Michael L. Littman. 2003a. "Learning Analogies and Semantic Relations". Technical Report ERB-1103. Ottawa, Ontario, Canada: National Research Council, Institute for Information Technology. Turney, Peter D. & Michael L. Littman. 2003b. "Measuring Praise and Criticism: Inference of Semantic Orientation from Association". ACMTransactions on Information Systems 21:4.315-346. Turney, Peter D., Michael L. Littman, Jeffrey Bigham & Victor Shnayder. 2003. "Combining Independent Modules to SolveMultiple-Choice Synonym and Analogy Problems". Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-03), 482-489. Borovets, Bulgaria. Xu, Lei, Adam Krzyzak & Ching Y. Suen. 1992. "Methods of Combining Multiple Classifiers and Their Applications to Handwriting Recognition". IEEE Transactions on Systems, Man and Cybernetics 22:3.418-435. relation_type: [] relation_uri: [] reportno: ~ rev_number: 12 series: ~ source: ~ status_changed: 2007-09-12 16:55:08 subjects: - comp-sci-stat-model - comp-sci-lang - ling-comput - ling-sem - comp-sci-mach-learn succeeds: ~ suggestions: ~ sword_depositor: ~ sword_slug: ~ thesistype: ~ title: Combining Independent Modules in Lexical Multiple-Choice Problems type: bookchapter userid: 2175 volume: ~