TY  - GEN
ID  - cogprints1796
UR  - http://cogprints.org/1796/
A1  - Turney, Peter
Y1  - 2001///
N2  - This paper presents a simple unsupervised learning algorithm for recognizing synonyms, based on statistical data acquired by querying a Web search engine. The algorithm, called PMI-IR, uses Pointwise Mutual Information (PMI) and Information Retrieval (IR) to measure the similarity of pairs of words. PMI-IR is empirically evaluated using 80 synonym test questions from the Test of English as a Foreign Language (TOEFL) and 50 synonym test questions from a collection of tests for students of English as a Second Language (ESL). On both tests, the algorithm obtains a score of 74%. PMI-IR is contrasted with Latent Semantic Analysis (LSA), which achieves a score of 64% on the same 80 TOEFL questions. The paper discusses potential applications of the new unsupervised learning algorithm and some implications of the results for LSA and LSI (Latent Semantic Indexing). 


PB  - Springer-Verlag
KW  - PMI-IR
KW  -  synonyms
KW  -  LSA
KW  -  LSI
KW  -  Latent Semantic Analysis
KW  -  text mining
KW  -  web mining
KW  -  TOEFL
KW  -  mutual information
TI  - Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL
SP  - 491
AV  - public
EP  - 502
ER  -

<script src="//archive-bar.soton.ac.uk/archive-bar.js"></script>