Cogprints

Learning algorithms for keyphrase extraction

Turney, Peter (2000) Learning algorithms for keyphrase extraction. [Journal (Paginated)]

Full text available as:

[img]
Preview
 Postscript
3457Kb
[img] PDF
267Kb

Abstract

Many academic journals ask their authors to provide a list of about five to fifteen keywords, to appear on the first page of each article. Since these key words are often phrases of two or more words, we prefer to call them keyphrases. There is a wide variety of tasks for which keyphrases are useful, as we discuss in this paper. We approach the problem of automatically extracting keyphrases from text as a supervised learning task. We treat a document as a set of phrases, which the learning algorithm must learn to classify as positive or negative examples of keyphrases. Our first set of experiments applies the C4.5 decision tree induction algorithm to this learning task. We evaluate the performance of nine different configurations of C4.5. The second set of experiments applies the GenEx algorithm to the task. We developed the GenEx algorithm specifically for automatically extracting keyphrases from text. The experimental results support the claim that a custom-designed algorithm (GenEx), incorporating specialized procedural domain knowledge, can generate better keyphrases than a general-purpose algorithm (C4.5). Subjective human evaluation of the keyphrases generated by GenEx suggests that about 80% of the keyphrases are acceptable to human readers. This level of performance should be satisfactory for a wide variety of applications.

Item Type:Journal (Paginated)
Keywords:machine learning, summarization, indexing, keywords, keyphrase extraction.
Subjects:Computer Science > Language
Computer Science > Machine Learning
Computer Science > Statistical Models
ID Code:1797
Deposited By: Turney, Peter
Deposited On:13 Sep 2001
Last Modified:11 Mar 2011 08:54

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

Brandow, R., Mitze, K., and Rau, L.R. (1995). The automatic condensation of electronic publications

by sentence selection. Information Processing and Management, 31 (5), 675-685.

Breiman, L. (1996a). Arcing Classifiers. Technical Report 460, Statistics Department, University of

California at Berkeley.

Breiman, L. (1996b). Bagging predictors. Machine Learning, 24 (2), 123-140.

Buntine, W. (1989). Stratifying samples to improve learning. In Proceedings of the IJCAI-89 Work-shop

on Knowledge Discovery in Databases. Detroit, Michigan.

Carter, C., and Catlett, J. (1987). Assessing credit card applications using machine learning. IEEE

Expert, Fall issue, 71-79.

Catlett, J. (1991). Megainduction: Machine Learning on Very Large Databases. Ph.D. Dissertation,

Basser Department of Computer Science, University of Sydney.

Croft, W.B., Turtle, H., and Lewis, D. (1991). The use of phrases and structured queries in information

retrieval. SIGIR-91: Proceedings of the 14th Annual International ACM SIGIR Conference on

Research and Development in Information Retrieval, pp. 32-45, New York: ACM.

Deming, W.E. (1978). Sample surveys: The field. In William H. Kruskal and Judith M. Tanur (Ed.),

International Encyclopedia of Statistics. New York: Free Press.

Edmundson, H.P. (1969). New methods in automatic extracting. Journal of the Association for Com-puting

Machinery, 16 (2), 264-285.

Fagan, J.L. (1987). Experiments in Automatic Phrase Indexing for Document Retrieval: A Comparison

of Syntactic and Non-Syntactic Methods. Ph.D. Dissertation, Department of Computer Science,

Cornell University, Report #87-868, Ithaca, New York.

Feelders, A. and Verkooijen, W. (1995). Which method learns the most from data? Methodological

issues in the analysis of comparative studies. Fifth International Workshop on Artificial Intelli-gence

and Statistics, Ft. Lauderdale, Florida, pp. 219-225.

Field, B.J. (1975). Towards automatic indexing: Automatic assignment of controlled-language index-ing

and classification from free indexing. Journal of Documentation, 31 (4), 246-265.

Frank, E., Paynter, G.W., Witten, I.H., Gutwin, C., and Nevill-Manning, C.G. (1999). Domain-specific

keyphrase extraction. Proceedings of the Sixteenth International Joint Conference on Artificial

Intelligence (IJCAI-99), pp. 668-673. California: Morgan Kaufmann.

Fraser, D.A.S. (1976). Probability and Statistics: Theory and Applications. Massachusetts: Duxbury

Press.

Freund, Y., and Schapire, R.E. (1996). Experiments with a new boosting algorithm. Machine Learn-ing:

Proceedings of the Thirteenth International Conference (ICML-96), pp. 148-156. California:

Morgan Kaufmann.

Ginsberg, A. (1993). A unified approach to automatic indexing and information retrieval. IEEE

Expert, 8, 46-56.

Grefenstette, J.J. (1983). A user’s guide to GENESIS. Technical Report CS-83-11, Computer Science

Department, Vanderbilt University.

Grefenstette, J.J. (1986). Optimization of control parameters for genetic algorithms. IEEE Transac-tions

on Systems, Man, and Cybernetics, 16, 122-128.

Gutwin, C., Paynter, G.W., Witten, I.H., Nevill-Manning, C.G., and Frank, E. (1999). Improving

browsing in digital libraries with keyphrase indexes. Decision Support Systems. In press.

Jang, D.-H., and Myaeng, S.H. (1997). Development of a document summarization system for effec-tive

information services. RIAO 97 Conference Proceedings: Computer-Assisted Information

Searching on Internet, pp. 101-111. Montreal, Canada.

Johnson, F.C., Paice, C.D., Black, W.J., and Neal, A.P. (1993). The application of linguistic process-ing

to automatic abstract generation. Journal of Document and Text Management, 1, 215-241.

Krovetz, R. (1993). Viewing morphology as an inference process. Proceedings of the Sixteenth Annual

International ACM SIGIR Conference on Research and Development in Information Retrieval,

SIGIR'93, 191-203.

Krulwich, B., and Burkey, C. (1996). Learning user information interests through the extraction of

semantically significant phrases. In M. Hearst and H. Hirsh, editors, AAAI 1996 Spring Sympo-sium

on Machine Learning in Information Access. California: AAAI Press.

Krupka, G. (1995). SRA: Description of the SRA system as used for MUC-6. Proceedings of the Sixth

Message Understanding Conference. California: Morgan Kaufmann.

Kubat, M., Holte, R., and Matwin, S. (1998). Machine learning for the detection of oil spills in satel-lite

radar images. Machine Learning, 30 (2/3), 195-215.

Kupiec, J., Pedersen, J., and Chen, F. (1995). A trainable document summarizer. In E.A. Fox, P. Ingw-ersen,

and R. Fidel, editors, SIGIR-95: Proceedings of the 18th Annual International ACM SIGIR

Conference on Research and Development in Information Retrieval, pp. 68-73, New York: ACM.

Leung, C.-H., and Kan, W.-K. (1997). A statistical learning approach to automatic indexing of con-trolled

index terms. Journal of the American Society for Information Science, 48 (1), 55-66.

Lovins, J.B. (1968). Development of a stemming algorithm. Mechanical Translation and Computa-tional

Linguistics, 11, 22-31.

Luhn, H.P. (1958). The automatic creation of literature abstracts. I.B.M. Journal of Research and

Development, 2 (2), 159-165.

Maclin, R., and Opitz, D. (1997). An empirical evaluation of bagging and boosting. Proceedings of the

Fourteenth National Conference on Artificial Intelligence (AAAI-97), pp 546-551. AAAI Press.

Marsh, E., Hamburger, H., and Grishman, R. (1984). A production rule system for message summari-zation.

In AAAI-84, Proceedings of the American Association for Artificial Intelligence, pp. 243-

246. Cambridge, MA: AAAI Press/MIT Press.

Mathieu, J. (1999). Adaptation of a keyphrase extractor for Japanese text. Proceedings of the 27th

Annual Conference of the Canadian Association for Information Science (CAIS-99), Sherbrooke,

Quebec, pp. 182-189.

MUC-3. (1991). Proceedings of the Third Message Understanding Conference. California: Morgan

Kaufmann.

MUC-4. (1992). Proceedings of the Fourth Message Understanding Conference. California: Morgan

Kaufmann.

MUC-5. (1993). Proceedings of the Fifth Message Understanding Conference. California: Morgan

Kaufmann.

MUC-6. (1995). Proceedings of the Sixth Message Understanding Conference. California: Morgan

Kaufmann.

Muñoz, A. (1996). Compound key word generation from document databases using a hierarchical

clustering ART model. Intelligent Data Analysis, 1 (1), Amsterdam: Elsevier.

Nakagawa, H. (1997). Extraction of index words from manuals. RIAO 97 Conference Proceedings:

Computer-Assisted Information Searching on Internet, pp. 598-611. Montreal, Canada.

Paice, C.D. (1990). Constructing literature abstracts by computer: Techniques and prospects. Informa-tion

Processing and Management, 26 (1), 171-186.

Paice, C.D., and Jones, P.A. (1993). The identification of important concepts in highly structured tech-nical

papers. SIGIR-93: Proceedings of the 16th Annual International ACM SIGIR Conference on

Research and Development in Information Retrieval, pp. 69-78, New York: ACM.

Porter, M.F. (1980). An algorithm for suffix stripping. Program; Automated Library and Information

Systems, 14 (3), 130-137.

Quinlan, J.R. (1987). Decision trees as probabilistic classifiers. In Langley, P. (Ed.), Proceedings of

the Fourth International Workshop on Machine Learning, pp. 31-37. California: Morgan Kauf-mann.

Quinlan, J.R. (1990). Probabilistic decision trees. In Y. Kodratoff and R.S. Michalski, (Eds.), Machine

Learning: An Artificial Intelligence Approach, Volume III, pp. 140-152, California: Morgan Kauf-mann.

Quinlan, J.R. (1993). C4.5: Programs for machine learning. California: Morgan Kaufmann.

Quinlan, J.R. (1996). Bagging, boosting, and C4.5. In Proceedings of the Thirteenth National Confer-ence

on Artificial Intelligence (AAAI-96), pp. 725-730. AAAI Press.

Salton, G. (1988). Syntactic approaches to automatic book indexing. Proceedings of the 26th Annual

Meeting of the Association for Computational Linguistics, pp. 120-138. New York: ACM.

Salton, G., Allan, J., Buckley, C., and Singhal, A. (1994). Automatic analysis, theme generation, and

summarization of machine-readable texts. Science, 264, 1421-1426.

Soderland, S., and Lehnert, W. (1994). Wrap-Up: A trainable discourse module for information

extraction. Journal of Artificial Intelligence Research, 2, 131-158.

Sparck Jones, K. (1973). Does indexing exhaustivity matter? Journal of the American Society for

Information Science, September-October, 313-316.

Steier, A. M., and Belew, R. K. (1993). Exporting phrases: A statistical analysis of topical language.

In R. Casey and B. Croft, editors, Second Symposium on Document Analysis and Information

Retrieval, pp. 179-190.

Turney, P.D. (1997). Extraction of Keyphrases from Text: Evaluation of Four Algorithms. National

Research Council, Institute for Information Technology, Technical Report ERB-1051.

Turney, P.D. (1999). Learning to Extract Keyphrases from Text. National Research Council, Institute

for Information Technology, Technical Report ERB-1057.

Whitley, D. (1989). The GENITOR algorithm and selective pressure. Proceedings of the Third Inter-national

Conference on Genetic Algorithms (ICGA-89), pp. 116-121. California: Morgan Kauf-mann.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.