creators_name: Cao, Huanhuan creators_name: Jiang, Daxin creators_name: Pei, Jian creators_name: Chen, Enhong creators_name: Li, Hang type: conference_item datestamp: 2009-04-06 19:08:54 lastmod: 2009-04-22 09:59:55 metadata_visibility: show title: Towards Context-Aware Search by Learning a Very Large Variable Length Hidden Markov Model from Search Logs ispublished: pub full_text_status: public pres_type: paper abstract: Capturing the context of a user’s query from the previous queries and clicks in the same session may help understand the user’s information need. A context-aware approach to document re-ranking, query suggestion, and URL recommendation may improve users’ search experience substantially. In this paper, we propose a general approach to context-aware search. To capture contexts of queries, we learn a variable length Hidden Markov Model (vlHMM) from search sessions extracted from log data. Although the mathematical model is intuitive, how to learn a large vlHMM with millions of states from hundreds of millions of search sessions poses a grand challenge. We develop a strategy for parameter initialization in vlHMM learning which can greatly reduce the number of parameters to be estimated in practice. We also devise a method for distributed vlHMM learning under the map-reduce model. We test our approach on a real data set consisting of 1.8 billion queries, 2.6 billion clicks, and 840 million search sessions, and evaluate the effectiveness of the vlHMM learned from the real data on three search applications: document re-ranking, query suggestion, and URL recommendation. The experimental results show that our approach is both effective and efficient. date: 2009-04 pagerange: 191-191 event_title: 18th International World Wide Web Conference event_location: Madrid, Spain event_dates: April 20th-24th, 2009 event_type: conference refereed: TRUE citation: Cao, Huanhuan and Jiang, Daxin and Pei, Jian and Chen, Enhong and Li, Hang (2009) Towards Context-Aware Search by Learning a Very Large Variable Length Hidden Markov Model from Search Logs. In: 18th International World Wide Web Conference, April 20th-24th, 2009, Madrid, Spain. document_url: http://www2009.eprints.org/20/1/p191.pdf document_url: http://www2009.eprints.org/20/2/vlHMM.ppt