Extensive annotated bibliography of the most important works in Information Retrieval since 1997. Covers topics including TREC results, scaling issues, index compression, multilingual retrieval, multimedia retrieval, statistical, vector and probabalistic approaches, evaluation and testing, and much more.
An academic expert on usable information retrieval suggests that if web entrepreneurs and VCs had known about the history of IR and library experiences, they would not have wasted investments in problematic approaches such as "push" technology. She offers seven suggestions to improve web retrieval: use faceted rather than hierarchical classification; don't try for a single "true" classification (and avoid the term 'ontology'); use subject and domain information retrieval vocabulary; remember the Bradford distribution; plan for explosive growth; provide tools for "human content processing"; learn from the history of information retrieval.
- Some
Batesian Inspiration
Bloug, July 10, 2002 by Lou Rosenfeld
A leading expert in Information Architecture and librarian comments on Marcia Bates' article.
Looks at trends in web search between 1997 and 1999, including fewer terms in queries, fewer queries per "session", and users viewing fewer pages of results per query. Topics of searches also changed.
Short summaries of many technical articles relating to search and information retrieval presented at the Tenth World Wide Web Conference (May 1-5 2001, Hong Kong).
This experiment compared the standard information-retrieval algorithms used at TREC with Excite, Google, Lycos (FAST) and AltaVista. The experimenters used real queries from the Excite logs, limiting them to those for home pages or web sites, but not for obvious query word - matches with domain names such as "ibm". They then indexed and searched a 17.8 million web page collection using a successful standard TREC "ad-hoc" (interactive) algorithm. When they compared this to the commercial search engines, they found that the TREC algorithm was much less likely to find the home page desired. This is probably due to high-frequency matches of the query words in other pages.
Based on analysis of the Excite search engine session logs, finds that most searches are short and simple, about half search for one item per session, and almost people view only one or two results pages. While a small number of search terms are common, there are also many unique words. Most of these findings confirm the authors' earlier studies.
Research looked at short queries (2 to 4 terms), comparing the first 10 results for simple queries of just the words against complex queries with Boolean or other query operators. Given the large amount of overlap between these two modes (about 7 out of 10 were the same), concludes that the search engines do well enough with just simple queries. The search engines are implementing the heuristic rule: Place at the top of the results list, those documents that contain all the query terms and that have all the query terms near each other.
Proposes that new site search engines provide better search experiences by using metadata and categories to integrate browsing features with search results. Also mentions other aspects of web search, including question-answering.
Topics include rich query languages, search behavior, relevance ranking, context, XML and data mining.
Covers current research in information retrieval, classification, user interaction, evaluation, etc. Volume 36, number 2 concentrates on Web search topics.
Results of a small study on web-searching behavior, including cognitive strategies and interaction with the systems.
- Introduction to the current state of information retrieval, including changes brought by the Web to a field that was previously oriented towards academia, libraries and corporate networks. Most of the book is not online, but the chapter on User Interfaces and Visualization in Modern Information Retrieval, written by UC Berkeley professor Marti Hearst, provides a valuable academic study of interfaces for information retrieval and searching, including graphical overviews and visualization. Get the book from Amazon or FatBrain.com and we'll get an affiliate fee.