On the Task of Finding One Highly Relevant Document with High Precision

スポンサーリンク

概要

• 論文の詳細を見る

• This paper examines the problem of evaluating systems that aim at finding one highly relevant document with high precision. Such a task is important for modern search environments such as the Web where recall is unimportant and/or unmeasurable. Reciprocal Rank is a metric designed for finding one relevant document with high precision, but it can only handle binary relevance. We therefore introduce a new metric called O-measure, for high precision, high relevance search, and show (a) How the task of finding one relevant document is different from that of finding as many relevant documents as possible, in both binary and graded relevance settings; and (b) How the task of finding one highly relevant document is different from that of finding any one relevant document. We use four test collections and the corresponding sets of formal runs from the NTCIR-3 Crosslingual Information Retrieval track to compare the tasks and metrics in terms of resemblance, stability and discrimination power in system ranking.

• 一般社団法人情報処理学会の論文
• 2006-03-15

著者

• Sakai Tetsuya

  Newswatch Inc.

• SAKAI TETSUYA

  Toshiba Corporate R&D Center

• SAKAI TETSUYA

  Toshiba Corporate R&D Center

関連論文

• Evaluating Information Retrieval Metrics Based on Bootstrap Hypothesis Tests
• A further note on alternatives to Bpref (情報学基礎)
• On the Task of Finding One Highly Relevant Document with High Precision
• Comparing Metrics across TREC and NTCIR : The Robustness to System Bias
• Comparing metrics across TREC and NTCIR: the robustness to system bias (データベースシステム・情報学基礎)

スポンサーリンク