2008 Quote by Hinrich Schütze
““A human is not a device that reliably reports a gold standard judgment of relevance of a document to a query.””
About This Quote
Source Academic paper: Information Retrieval, 2005
A document’s relevance to a query cannot be captured by a single, perfect judgment; relevance is subjective and context‑dependent.
In simple terms: Relevance judgments are imperfect and subjective.
Key Takeaway
Accept relevance uncertainty.
Themes
information retrieval
subjectivity
evaluation
bias
Mood
analytical
critical
Type
theoretical
philosophical
When to use this quote
- search engine design
- user studies
- content ranking
Key Concepts
relevance assessment
human judgment
Questions to Reflect On
- How can we capture nuanced relevance?
- What role should user feedback play?
A Different Perspective
Over‑reliance on metrics may ignore nuanced relevance.
