This paper examines the reliability of implicit feedback derived from clickthrough data in web search. Using eye-tracking and comparing implicit feedback with manual relevance judgments, the authors conclude that clicks are informative but biased. While clicks are not absolute relevance judgments, relative preferences derived from clicks are reasonably accurate on average. The study involved two user studies to understand how users interact with search results and how their behavior can be interpreted as relevance judgments. Eye-tracking was used to analyze user behavior on Google's results page, revealing that users scan results from top to bottom, and that clicks are influenced by both the relevance of the content and the overall quality of the results. Users tend to click more on higher-ranked links, indicating a "trust bias," and are more likely to click on links that appear more relevant, indicating a "quality bias." The study also shows that clicks can be interpreted as relative relevance judgments, and several strategies for extracting such judgments from clicks were proposed. These strategies were compared with explicit relevance judgments, and it was found that they agree reasonably well with explicit judgments. However, the accuracy of these strategies varies depending on the quality of the search results. The paper also discusses the limitations of using implicit feedback, such as the potential for noise and the need for careful interpretation. The authors suggest that machine learning methods for pairwise preferences can help improve the accuracy of implicit feedback. The study highlights the importance of considering both the trust users have in the search engine and the quality of the results when interpreting clicks as relevance feedback. Overall, the findings suggest that while clicks are not perfect indicators of relevance, they can be used effectively to generate relative relevance judgments in web search.This paper examines the reliability of implicit feedback derived from clickthrough data in web search. Using eye-tracking and comparing implicit feedback with manual relevance judgments, the authors conclude that clicks are informative but biased. While clicks are not absolute relevance judgments, relative preferences derived from clicks are reasonably accurate on average. The study involved two user studies to understand how users interact with search results and how their behavior can be interpreted as relevance judgments. Eye-tracking was used to analyze user behavior on Google's results page, revealing that users scan results from top to bottom, and that clicks are influenced by both the relevance of the content and the overall quality of the results. Users tend to click more on higher-ranked links, indicating a "trust bias," and are more likely to click on links that appear more relevant, indicating a "quality bias." The study also shows that clicks can be interpreted as relative relevance judgments, and several strategies for extracting such judgments from clicks were proposed. These strategies were compared with explicit relevance judgments, and it was found that they agree reasonably well with explicit judgments. However, the accuracy of these strategies varies depending on the quality of the search results. The paper also discusses the limitations of using implicit feedback, such as the potential for noise and the need for careful interpretation. The authors suggest that machine learning methods for pairwise preferences can help improve the accuracy of implicit feedback. The study highlights the importance of considering both the trust users have in the search engine and the quality of the results when interpreting clicks as relevance feedback. Overall, the findings suggest that while clicks are not perfect indicators of relevance, they can be used effectively to generate relative relevance judgments in web search.