EXAMINING INDIVIDUAL DOCUMENT EVALUATION: AN EYE-TRACKING ANALYSIS

Abstract

Previous research on relevance judgment while searching for information on the web has been investigated in various contexts by scholars. This dissertation examines the behavior of users seeking health-related information online, specifically how those users operationalize both list and document evaluation. It focuses especially on document evaluation because it reflects the symbiotic relationship between the computer and the human. One of the broader aims of this study is to understand how users’ relevance judgments change during individual document evaluation. It examines whether there are any associations between eye-tracking variables and the perceived importance of each document. The perceived importance of each document element was measured using the ordinal scale while the gaze behavior was measured by using fixation-related eye-tracking variables with a ratio scale. Moreover, this study triangulates qualitative data about users’ perceptions with quantitative data. The results reveal a close relationship between the perceived importance of each document element and eye-tracking variables, specifically, showing a relationship between eye-tracking measurements and cognitive level of relevance. This study suggests that future researchers should be aware of the relationship when designing studies on relevance or element research. Finally, these findings enhance Saracevic’s stratified model on relevance in terms of confirming the crucial role of users’ cognitive relevance.