Comparative Analysis of Mouse-clicking and Eye-tracking Dataset using Saliency Modeling on Fashion Advertisement Images

Author

Gokarna Karki, Southern Illinois University CarbondaleFollow

Date of Award

8-1-2023

Degree Name

Master of Science

Department

Biomedical Engineering

First Advisor

Qin, Jun

Abstract

Saliency modeling creates 2D topological maps that indicate the probabilistic distribution of visual attention priorities in an image. Saliency maps have numerous applications, including fashion design and advertising fields. In this study, we investigated two human subject studies, mouse-clicking and eye-tracking, in determining human visual attentionon 75 fashion images. Using the Gaussian blurring function, binary ground-truth fixation maps for these images were created based on experimentally collected human visual attention data. We conducted a comparison between the mouse-clicking and eye-tracking dataset over two baseline models. The DeepFeat model was thoroughly investigated and 11 sets of bottom-up saliency maps were formed utilizing the features extracted from GoogLeNet and ResNet50 networks. The mouse-clicking and eye-tracking datasets were compared within these different implementations of the DeepFeat model. Furthermore, we evaluated the performance of three deep learning-based saliency models and six conventional saliency models, on the mouse-clicking and eye-tracking dataset using six evaluation metrics, including KL, CC, SIM, NSS, AUC Judd, and IG. Our study suggests that both mouse-clicking and eye-tracking methods are effective in obtaining salient locations that closely align with most of the saliency maps in fashion advertisement images. The results of comparing datasets can be significantly impacted by the choice of evaluation metrics, as mouse-clicking data performs better with distribution-based metrics while eye-tracking data performs better with location-based metrics. Therefore, it is crucial to consider the type of metric used when comparing dataset. Our study sheds light on the effectiveness of both human subject studies and saliency models in predicting visual attention in fashion images, contributing to the fields of fashion design and advisement.