Title
Query Interpretation – an Application of Semiotics in Image Retrieval
Author
de Boer, M.H.T.
Brandt, P.
Sappelli, M.
Daniele, L.M.
Schutte, K.
Kraaij, W.
Publication year
2015
Abstract
One of the challenges in the field of content-based image retrieval is to bridge the semantic gap that exists between the information extracted from visual data using classifiers, and the interpretation of this data made by the end users. The semantic gap is a cascade of 1) the transformation of image pixels into labelled objects and 2) the semantic distance between the label used to name the classifier and that what it refers to for the end-user. In this paper, we focus on the second part and specifically on (semantically) scalable solutions that are independent from domain-specific vocabularies. To this end, we propose a generic semantic reasoning approach that applies semiotics in its query interpretation. Semiotics is about how humans interpret signs, and we use its text analysis structures to guide the query expansion that we apply. We evaluated our approach using a general-purpose image search engine. In our experiments, we compared several semiotic structures to determine to what extent semiotic structures contribute to the semantic interpretation of user queries. From the results of the experiments we conclude that semiotic structures can contribute to a significantly higher semantic interpretation of user queries and significantly higher image retrieval performance, measured in quality and effectiveness and compared to a baseline with only synonym expansions.
Subject
Observation, Weapon & Protection Systems ; ICT
II - Intelligent Imaging ; MNS - Media & Network Services
TS - Technical Sciences
Infostructures
Informatics
Information Society
Query expansion
Natural language queries
Image retrieval
Semantic reasoning
Computational semiotic
GOOSE
To reference this document use:
http://resolver.tudelft.nl/uuid:d9712227-977c-4b2c-ae25-a934b1175c37
TNO identifier
531053
Source
International Journal on Advances in Software, 8 (3/4), 435-449
Document type
article