Skip to search boxSkip to navigationSkip to main content

Semantic Hilbert space for interactive image retrieval

  • Amit Kumar Jaiswal
    ,
  • Haiming Liu
    ,
  • Ingo Frommholz
Research Output: Chapter in Book/Report/Conference proceeding Conference contribution Peer-review

Abstract

The paper introduces a model for interactive image retrieval utilising the geometrical framework of information retrieval (IR). We tackle the problem of image retrieval based on an expressive user information need in form of a textual-visual query, where a user is attempting to find an image similar to the picture in their mind during querying. The user information need is expressed using guided visual feedback based on Information Foraging which lets the user perception embed within the model via semantic Hilbert space (SHS). This framework is based on the mathematical formalism of quantum probabilities and aims to understand the relationship between user textual and image input, where the image in the input is considered a form of visual feedback. We propose SHS, a quantum-inspired approach where the textual-visual query is regarded analogously to a physical system that allows for modelling different system states and their dynamic changes thereof based on observations (such as queries, relevance judgements). We will be able to learn the input multimodal representation and relationships between textual-image queries for retrieving images. Our experiments are conducted on the MIT States and Fashion200k datasets that demonstrate the effectiveness of finding particular images autocratically when the user inputs are semantically expressive.

Publication Information

Output type

Research Output: Chapter in Book/Report/Conference proceeding Conference contribution Peer-review

Original language

English

Pages from-to (Number of pages)

Pages 307-315 (9 pages)

Publication milestones

  • Published - 11/07/2021

Publication status

Published - 11/07/2021

Publisher

Association for Computing Machinery, United States

Publication series

  • Publication series name: ICTIR 2021 - Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval

ISBN (Print)

9781450386111

ISBN (Electronic)

9781450386111

External Publication IDs

  • handle.net: 10547/625258
  • Scopus: 85114510487

Host publication title

ICTIR 2021 - Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval

Access to documents

Publication metrics