Exploiting the cognitive synergy between different media modalities in multimodal information retrieval

Zhongfei Zhang, Ruofei Zhang, Jun Ohya

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

This is a position paper reporting an on-going collaboration project between SUNY Binghamton, USA, and Waseda University, Japan, on multimodal information retrieval through exploiting the cognitive synergy across the different modalities of the information to facilitate an effective retrieval. Specifically we focus on image retrieval in the applications where imagery data appear along with collateral text. It is noted that these applications are ubiquitous. We have proposed the Synergistic Indexing Scheme (SIS) to explicitly exploit the synergy between the information of imagery and text modalities. Since the synergy we have exploited between the information of imagery and text modalities is subjective and depends on specific cognitive context, we call this type of synergy as cognitive synergy. We have reported part of the empirical evaluation and are in the process to fully implement the SIS prototype for an extensive evaluation.

Original languageEnglish
Title of host publication2004 IEEE International Conference on Multimedia and Expo (ICME)
Pages2227-2230
Number of pages4
Publication statusPublished - 2004 Dec 1
Event2004 IEEE International Conference on Multimedia and Expo (ICME) - Taipei, Taiwan, Province of China
Duration: 2004 Jun 272004 Jun 30

Publication series

Name2004 IEEE International Conference on Multimedia and Expo (ICME)
Volume3

Conference

Conference2004 IEEE International Conference on Multimedia and Expo (ICME)
CountryTaiwan, Province of China
CityTaipei
Period04/6/2704/6/30

Keywords

  • A semantic graph
  • Abstract word
  • Cognitive synergy
  • Collateral text
  • Image retrieval
  • Inverted file
  • Multimodal information retrieval
  • Synergistic indexing scheme (SIS)
  • Visible word
  • Visual word

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'Exploiting the cognitive synergy between different media modalities in multimodal information retrieval'. Together they form a unique fingerprint.

  • Cite this

    Zhang, Z., Zhang, R., & Ohya, J. (2004). Exploiting the cognitive synergy between different media modalities in multimodal information retrieval. In 2004 IEEE International Conference on Multimedia and Expo (ICME) (pp. 2227-2230). (2004 IEEE International Conference on Multimedia and Expo (ICME); Vol. 3).