Exploiting the cognitive synergy between different media modalities in multimodal information retrieval

Zhongfei Zhang, Ruofei Zhang, Jun Ohya

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    5 Citations (Scopus)

    Abstract

    This is a position paper reporting an on-going collaboration project between SUNY Binghamton, USA, and Waseda University, Japan, on multimodal information retrieval through exploiting the cognitive synergy across the different modalities of the information to facilitate an effective retrieval. Specifically we focus on image retrieval in the applications where imagery data appear along with collateral text. It is noted that these applications are ubiquitous. We have proposed the Synergistic Indexing Scheme (SIS) to explicitly exploit the synergy between the information of imagery and text modalities. Since the synergy we have exploited between the information of imagery and text modalities is subjective and depends on specific cognitive context, we call this type of synergy as cognitive synergy. We have reported part of the empirical evaluation and are in the process to fully implement the SIS prototype for an extensive evaluation.

    Original languageEnglish
    Title of host publication2004 IEEE International Conference on Multimedia and Expo (ICME)
    Pages2227-2230
    Number of pages4
    Volume3
    Publication statusPublished - 2004
    Event2004 IEEE International Conference on Multimedia and Expo (ICME) - Taipei
    Duration: 2004 Jun 272004 Jun 30

    Other

    Other2004 IEEE International Conference on Multimedia and Expo (ICME)
    CityTaipei
    Period04/6/2704/6/30

      Fingerprint

    Keywords

    • A semantic graph
    • Abstract word
    • Cognitive synergy
    • Collateral text
    • Image retrieval
    • Inverted file
    • Multimodal information retrieval
    • Synergistic indexing scheme (SIS)
    • Visible word
    • Visual word

    ASJC Scopus subject areas

    • Engineering(all)

    Cite this

    Zhang, Z., Zhang, R., & Ohya, J. (2004). Exploiting the cognitive synergy between different media modalities in multimodal information retrieval. In 2004 IEEE International Conference on Multimedia and Expo (ICME) (Vol. 3, pp. 2227-2230)