Exploiting the cognitive synergy between different media modalities in multimodal information retrieval

Zhongfei Zhang, Ruofei Zhang, Jun Ohya

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    5 Citations (Scopus)

    Abstract

    This is a position paper reporting an on-going collaboration project between SUNY Binghamton, USA, and Waseda University, Japan, on multimodal information retrieval through exploiting the cognitive synergy across the different modalities of the information to facilitate an effective retrieval. Specifically we focus on image retrieval in the applications where imagery data appear along with collateral text. It is noted that these applications are ubiquitous. We have proposed the Synergistic Indexing Scheme (SIS) to explicitly exploit the synergy between the information of imagery and text modalities. Since the synergy we have exploited between the information of imagery and text modalities is subjective and depends on specific cognitive context, we call this type of synergy as cognitive synergy. We have reported part of the empirical evaluation and are in the process to fully implement the SIS prototype for an extensive evaluation.

    Original languageEnglish
    Title of host publication2004 IEEE International Conference on Multimedia and Expo (ICME)
    Pages2227-2230
    Number of pages4
    Volume3
    Publication statusPublished - 2004
    Event2004 IEEE International Conference on Multimedia and Expo (ICME) - Taipei
    Duration: 2004 Jun 272004 Jun 30

    Other

    Other2004 IEEE International Conference on Multimedia and Expo (ICME)
    CityTaipei
    Period04/6/2704/6/30

    Fingerprint

    Information retrieval
    Image retrieval

    Keywords

    • A semantic graph
    • Abstract word
    • Cognitive synergy
    • Collateral text
    • Image retrieval
    • Inverted file
    • Multimodal information retrieval
    • Synergistic indexing scheme (SIS)
    • Visible word
    • Visual word

    ASJC Scopus subject areas

    • Engineering(all)

    Cite this

    Zhang, Z., Zhang, R., & Ohya, J. (2004). Exploiting the cognitive synergy between different media modalities in multimodal information retrieval. In 2004 IEEE International Conference on Multimedia and Expo (ICME) (Vol. 3, pp. 2227-2230)

    Exploiting the cognitive synergy between different media modalities in multimodal information retrieval. / Zhang, Zhongfei; Zhang, Ruofei; Ohya, Jun.

    2004 IEEE International Conference on Multimedia and Expo (ICME). Vol. 3 2004. p. 2227-2230.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Zhang, Z, Zhang, R & Ohya, J 2004, Exploiting the cognitive synergy between different media modalities in multimodal information retrieval. in 2004 IEEE International Conference on Multimedia and Expo (ICME). vol. 3, pp. 2227-2230, 2004 IEEE International Conference on Multimedia and Expo (ICME), Taipei, 04/6/27.
    Zhang Z, Zhang R, Ohya J. Exploiting the cognitive synergy between different media modalities in multimodal information retrieval. In 2004 IEEE International Conference on Multimedia and Expo (ICME). Vol. 3. 2004. p. 2227-2230
    Zhang, Zhongfei ; Zhang, Ruofei ; Ohya, Jun. / Exploiting the cognitive synergy between different media modalities in multimodal information retrieval. 2004 IEEE International Conference on Multimedia and Expo (ICME). Vol. 3 2004. pp. 2227-2230
    @inproceedings{2dfc3322a7f140a08986033fafed3bb5,
    title = "Exploiting the cognitive synergy between different media modalities in multimodal information retrieval",
    abstract = "This is a position paper reporting an on-going collaboration project between SUNY Binghamton, USA, and Waseda University, Japan, on multimodal information retrieval through exploiting the cognitive synergy across the different modalities of the information to facilitate an effective retrieval. Specifically we focus on image retrieval in the applications where imagery data appear along with collateral text. It is noted that these applications are ubiquitous. We have proposed the Synergistic Indexing Scheme (SIS) to explicitly exploit the synergy between the information of imagery and text modalities. Since the synergy we have exploited between the information of imagery and text modalities is subjective and depends on specific cognitive context, we call this type of synergy as cognitive synergy. We have reported part of the empirical evaluation and are in the process to fully implement the SIS prototype for an extensive evaluation.",
    keywords = "A semantic graph, Abstract word, Cognitive synergy, Collateral text, Image retrieval, Inverted file, Multimodal information retrieval, Synergistic indexing scheme (SIS), Visible word, Visual word",
    author = "Zhongfei Zhang and Ruofei Zhang and Jun Ohya",
    year = "2004",
    language = "English",
    isbn = "0780386035",
    volume = "3",
    pages = "2227--2230",
    booktitle = "2004 IEEE International Conference on Multimedia and Expo (ICME)",

    }

    TY - GEN

    T1 - Exploiting the cognitive synergy between different media modalities in multimodal information retrieval

    AU - Zhang, Zhongfei

    AU - Zhang, Ruofei

    AU - Ohya, Jun

    PY - 2004

    Y1 - 2004

    N2 - This is a position paper reporting an on-going collaboration project between SUNY Binghamton, USA, and Waseda University, Japan, on multimodal information retrieval through exploiting the cognitive synergy across the different modalities of the information to facilitate an effective retrieval. Specifically we focus on image retrieval in the applications where imagery data appear along with collateral text. It is noted that these applications are ubiquitous. We have proposed the Synergistic Indexing Scheme (SIS) to explicitly exploit the synergy between the information of imagery and text modalities. Since the synergy we have exploited between the information of imagery and text modalities is subjective and depends on specific cognitive context, we call this type of synergy as cognitive synergy. We have reported part of the empirical evaluation and are in the process to fully implement the SIS prototype for an extensive evaluation.

    AB - This is a position paper reporting an on-going collaboration project between SUNY Binghamton, USA, and Waseda University, Japan, on multimodal information retrieval through exploiting the cognitive synergy across the different modalities of the information to facilitate an effective retrieval. Specifically we focus on image retrieval in the applications where imagery data appear along with collateral text. It is noted that these applications are ubiquitous. We have proposed the Synergistic Indexing Scheme (SIS) to explicitly exploit the synergy between the information of imagery and text modalities. Since the synergy we have exploited between the information of imagery and text modalities is subjective and depends on specific cognitive context, we call this type of synergy as cognitive synergy. We have reported part of the empirical evaluation and are in the process to fully implement the SIS prototype for an extensive evaluation.

    KW - A semantic graph

    KW - Abstract word

    KW - Cognitive synergy

    KW - Collateral text

    KW - Image retrieval

    KW - Inverted file

    KW - Multimodal information retrieval

    KW - Synergistic indexing scheme (SIS)

    KW - Visible word

    KW - Visual word

    UR - http://www.scopus.com/inward/record.url?scp=11244321849&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=11244321849&partnerID=8YFLogxK

    M3 - Conference contribution

    AN - SCOPUS:11244321849

    SN - 0780386035

    SN - 9780780386037

    VL - 3

    SP - 2227

    EP - 2230

    BT - 2004 IEEE International Conference on Multimedia and Expo (ICME)

    ER -