Two-encoder pointer-generator network for summarizing segments of long articles

Junhao Li, Mizuho Iwaihara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Usually long documents contain many sections and segments. In Wikipedia, one article can usually be divided into sections and one section can be divided into segments. But although one article is already divided into smaller segments, one segment can still be too long to read. So, we consider that segments should have a short summary for readers to grasp a quick view of the segment. This paper discusses applying neural summarization models including Seq2Seq model and pointer generator network model to segment summarization. These models for summarization can take target segments as the only input to the model. However, in our case, it is very likely that the remaining segments in the same article contain descriptions related to the target segment. Therefore, we propose several ways to extract an additional sequence from the whole article and then combine with the target segment, to be supplied as the input for summarization. We compare the results against the original models without additional sequences. Furthermore, we propose a new model that uses two encoders to process the target segment and additional sequence separately. Our results show our two-encoder model outperforms the original models in terms of ROGUE and METEOR scores.

Original languageEnglish
Title of host publicationWeb and Big Data - 3rd International Joint Conference, APWeb-WAIM 2019, Proceedings
EditorsJie Shao, Wei Wang, Man Lung Yiu, Dongxiang Zhang, Bin Cui, Masashi Toyoda
PublisherSpringer-Verlag
Pages299-313
Number of pages15
ISBN (Print)9783030260712
DOIs
Publication statusPublished - 2019 Jan 1
Event3rd APWeb and WAIM Joint Conference on Web and Big Data, APWeb-WAIM 2019 - Chengdu, China
Duration: 2019 Aug 12019 Aug 3

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11641 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd APWeb and WAIM Joint Conference on Web and Big Data, APWeb-WAIM 2019
CountryChina
CityChengdu
Period19/8/119/8/3

Fingerprint

Encoder
Generator
Summarization
Target
Model
Wikipedia
Network Model
Likely

Keywords

  • Deep learning
  • Multi-encoder
  • Pointer generator network
  • Seq2Seq
  • Text summarization

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Li, J., & Iwaihara, M. (2019). Two-encoder pointer-generator network for summarizing segments of long articles. In J. Shao, W. Wang, M. L. Yiu, D. Zhang, B. Cui, & M. Toyoda (Eds.), Web and Big Data - 3rd International Joint Conference, APWeb-WAIM 2019, Proceedings (pp. 299-313). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11641 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-26072-9_23

Two-encoder pointer-generator network for summarizing segments of long articles. / Li, Junhao; Iwaihara, Mizuho.

Web and Big Data - 3rd International Joint Conference, APWeb-WAIM 2019, Proceedings. ed. / Jie Shao; Wei Wang; Man Lung Yiu; Dongxiang Zhang; Bin Cui; Masashi Toyoda. Springer-Verlag, 2019. p. 299-313 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11641 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, J & Iwaihara, M 2019, Two-encoder pointer-generator network for summarizing segments of long articles. in J Shao, W Wang, ML Yiu, D Zhang, B Cui & M Toyoda (eds), Web and Big Data - 3rd International Joint Conference, APWeb-WAIM 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11641 LNCS, Springer-Verlag, pp. 299-313, 3rd APWeb and WAIM Joint Conference on Web and Big Data, APWeb-WAIM 2019, Chengdu, China, 19/8/1. https://doi.org/10.1007/978-3-030-26072-9_23
Li J, Iwaihara M. Two-encoder pointer-generator network for summarizing segments of long articles. In Shao J, Wang W, Yiu ML, Zhang D, Cui B, Toyoda M, editors, Web and Big Data - 3rd International Joint Conference, APWeb-WAIM 2019, Proceedings. Springer-Verlag. 2019. p. 299-313. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-26072-9_23
Li, Junhao ; Iwaihara, Mizuho. / Two-encoder pointer-generator network for summarizing segments of long articles. Web and Big Data - 3rd International Joint Conference, APWeb-WAIM 2019, Proceedings. editor / Jie Shao ; Wei Wang ; Man Lung Yiu ; Dongxiang Zhang ; Bin Cui ; Masashi Toyoda. Springer-Verlag, 2019. pp. 299-313 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{4b51c57c08d34d6aba28565df893261e,
title = "Two-encoder pointer-generator network for summarizing segments of long articles",
abstract = "Usually long documents contain many sections and segments. In Wikipedia, one article can usually be divided into sections and one section can be divided into segments. But although one article is already divided into smaller segments, one segment can still be too long to read. So, we consider that segments should have a short summary for readers to grasp a quick view of the segment. This paper discusses applying neural summarization models including Seq2Seq model and pointer generator network model to segment summarization. These models for summarization can take target segments as the only input to the model. However, in our case, it is very likely that the remaining segments in the same article contain descriptions related to the target segment. Therefore, we propose several ways to extract an additional sequence from the whole article and then combine with the target segment, to be supplied as the input for summarization. We compare the results against the original models without additional sequences. Furthermore, we propose a new model that uses two encoders to process the target segment and additional sequence separately. Our results show our two-encoder model outperforms the original models in terms of ROGUE and METEOR scores.",
keywords = "Deep learning, Multi-encoder, Pointer generator network, Seq2Seq, Text summarization",
author = "Junhao Li and Mizuho Iwaihara",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-26072-9_23",
language = "English",
isbn = "9783030260712",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag",
pages = "299--313",
editor = "Jie Shao and Wei Wang and Yiu, {Man Lung} and Dongxiang Zhang and Bin Cui and Masashi Toyoda",
booktitle = "Web and Big Data - 3rd International Joint Conference, APWeb-WAIM 2019, Proceedings",

}

TY - GEN

T1 - Two-encoder pointer-generator network for summarizing segments of long articles

AU - Li, Junhao

AU - Iwaihara, Mizuho

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Usually long documents contain many sections and segments. In Wikipedia, one article can usually be divided into sections and one section can be divided into segments. But although one article is already divided into smaller segments, one segment can still be too long to read. So, we consider that segments should have a short summary for readers to grasp a quick view of the segment. This paper discusses applying neural summarization models including Seq2Seq model and pointer generator network model to segment summarization. These models for summarization can take target segments as the only input to the model. However, in our case, it is very likely that the remaining segments in the same article contain descriptions related to the target segment. Therefore, we propose several ways to extract an additional sequence from the whole article and then combine with the target segment, to be supplied as the input for summarization. We compare the results against the original models without additional sequences. Furthermore, we propose a new model that uses two encoders to process the target segment and additional sequence separately. Our results show our two-encoder model outperforms the original models in terms of ROGUE and METEOR scores.

AB - Usually long documents contain many sections and segments. In Wikipedia, one article can usually be divided into sections and one section can be divided into segments. But although one article is already divided into smaller segments, one segment can still be too long to read. So, we consider that segments should have a short summary for readers to grasp a quick view of the segment. This paper discusses applying neural summarization models including Seq2Seq model and pointer generator network model to segment summarization. These models for summarization can take target segments as the only input to the model. However, in our case, it is very likely that the remaining segments in the same article contain descriptions related to the target segment. Therefore, we propose several ways to extract an additional sequence from the whole article and then combine with the target segment, to be supplied as the input for summarization. We compare the results against the original models without additional sequences. Furthermore, we propose a new model that uses two encoders to process the target segment and additional sequence separately. Our results show our two-encoder model outperforms the original models in terms of ROGUE and METEOR scores.

KW - Deep learning

KW - Multi-encoder

KW - Pointer generator network

KW - Seq2Seq

KW - Text summarization

UR - http://www.scopus.com/inward/record.url?scp=85069983126&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069983126&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-26072-9_23

DO - 10.1007/978-3-030-26072-9_23

M3 - Conference contribution

AN - SCOPUS:85069983126

SN - 9783030260712

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 299

EP - 313

BT - Web and Big Data - 3rd International Joint Conference, APWeb-WAIM 2019, Proceedings

A2 - Shao, Jie

A2 - Wang, Wei

A2 - Yiu, Man Lung

A2 - Zhang, Dongxiang

A2 - Cui, Bin

A2 - Toyoda, Masashi

PB - Springer-Verlag

ER -