Unsupervised Keyphrase Generation by Utilizing Masked Words Prediction and Pseudo-label BART Finetuning

A keyphrase is a short phrase of one or a few words that summarizes the key idea discussed in the document. Keyphrase generation is the process of predicting both present and absent keyphrases from a given document. Recent studies based on sequence-to-sequence (Seq2Seq) deep learning framework have been widely used in keyphrase generation. However, the excellent performance of these models on the keyphrase generation task is acquired at the expense of a large quantity of annotated documents. In this paper, we propose an unsupervised method called MLMPBKG, based on masked language model (MLM) and pseudo-label BART finetuning. We mask noun phrases in the article, and apply MLM to predict replaceable words. We observe that absent keyphrases can be found in these words. Based on the observation, we first propose MLMKPG, which utilizes MLM to generate keyphrase candidates and use a sentence embedding model to rank the candidate phrases. Furthermore, we use these top-ranked phrases as pseudo-labels to finetune BART for obtaining more absent keyphrases. Experimental results show that our method achieves remarkable results on both present and abstract keyphrase predictions, even surpassing supervised baselines in certain cases.