Argument Mining with Modular BERT and Transfer Learning

Abstrakt

We introduce BERT-MINUS, a modular, feature-enriched and transfer learning enabled model for Argument Mining. BERT-MINUS consists of: 1) a joint module which embeds the paragraph text, and 2) a dedicated module, consisting of three customized BERT models, which contextualize the argument markers, argument components and additional features given as text, respectively. BERT-MINUS implements two kinds of transfer learning - auto-transfer (transfer from a task to itself) and cross-transfer (classical transfer from one task to another) - via a novel Selective Fine-tuning mechanism. BERT-MINUS achieves state-of-the-art results on the Link Identification task and competitive results on the Argument Type Classification task. The synergy between the Features as Text and Selective Fine-tuning mechanisms significantly improves the performance of the model. Our work reveals the importance and potential of transfer learning via selective fine-tuning for modular Language Models. Moreover, this study dovetails naturally into the Prompt Engineering paradigm in NLP.