Previous work on Arabic information extraction has mainly focused on named entity recognition and very little work has been done on Arabic relation extraction and event recognition. Moreover, modeling Arabic data for such tasks is not straightforward because of the morphological richness and idiosyncrasies of the Arabic language. We propose in this article the first neural joint information extraction system for the Arabic language.

Extractive QA is an important NLP task with numerous real-world applications. The most common method for extractive QA is to encode the input sequence with a pretrained Transformer such as BERT, and then compute the probability of the start and end positions of span answers using two leaned query vectors. This method has been shown to be effective and hard to outperform. However, the query vectors are static, meaning they are the same regardless of the input, which can be a challenging issue in improving the model's performance. To address this problem, we propose \texttt{DyReF} (\texttt{Dy}namic \texttt{Re}presentation for \texttt{F}ree), a model that dynamically learns query vectors for free, i.e. without adding any parameters, by concatenating the query vectors with the embeddings of the input tokens of the Transformer layers. In this way, the query vectors can aggregate information from the source sentence and adapt to the question, while the representations of the input tokens are also dependent on the queries, allowing for better task specialization. We demonstrate empirically that our simple approach outperforms strong baseline in a variety of extractive question answering benchmark datasets. The code is publicly available at \url{https://github.com/urchade/DyReF}.

Extractive question answering (ExQA) is an essential task for Natural Language Processing. The dominant approach to ExQA is one that represents the input sequence tokens (question and passage) with a pre-trained transformer, then uses two learned query vectors to compute distributions over the start and end answer span positions. These query vectors lack the context of the inputs, which can be a bottleneck for the model performance. To address this problem, we propose \textit{DyREx}, a generalization of the \textit{vanilla} approach where we dynamically compute query vectors given the input, using an attention mechanism through transformer layers. Empirical observations demonstrate that our approach consistently improves the performance over the standard one. The code and accompanying files for running the experiments are available at \url{https://github.com/urchade/DyReX}.