Understanding Question Answering

At the end of this article you will have an understanding of what Question Answering (QA) is, types of QA models and how they work.

Introduction

Remember when you were in high school, in English exams, you would have a section where you were expected to read a passage and answer the following questions based on your understanding of the passage (contexts); this is analogous to the Reading Comprehension(RC) problem in NLP. In RC tasks – a simpler form of QA – we provide the model, a question, and context, for the model to extract an answer from the context.

QA models retrieve answers to a question from a given body of text with or without context. So, as a subfield of artificial intelligence, QA is the process or task of building systems that respond to questions presented in natural language by people.

Types of Question Answering.

There are two main types of Question Answering systems:

1. Open Domain

2. Closed Domain

Each of these types can be divided into open and closed book QA systems

Open-Domain QA 

For open-domain QA, the models are not restricted to a specific domain. The texts are retrieved from sources such as books, the internet, Wikipedia, tables, graphs, knowledge bases, etc. 

Closed-Domain QA

In closed-domain QA, the models are geared towards a specific domain, such as documents focused on the legal and healthcare sectors.

Open-book QA.

Open-book QA is where the model has access to external sources of data to answer questions, including Wikipedia, internal/company documents, etc. It is similar to open-book exams, where students can access information in their books. 

Closed-book QA.

In closed-book QA, the model responds to questions without being explicitly given a context because it has learned some information encoded into its parameters during training.

Components of a QA System

Retriever

The Retriever model is used in Open-book QA because it extracts data from an external source. It retrieves a set of contexts for our QA model to extract answers. 

Reader

The Reader model takes a question and a context. Then it selects a span of the context, which is the answer to the question. Using the Reader’s model alone solves the reading comprehension (RC) QA task, but if it is used with the Retriever model, we have the Open-Domain QA model. 

Generator

This model doesn’t extract answers from a context; it generates text directly to answer the question using language models such as GPT-3 and optionally takes a context. 

Now that you know the components of QA systems, let’s examine how they all work together.

Approaches to Question-Answering

In this section, we will talk about the common architectures of QA. The quadrant above shows the combination of different components of QA systems and how they form different types of architectures, which also inform the classification of QA systems.

Retriever – Reader (Open-book Extractive QA)

The combination of the retriever and the reader model is used in open-book QA – the most common form of QA. As shown in the image above, the retriever takes the question and context, which is indexed in the database. It creates a query vector(vector representation) that is compared against all indexed context vectors in the database and returns the (n) most similar contexts. The reader model then takes in the most similar contexts and the question to provide a span selection of the answer.

Retriever – Generator (Open-book Abstractive QA)

Similar to extractive QA, the retriever model is still used in this system. The only difference is the reader model is replaced with a generator model. The generator model uses the question, alongside the contexts and its knowledge, to generate answers. Hence, it is called abstractive QA and open-book because the data source is external. 

We use this system to answer more abstract questions that don’t require exact answers. The system returns answers that are more suggestions or opinions, not direct.

Generator (Closed-book Abstractive QA)

This type of architecture is called closed-book abstractive QA because there is no external data source, and the model uses its knowledge to provide answers. Here we have removed the retriever and the reader models, and the generative model generates answers. 

Depending on the generative model used, we might not get the right answer to the question asked. So, it is recommended to use open-book approaches to get answers and not generate text(answers) following the provided question. 

GPT-3 and GPT-J are examples of generative models that can be used in this approach.

Conclusion

This article explained different variants of QA models and how they are classified. It also discussed the approaches to solving QA tasks. The key takeaway is a general understanding of the question-answering system as a foundation for more articles on QA and how Mantium is using QA to solve problems in document intelligence.