Development of Dialogue Management System for Banking Services
Date Issued
2021
Author(s)
Rustamov, Samir
Bayramova, Aygul
Alasgarov, Emin
Abstract
Rapid increase in conversational AI and user chat data lead to intensive development
of dialogue management systems (DMS) for various industries. Yet, for low-resource languages,
such as Azerbaijani, very little research has been conducted. The main purpose of this work is to
experiment with various DMS pipeline set-ups to decide on the most appropriate natural language
understanding and dialogue manager settings. In our project, we designed and evaluated different
DMS pipelines with respect to the conversational text data obtained from one of the leading retail
banks in Azerbaijan. In the work, the main two components of DMS—Natural language Understanding (NLU) and Dialogue Manager—have been investigated. In the first step of NLU, we utilized
a language identification (LI) component for language detection. We investigated both built-in LI
methods such as fastText and custom machine learning (ML) models trained on the domain-based
dataset. The second step of the work was a comparison of the classic ML classifiers (logistic regression,
neural networks, and SVM) and Dual Intent and Entity Transformer (DIET) architecture for user
intention detection. In these experiments we used different combinations of feature extractors such
as CountVectorizer, Term Frequency-Inverse Document Frequency (TF-IDF) Vectorizer, and word
embeddings for both word and character n-gram based tokens. To extract important information
from the text messages, Named Entity Extraction (NER) component was added to the pipeline. The
best NER model was chosen among conditional random fields (CRF) tagger, deep neural networks
(DNN), models and build in entity extraction component inside DIET architecture. Obtained entity
tags fed to the Dialogue Management module as features. All NLU set-ups were followed by the
Dialogue Management module that contains a Rule-based Policy to handle FAQs and chitchats as
well as a Transformer Embedding Dialogue (TED) Policy to handle more complex and unexpected
dialogue inputs. As a result, we suggest a DMS pipeline for a financial assistant, which is capable
of identifying intentions, named entities, and a language of text followed by policies that allow
generating a proper response (based on the designed dialogues) and suggesting the best next action.
of dialogue management systems (DMS) for various industries. Yet, for low-resource languages,
such as Azerbaijani, very little research has been conducted. The main purpose of this work is to
experiment with various DMS pipeline set-ups to decide on the most appropriate natural language
understanding and dialogue manager settings. In our project, we designed and evaluated different
DMS pipelines with respect to the conversational text data obtained from one of the leading retail
banks in Azerbaijan. In the work, the main two components of DMS—Natural language Understanding (NLU) and Dialogue Manager—have been investigated. In the first step of NLU, we utilized
a language identification (LI) component for language detection. We investigated both built-in LI
methods such as fastText and custom machine learning (ML) models trained on the domain-based
dataset. The second step of the work was a comparison of the classic ML classifiers (logistic regression,
neural networks, and SVM) and Dual Intent and Entity Transformer (DIET) architecture for user
intention detection. In these experiments we used different combinations of feature extractors such
as CountVectorizer, Term Frequency-Inverse Document Frequency (TF-IDF) Vectorizer, and word
embeddings for both word and character n-gram based tokens. To extract important information
from the text messages, Named Entity Extraction (NER) component was added to the pipeline. The
best NER model was chosen among conditional random fields (CRF) tagger, deep neural networks
(DNN), models and build in entity extraction component inside DIET architecture. Obtained entity
tags fed to the Dialogue Management module as features. All NLU set-ups were followed by the
Dialogue Management module that contains a Rule-based Policy to handle FAQs and chitchats as
well as a Transformer Embedding Dialogue (TED) Policy to handle more complex and unexpected
dialogue inputs. As a result, we suggest a DMS pipeline for a financial assistant, which is capable
of identifying intentions, named entities, and a language of text followed by policies that allow
generating a proper response (based on the designed dialogues) and suggesting the best next action.