Chatbots are now used in many applications for different purposes. The popularity of this type widget can be estimated from this fact:
As of August 2019, search results on Google for the following keywords:
chatbot – Volume: 246,000 searches per month and found 32,700,000 results
neural net – Volume 3600 searches per month and 127,000,000 results
One of the key components of chatbot architecture is Dialog Management. In order to incorporate some degree of uncertainty in classifying intents and entities, partially observable Markov decision process (POMDP) algorithm was proposed for building chatbots with Dialog Management. [1],[2]
How can we run POMDP? There is a package in R for this.
But what if we want run from python? One of the way could be call R from python. Here is how it can be done.
First we need to create R program. For running POMDP in R we need first download and install (only one time). We also need set working directory (if it is different than default) and specify pdf file name – this is where the R
script will output charts and diagrams. See below lines for this:
r = getOption("repos")
r["CRAN"] = "http://cran.us.r-project.org"
options(repos = r)
setwd ("C://Users//username//Documents")
## the below line need to be run only first time and it must be run with admin priv.
##install.packages("pomdp")
library("pomdp")
pdf('rplot123.pdf')
Now we can continue with our R program. Documentation for POMDP [3] has R code for tiger problem, that we can insert here.
Running POMDP R Package from Python
Now we need to create python script that will call R script. In addition to calling the R program, python will also display output from running R program. This is possible because when we call R program, we save output within txt file. So the python script will save output into file and also output on screen.
# -*- coding: utf-8 -*-
import os
os.system('"C:\\Program Files\\R\\R-3.4.3\\bin\\Rscript" C:\\Users\\username\\POMDP_R\\r_example.r > C:\\Users\\username\\output_file_ex.txt')
with open('output_file_ex.txt', 'r') as reader:
print(reader.read())
Now we can run R package POMDP or any other R program from python and start build advanced chatbot dialog system.
In the previous post How to Create a Chatbot with ChatBot Open Source and Deploy It on the Web I wrote how to deploy ChatterBot on pythonanywhere hosting site with Django webfamework. In this post we will look at few useful chatbots examples for implementing logic in our chatbot. This chatbot was developed in the previous post and is based on ChatterBot python library.
Making Chatbot Start Conversation with Specific Question
Suppose we want to start conversation with specific sentence that chatbot needs to show. For example, when the user open website, the chatbot can start with the specific question: “Did you find what you were looking for?”
Or, if you are building chatbot for conversation about previous day/week work, you probably want to start with “How was you previous day/week in terms of progress to goal?” How can we do this with ChatterBot framework?
Conversation diagram
It turns out, that ChatterBot has several logic adapters that allow to build conversation per different requirements.
Here is how I used logic adapter SpecificResponseAdapter for chatbot to start with initial predefined question:
chatbot = ChatBot("mybot",
logic_adapters=[
{
'import_path': 'chatterbot.logic.SpecificResponseAdapter',
'input_text': 'prev_day_disk',
'output_text': 'How much did you do toward your goal on previous day?'
}
.....
In views.py that was created in prev. post[1], I put input “prev_day_disk” to runbot instead of blank string. Because in the beginning of chat there is no user input and I used this to enter input_text and get desired output as specified in output_text.
SpecificResponseAdapter can be used also in other places of conversation (not just in the beginning). For example we could use criteria if there is no input from user during 15 secs and user is not typing anything (not sure yet how easily it is to check if user typing or not) then switch conversation to new topic by making chatbot app send new question.
How to Add Intelligence to Chatbot App
After the user replied to response how was his/her week, I want chatbot to be able to recognize the response as belonging to one of the 3 groups: bad, so-so, good. This is a machine learning classification problem. However here we will not create text classification algorithm, instead we will use built in functionality.
We will use another logic adapter, called BestMatch.
With this adapter we need specify statement_comparison_function and response_selection_method :
chatbot = ChatBot("mybot",
logic_adapters=[
{
'import_path': 'chatterbot.logic.SpecificResponseAdapter',
'input_text': 'How much did you do toward your goal on previous day?',
'output_text': 'How much did you do toward your goal on previous day?'
},
{
"import_path": "chatterbot.logic.BestMatch",
"statement_comparison_function": "chatterbot.comparisons.levenshtein_distance",
"response_selection_method": "chatterbot.response_selection.get_first_response"
}
Best Match Adapter – is a logic adapter that returns a response based on known responses to the closest matches to the input statement. [2]
The best match adapter uses a function to compare the input statement to known statements. Once it finds the closest match to the input statement, it uses another function to select one of the known responses to that statement.
To use this adapter for the above example I need at minimum create 1 samples per each group. In the below example I used following for testing of chatbot on 2 groups (skipped so-so).
if (train_bot == True):
chatbot.train([
"I did not do much this week",
"Did you run into the problems with programs or just did not have time?"
])
chatbot.train([
"I did a lot of progress",
"Fantastic! Keep going on"
])
After the training, if the user enters something close to “I did not do much this week” the chatbot will respond with “Did you run into the problems with programs or just did not have time?”, and if user enters something like “Did a lot of progress” the bot response will be “Fantastic! Keep going on” even if the input is slightly different from training.
So we looked how to build a chatbot with logic that makes chatbot able to ask questions as needed or classify user input in several buckets and respond to user input accordingly. The code for this post is provided at the link listed below[3]
Chatbots have become very popular due to progress in AI, ML and NLP. They are now used on many websites. With increased popularity of chatbots there are many different frameworks to create chatbot. We will explore one of such framework in this post. We will review how to create a chatbot and deploy online based on open source for ChatterBot. Our platform will be Django on pythonanywhere.
This chatbot tutorial introduces the basic concepts and terms needed to understand and deploy chatbot ChatterBot and provides a simple usage example.
What is ChatterBot?
ChatterBot is a Python library that makes it easy to generate automated responses to a user’s input. To produce different types of responses ChatterBot uses a selection of machine learning algorithms such as search and /or classification algorithms. This makes it easy for developers to create chat bots and automate conversations with users. [1]
To install this chatbot open source you need just run:
pip install chatterbot
ChatterBot Features
The functionality of chatterBot is provided through adapters – a pluggable class that allows a ChatBot instance to execute some kind of functionality.
ChatBot has the following groups of adapters
input adapters
output adapters
storage adapters
logic adapters
Inside of each group there several adapters that support different functionalities. For example within logic adapters we can use Best Match Adapter, Time Logic Adapter and few other.
Here is the example how do you run chatbot. Below is the output of running the code:
Result of program run:
USER: How are you doing?
BOT:I am fine
USER: Hello
BOT:Hi there!
USER: Good morning!
BOT:How are today!
USER: Do you like machine learning?
BOT:Yes, I like machine learning
USER: How do I make a neural network?
BOTI am sorry, but I do not understand.
USER: Let us talk about current activities
BOT:What are working now?
USER: I am just browsing Internet for news
BOT:What a waste of time! Dont you have any other things to do?
USER: I am working on python script to make new chatbot
BOT:This is great. Keep working on this
USER: Bye
BOT:Bye
During the testing I confirmed that it correctly responds to new (but similar) inputs even if we did not train exactly.
Once you trained the bot, the result of training is staying there even after turning off/on PC. So if you run program multiple times, you need run training just first time. You still can run training later, for example if you want retrain or update python chatbot code.
Deployment
This section describes how to deploy ChatterBot on pythonanywhere web hosting site with Django programming web framework.
PythonAnywhere is an online integrated development environment (IDE) and web hosting service based on the Python programming language.[3] It has free account that allows to deploy our Chatbot. Django is needed to connect web front interface with our python code for ChatterBot in the backend. Other web frameworks like Flask can be used on pythonanywhere (instead of Django).
Below is the diagram that is showing setup that will be described below. Chatbot online diagram
Here is how ChatterBot can be deployed on pythonanywhere with Django:
Go to pythonanywhere and select plan signup
Select Create new web framework
Select Django
Select python version (3.6 was used in this example)
Select project name directory
It will create the following:
/home/user/project_name
manage.py
my_template.html
views.py
——————-project_name
init_.py
settings.py
urls.py
wsgi.py
create folder cbot under user folder
/home/user/cbot
inside of this folder create
__init__.py (this is just empty file)
chatbotpy.py
Inside chatbotpy wrap everything into function runbot like below. In this function we are initiating chatbot object, taking user input, if needed we train chatbot and then asking for chatbot response. The response provided by chatbot is the output of this function. The input of this function is the user input that we are getting through web.
def runbot(inp, train_bot=False):
from chatterbot import ChatBot
chatbot = ChatBot("mychatbot",
logic_adapters=[
{
"import_path": "chatterbot.logic.BestMatch",
"statement_comparison_function": "chatterbot.comparisons.levenshtein_distance",
"response_selection_method": "chatterbot.response_selection.get_first_response"
},
{
'import_path': 'chatterbot.logic.LowConfidenceAdapter',
'threshold': 0.65,
'default_response': 'I am sorry, but I do not understand.'
}
],
trainer='chatterbot.trainers.ListTrainer')
if (train_bot == True):
print "Training"
"""
Insert here training code from the python code for ChatterBot example
"""
response = chatbot.get_response(inp)
return (response)
Now update views.py like below code box. Here we are taking user input from web, feeding this input to runbot function and sending output of runbot function (which is chatbot reponse) to web template.
Now update some configuration.
Update or confirm manage.py to include the line with settings.
if __name__ == '__main__':
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'cbotdjango.settings')
Update or confirm urls.py to have path like below
import sys
path = "/home/user/cbotdjango"
if path not in sys.path:
sys.path.append(path)
Now you are ready do testing chatbot online and should see the screen similar in the setup diagram on right side.
Conclusion
We saw how to train ChatterBot and some functionality of it. We investigated how to install ChatterBot on pythonanywhere with Django web framework. Hope this will make easy to deploy chatbot in case you are going this route. If you have any tips or anything else to add, please leave a comment below.