Running R Package POMDP from Python

python R

Chatbots are now used in many applications for different purposes. The popularity of this type widget can be estimated from this fact:
As of August 2019, search results on Google for the following keywords:

  • chatbot – Volume: 246,000 searches per month and found 32,700,000 results
  • neural net – Volume 3600 searches per month and 127,000,000 results
  • One of the key components of chatbot architecture is Dialog Management. In order to incorporate some degree of uncertainty in classifying intents and entities, partially observable Markov decision process (POMDP) algorithm was proposed for building chatbots with Dialog Management. [1],[2]

    How can we run POMDP? There is a package in R for this.

    But what if we want run from python? One of the way could be call R from python. Here is how it can be done.

    First we need to create R program. For running POMDP in R we need first download and install (only one time). We also need set working directory (if it is different than default) and specify pdf file name – this is where the R
    script will output charts and diagrams. See below lines for this:

    r = getOption("repos")
    r["CRAN"] = ""
    options(repos = r)
    setwd ("C://Users//username//Documents")
    ## the below line need to be run only first time and it must be run with admin priv.

    Now we can continue with our R program. Documentation for POMDP [3] has R code for tiger problem, that we can insert here.

    Running POMDP R Package from Python

    Now we need to create python script that will call R script. In addition to calling the R program, python will also display output from running R program. This is possible because when we call R program, we save output within txt file. So the python script will save output into file and also output on screen.

    # -*- coding: utf-8 -*-
    import os
    os.system('"C:\\Program Files\\R\\R-3.4.3\\bin\\Rscript" C:\\Users\\username\\POMDP_R\\r_example.r > C:\\Users\\username\\output_file_ex.txt')
    with open('output_file_ex.txt', 'r') as reader:

    Now we can run R package POMDP or any other R program from python and start build advanced chatbot dialog system.

    1. Dialog management
    2. An Improved Approach of Intention Discovery with Machine Learning for POMDP-based Dialogue Management
    3. POMDP: Introduction to Partially Observable Markov Decision Processes

    Chatbots Examples with ChatterBot – How to Add Logic

    In the previous post How to Create a Chatbot with ChatBot Open Source and Deploy It on the Web I wrote how to deploy ChatterBot on pythonanywhere hosting site with Django webfamework. In this post we will look at few useful chatbots examples for implementing logic in our chatbot. This chatbot was developed in the previous post and is based on ChatterBot python library.

    Making Chatbot Start Conversation with Specific Question

    Suppose we want to start conversation with specific sentence that chatbot needs to show. For example, when the user open website, the chatbot can start with the specific question: “Did you find what you were looking for?”
    Or, if you are building chatbot for conversation about previous day/week work, you probably want to start with “How was you previous day/week in terms of progress to goal?” How can we do this with ChatterBot framework?

    Conversation diagram

    It turns out, that ChatterBot has several logic adapters that allow to build conversation per different requirements.

    Here is how I used logic adapter SpecificResponseAdapter for chatbot to start with initial predefined question:

     chatbot = ChatBot("mybot",
                'import_path': 'chatterbot.logic.SpecificResponseAdapter',
                'input_text': 'prev_day_disk',
                'output_text': 'How much did you do toward your goal on previous day?'

    In that was created in prev. post[1], I put input “prev_day_disk” to runbot instead of blank string. Because in the beginning of chat there is no user input and I used this to enter input_text and get desired output as specified in output_text.

    def press_my_buttons(request):
        if request.POST:
            conv=request.POST.get('conv', '')
            user_input=request.POST.get('user_input', '')
            userid=request.POST.get('userid', '')
            if (userid == ""):
            resp=runbot(user_input, request, userid)
            conv=conv + "" + str(user_input) + "\n" + "BOT:"+ str(resp) + "\n"
            resp=runbot("prev_day_disk", request, "")
            conv =  "BOT:"+ str(resp) + "\n";
        return render(request, 'my_template.html', {'conv': conv })

    SpecificResponseAdapter can be used also in other places of conversation (not just in the beginning). For example we could use criteria if there is no input from user during 15 secs and user is not typing anything (not sure yet how easily it is to check if user typing or not) then switch conversation to new topic by making chatbot app send new question.

    How to Add Intelligence to Chatbot App

    After the user replied to response how was his/her week, I want chatbot to be able to recognize the response as belonging to one of the 3 groups: bad, so-so, good. This is a machine learning classification problem. However here we will not create text classification algorithm, instead we will use built in functionality.

    We will use another logic adapter, called BestMatch.
    With this adapter we need specify statement_comparison_function and response_selection_method :

    chatbot = ChatBot("mybot",
                'import_path': 'chatterbot.logic.SpecificResponseAdapter',
                'input_text': 'How much did you do toward your goal on previous day?',
                'output_text': 'How much did you do toward your goal on previous day?'
                "import_path": "chatterbot.logic.BestMatch",
                "statement_comparison_function": "chatterbot.comparisons.levenshtein_distance",
                "response_selection_method": "chatterbot.response_selection.get_first_response"

    Best Match Adapter – is a logic adapter that returns a response based on known responses to the closest matches to the input statement. [2]

    The best match adapter uses a function to compare the input statement to known statements. Once it finds the closest match to the input statement, it uses another function to select one of the known responses to that statement.

    To use this adapter for the above example I need at minimum create 1 samples per each group. In the below example I used following for testing of chatbot on 2 groups (skipped so-so).

     if (train_bot == True):
        "I did not do much this week",
        "Did you run into the problems with programs or just did not have time?"
        "I did a lot of progress",
        "Fantastic! Keep going on"

    After the training, if the user enters something close to “I did not do much this week” the chatbot will respond with “Did you run into the problems with programs or just did not have time?”, and if user enters something like “Did a lot of progress” the bot response will be “Fantastic! Keep going on” even if the input is slightly different from training.

    So we looked how to build a chatbot with logic that makes chatbot able to ask questions as needed or classify user input in several buckets and respond to user input accordingly. The code for this post is provided at the link listed below[3]

    1. How to Create a Chatbot with ChatBot Open Source and Deploy It on the Web
    2. ChatterBot – Logic
    3.Python Chatterbot Example with Added Logic – source code

    How to Create a Chatbot with ChatBot Open Source and Deploy It on the Web

    Chatbots have become very popular due to progress in AI, ML and NLP. They are now used on many websites. With increased popularity of chatbots there are many different frameworks to create chatbot. We will explore one of such framework in this post. We will review how to create a chatbot and deploy online based on open source for ChatterBot. Our platform will be Django on pythonanywhere.

    This chatbot tutorial introduces the basic concepts and terms needed to understand and deploy chatbot ChatterBot and provides a simple usage example.

    What is ChatterBot?

    ChatterBot is a Python library that makes it easy to generate automated responses to a user’s input. To produce different types of responses ChatterBot uses a selection of machine learning algorithms such as search and /or classification algorithms. This makes it easy for developers to create chat bots and automate conversations with users. [1]

    To install this chatbot open source you need just run:
    pip install chatterbot

    ChatterBot Features

    The functionality of chatterBot is provided through adapters – a pluggable class that allows a ChatBot instance to execute some kind of functionality.

    ChatBot has the following groups of adapters
    input adapters
    output adapters
    storage adapters
    logic adapters

    Inside of each group there several adapters that support different functionalities. For example within logic adapters we can use Best Match Adapter, Time Logic Adapter and few other.

    Here is the example how do you run chatbot. Below is the output of running the code:

    Result of program run:
    USER: How are you doing?
    BOT:I am fine
    USER: Hello
    BOT:Hi there!
    USER: Good morning!
    BOT:How are today!
    USER: Do you like machine learning?
    BOT:Yes, I like machine learning
    USER: How do I make a neural network?
    BOTI am sorry, but I do not understand.
    USER: Let us talk about current activities
    BOT:What are working now?
    USER: I am just browsing Internet for news
    BOT:What a waste of time! Dont you have any other things to do?
    USER: I am working on python script to make new chatbot
    BOT:This is great. Keep working on this
    USER: Bye

    During the testing I confirmed that it correctly responds to new (but similar) inputs even if we did not train exactly.

    Once you trained the bot, the result of training is staying there even after turning off/on PC. So if you run program multiple times, you need run training just first time. You still can run training later, for example if you want retrain or update python chatbot code.


    This section describes how to deploy ChatterBot on pythonanywhere web hosting site with Django programming web framework.

    PythonAnywhere is an online integrated development environment (IDE) and web hosting service based on the Python programming language.[3] It has free account that allows to deploy our Chatbot. Django is needed to connect web front interface with our python code for ChatterBot in the backend. Other web frameworks like Flask can be used on pythonanywhere (instead of Django).

    Below is the diagram that is showing setup that will be described below.

    Chatbot online diagram
    Chatbot online diagram

    Here is how ChatterBot can be deployed on pythonanywhere with Django:

  • Go to pythonanywhere and select plan signup
  • Select Create new web framework
  • Select Django
  • Select python version (3.6 was used in this example)
  • Select project name directory
  • It will create the following:

  • create folder cbot under user folder
  • /home/user/cbot

  • inside of this folder create
  • (this is just empty file)

    Inside chatbotpy wrap everything into function runbot like below. In this function we are initiating chatbot object, taking user input, if needed we train chatbot and then asking for chatbot response. The response provided by chatbot is the output of this function. The input of this function is the user input that we are getting through web.

    def runbot(inp, train_bot=False):
     from chatterbot import ChatBot
     chatbot = ChatBot("mychatbot",
                "import_path": "chatterbot.logic.BestMatch",
                "statement_comparison_function": "chatterbot.comparisons.levenshtein_distance",
                "response_selection_method": "chatterbot.response_selection.get_first_response"
                'import_path': 'chatterbot.logic.LowConfidenceAdapter',
                'threshold': 0.65,
                'default_response': 'I am sorry, but I do not understand.'
     if (train_bot == True):
        print "Training"
      Insert here training code from the python code for ChatterBot example 
     response = chatbot.get_response(inp)
     return (response)

    Now update like below code box. Here we are taking user input from web, feeding this input to runbot function and sending output of runbot function (which is chatbot reponse) to web template.

    from django.shortcuts import render
    from cbot.chatbotpy import runbot
    def press_my_buttons(request):
        if request.POST:
            conv=request.POST.get('conv', '')
            user_input=request.POST.get('user_input', '')
            conv=conv + "" + str(user_input) + "\n" + "BOT:"+ str(resp) + "\n"
            conv =  "BOT:"+ str(resp) + "\n";
        return render(request, 'my_template.html', {'conv': conv})

    Now update my_template.html like below. Here we just show new response together with previous conversation information.

    <form method="post">
        {% csrf_token %}
        <textarea rows=20 cols=60>{{conv}}</textarea>
        <input type="textbox" name="user_input" value=""/>
        <button type="submit">Submit</button>
        <input type="hidden" name =conv  value="{{conv}}" />

    Now update some configuration.
    Update or confirm to include the line with settings.

    if __name__ == '__main__':
        os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'cbotdjango.settings')

    Update or confirm to have path like below

    import sys
    path = "/home/user/cbotdjango"
    if path not in sys.path:

    Now you are ready do testing chatbot online and should see the screen similar in the setup diagram on right side.


    We saw how to train ChatterBot and some functionality of it. We investigated how to install ChatterBot on pythonanywhere with Django web framework. Hope this will make easy to deploy chatbot in case you are going this route. If you have any tips or anything else to add, please leave a comment below.

    1. ChatterBot
    2. Python chatbot code example
    3. Python Anywhere