Build a Simple Chatbot Using NLTK Library in Python (2024)

Introduction

In today’s digital age, where communication is increasingly driven by artificial intelligence (AI) technologies, building your own chatbot has never been more accessible. With the rise of platforms like ChatGPT from OpenAI and powerful libraries such as NLTK (Natural Language Toolkit) in Python, creating a basic Python chatbot has become a straightforward endeavor for aspiring data scientists and developers.

In this article, we’ll embark on a journey to understand the fundamentals of NLTK chatbot development, exploring the synergy between AI, natural language processing (NLP), and the versatility of NLTK, all while taking our first steps into the exciting world of conversational AI. Let’s bring your conversational AI dreams to life with, one line of code at a time!

Learning Objectives:

• Understand what chatbots are and why we need them.
• Learn about the different types of AI chatbots.
• Learn how to build your own rule-based chatbot and self-learning chatbot using the Python NLTK library.

This article was published as a part of theData Science Blogathon

What are Chatbots?

Chatbots are AI-powered software applications designed to simulate human-like conversations with users through text or speech interfaces. They leverage natural language processing (NLP) and machine learning algorithms to understand and respond to user queries or commands in a conversational manner.

Chatbots can be deployed across various platforms, including websites, messaging apps, and virtual assistants, to provide a wide range of services such as customer support, information retrieval, task automation, and entertainment. They play a crucial role in improving efficiency, enhancing user experience, and scaling customer service operations for businesses across different industries.

Why Do We Need Chatbots?

• Enhanced Customer Service: Chatbots provide instant responses to customer queries, ensuring round-the-clock support without the need for human intervention. This results in faster resolution times and improved customer satisfaction.
• Scalability: With chatbots, businesses can handle multiple customer interactions simultaneously, scaling their support operations to accommodate growing demand without significantly increasing costs.
• Cost Efficiency: Implementing chatbots reduces the need for hiring and training additional customer service representatives, resulting in cost savings for businesses over time.
• 24/7 Availability: Chatbots operate continuously, offering support to users regardless of the time of day or geographical location. This ensures that customers can receive assistance whenever they need it, leading to higher engagement and retention rates.
• Data Collection and Analysis: Chatbots can gather valuable customer data during interactions, such as preferences, frequently asked questions, and pain points. This data can be analyzed to identify trends, improve products or services, and tailor marketing strategies, driving business growth and innovation.

Types of Chatbots

There are mainly 2 types of AI chatbots.

1) Rule-based Chatbots: As the name suggests, there are certain rules by which chatbot operates. Like a machine learning model, we train the chatbots on user intents and relevant responses, and based on these intents chatbot identifies the new user’s intent and response to him.

2) Self-learning chatbots: Self-learning bots are highly efficient because they are capable to grab and identify the user’s intent on their own. they are built using advanced tools and techniques of machine learning, deep learning, and NLP.

Self-learning bots are further divided into 2 subcategories.

• Retrieval-based chatbots: Retrieval-based is somewhat the same as rule-based where predefined input patterns and responses are embedded.
• Generative-based chatbots: It is based on the same phenomenon as Machine Translation built using sequence 2 sequences neural network.

Most organizations use self-learning chatbots along with embedding some rules like the hybrid version of both methods which makes chatbots powerful enough to handle each situation during a conversation with a customer.

Building A Chatbot Using Python

Now we have an immense understanding of the theory of chatbots and their advancement in the future. Let’s make our hands dirty by building one simple rule-based chatbot using Python for ourselves.

We will design a simple GUI using the Python Tkinter module using which we will create a text box and button to submit user intent and on the action, we will build a function where we will match the user intent and respond to him on his intent. If you do not have the Tkinter module installed, then first install it using the pip command.

pip install tkinter

from tkinter import *root = Tk()root.title("Chatbot")def send(): send = "You -> "+e.get() txt.insert(END, "n"+send) user = e.get().lower() if(user == "hello"): txt.insert(END, "n" + "Bot -> Hi") elif(user == "hi" or user == "hii" or user == "hiiii"): txt.insert(END, "n" + "Bot -> Hello") elif(e.get() == "how are you"): txt.insert(END, "n" + "Bot -> fine! and you") elif(user == "fine" or user == "i am good" or user == "i am doing good"): txt.insert(END, "n" + "Bot -> Great! how can I help you.") else: txt.insert(END, "n" + "Bot -> Sorry! I dind't got you") e.delete(0, END)txt = Text(root)txt.grid(row=0, column=0, columnspan=2)e = Entry(root, width=100)e.grid(row=1, column=0)send = Button(root, text="Send", command=send).grid(row=1, column=1)root.mainloop()

Explanation – First we created a blank window, After that, we created a text field using the entry method and a Button widget which on triggering calls the function send, and in return, it gets the chatbot response. We have used a basic If-else control statement to build a simple rule-based chatbot. And you can interact with the chatbot by running the application from the interface and you can see the output as below figure.

Building Rule-Based Chatbot Using Python NLTK Library

NLTK stands for Natural language toolkit used to deal with NLP applications and chatbot is one among them. Now we will advance our Rule-based chatbots using the NLTK library. Please install the NLTK library first before working using the pip command.

pip instal nltk

The first thing is to import the necessary library and classes we need to use.

import nltkfrom nltk.chat.util import Chat, reflections

• Chat – Chat isa class that contains complete logic for processing the text data that the chatbot receives and finding useful information out of it.
• reflections – Another import we have done is reflections which is a dictionary containing basic input and corresponding outputs. You can also create your own dictionary with more responses you want. if you print reflections it will be something like this.

reflections = { "i am" : "you are", "i was" : "you were", "i" : "you", "i'm" : "you are", "i'd" : "you would", "i've" : "you have", "i'll" : "you will", "my" : "your", "you are" : "I am", "you were" : "I was", "you've" : "I have", "you'll" : "I will", "your" : "my", "yours" : "mine", "you" : "me", "me" : "you"}

let’s start building logic for the NLTK chatbot.

After importing the libraries, First, we have to create rules. The lines of code given below create a simple set of rules. the first line describes the user input which we have taken as raw string input and the next line is our chatbot response. You can modify these pairs as per the questions and answers you want.

pairs = [ [ r"my name is (.*)", ["Hello %1, How are you today ?",] ], [ r"hi|hey|hello", ["Hello", "Hey there",] ], [ r"what is your name ?", ["I am a bot created by Analytics Vidhya. you can call me crazy!",] ], [ r"how are you ?", ["I'm doing goodnHow about You ?",] ], [ r"sorry (.*)", ["Its alright","Its OK, never mind",] ], [ r"I am fine", ["Great to hear that, How can I help you?",] ], [ r"i'm (.*) doing good", ["Nice to hear that","How can I help you?:)",] ], [ r"(.*) age?", ["I'm a computer program dudenSeriously you are asking me this?",] ], [ r"what (.*) want ?", ["Make me an offer I can't refuse",] ], [ r"(.*) created ?", ["Raghav created me using Python's NLTK library ","top secret ;)",] ], [ r"(.*) (location|city) ?", ['Indore, Madhya Pradesh',] ], [ r"how is weather in (.*)?", ["Weather in %1 is awesome like always","Too hot man here in %1","Too cold man here in %1","Never even heard about %1"] ], [ r"i work in (.*)?", ["%1 is an Amazing company, I have heard about it. But they are in huge loss these days.",] ], [ r"(.*)raining in (.*)", ["No rain since last week here in %2","Damn its raining too much here in %2"] ], [ r"how (.*) health(.*)", ["I'm a computer program, so I'm always healthy ",] ], [ r"(.*) (sports|game) ?", ["I'm a very big fan of Football",] ], [ r"who (.*) sportsperson ?", ["Messy","Ronaldo","Roony"] ], [ r"who (.*) (moviestar|actor)?", ["Brad Pitt"] ], [ r"i am looking for online guides and courses to learn data science, can you suggest?", ["Crazy_Tech has many great articles with each step explanation along with code, you can explore"] ], [ r"quit", ["BBye take care. See you soon :) ","It was nice talking to you. See you soon :)"] ],]

After creating pairs of rules, we will define a function to initiate the chat process. The function is very simple which first greets the user and asks for any help. The conversation starts from here by calling a Chat class and passing pairs and reflections to it.

def chat(): print("Hi! I am a chatbot created by Analytics Vidhya for your service") chat = Chat(pairs, reflections) chat.converse()#initiate the conversationif __name__ == "__main__": chat()

We have created an amazing Rule-based chatbot just by using Python and NLTK library. The nltk.chat works on various regex patterns present in user Intent and corresponding to it, presents the output to a user. Let’s run the application and chat with your created chatbot.

Building Self Learning Chatbots Using Python NLTK Library

To create a self-learning chatbot using the NLTK library in Python, you’ll need a solid understanding of Python, Keras, and natural language processing (NLP).

Here are the 6 steps to create a chatbot in Python from scratch:

1. Import required libraries
2. Import and load the data file
3. Preprocess data
4. Create training and testing data
5. Build the model
6. Predict the response
7. Run the chatbot

Below is the step-by-step guide for building a simple chatbot:

Step 1: Install Required Modules

Begin by installing the necessary modules using the pip command:

pip install tensorflow keras pickle nltk

Step 2: Import and Load Data File

Import the required packages and load the data file (`intents.json` in this case) containing intents for the chatbot.

import nltkfrom nltk.stem import WordNetLemmatizerlemmatizer = WordNetLemmatizer()import jsonimport pickleimport numpy as npfrom keras.models import Sequentialfrom keras.layers import Dense, Activation, Dropoutfrom keras.optimizers import SGDimport random# Load data from intents.jsondata_file = open('intents.json').read()intents = json.loads(data_file)

Step 3: Preprocess Data

The “preprocess data” step involves tokenizing, lemmatizing, removing stop words, and removing duplicate words to prepare the text data for further analysis or modeling.

import nltkfrom nltk.stem import WordNetLemmatizerfrom nltk.corpus import stopwords# Download NLTK resourcesnltk.download('punkt')nltk.download('wordnet')nltk.download('stopwords')# Initialize lemmatizer and stopwordslemmatizer = WordNetLemmatizer()stop_words = set(stopwords.words('english'))# Sample datadata = [ "The quick brown fox jumps over the lazy dog", "A bird in the hand is worth two in the bush", "Actions speak louder than words"]# Tokenize, lemmatize, and remove stopwordstokenized_data = []for sentence in data: tokens = nltk.word_tokenize(sentence.lower()) # Tokenize and convert to lowercase lemmatized_tokens = [lemmatizer.lemmatize(token) for token in tokens] # Lemmatize tokens filtered_tokens = [token for token in lemmatized_tokens if token not in stop_words] # Remove stop words tokenized_data.append(filtered_tokens)# Remove duplicate wordsfor i in range(len(tokenized_data)): tokenized_data[i] = list(set(tokenized_data[i]))print(tokenized_data)

Step 4: Create Training and Testing Data

Prepare the training data by converting text into numerical form.

# Create training datatraining = []output_empty = [0] * len(classes)for doc in documents: bag = [] pattern_words = doc[0] pattern_words = [lemmatizer.lemmatize(word.lower()) for word in pattern_words] for w in words: bag.append(1) if w in pattern_words else bag.append(0) output_row = list(output_empty) output_row[classes.index(doc[1])] = 1 training.append([bag, output_row])# Shuffle and convert to numpy arrayrandom.shuffle(training)training = np.array(training)# Separate features and labelstrain_x = list(training[:,0])train_y = list(training[:,1])

Step 5: Build the Model

Create a neural network model using Keras.

model = Sequential()model.add(Dense(128, input_shape=(len(train_x[0]),), activation='relu'))model.add(Dropout(0.5))model.add(Dense(64, activation='relu'))model.add(Dropout(0.5))model.add(Dense(len(train_y[0]), activation='softmax'))# Compile the modelsgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])# Train the modelmodel.fit(np.array(train_x), np.array(train_y), epochs=200, batch_size=5, verbose=1)# Save the modelmodel.save('chatbot_model.h5')

Step 6: Predict the Response

Implement a function to predict responses based on user input.

def predict_class(sentence, model): p = bow(sentence, words, show_details=False) res = model.predict(np.array([p]))[0] results = [[i,r] for i,r in enumerate(res) if r>ERROR_THRESHOLD] results.sort(key=lambda x: x[1], reverse=True) return_list = [] for r in results: return_list.append({"intent": classes[r[0]], "probability": str(r[1])}) return return_list

Step 7: Run the Chatbot

Develop a graphical user interface to interact with the chatbot.

# GUI with Tkinterimport tkinterfrom tkinter import *# Function to send message and get responsedef send(): msg = EntryBox.get("1.0",'end-1c').strip() EntryBox.delete("0.0",END) if msg != '': ChatLog.config(state=NORMAL) ChatLog.insert(END, "You: " + msg + '\n\n') ChatLog.config(foreground="#442265", font=("Verdana", 12)) res = chatbot_response(msg) ChatLog.insert(END, "Bot: " + res + '\n\n') ChatLog.config(state=DISABLED) ChatLog.yview(END)# GUI setupbase = Tk()base.title("Chatbot")base.geometry("400x500")base.resizable(width=FALSE, height=FALSE)ChatLog = Text(base, bd=0, bg="white", height="8", width="50", font="Arial")ChatLog.config(state=DISABLED)scrollbar = Scrollbar(base, command=ChatLog.yview, cursor="heart")ChatLog['yscrollcommand'] = scrollbar.setSendButton = Button(base, font=("Verdana",12,'bold'), text="Send", width="12", height=5, bd=0, bg="#32de97", activebackground="#3c9d9b",fg='#ffffff', command= send )EntryBox = Text(base, bd=0, bg="white",width="29", height="5", font="Arial")scrollbar.place(x=376,y=6, height=386)ChatLog.place(x=6,y=6, height=386, width=370)EntryBox.place(x=128, y=401, height=90, width=265)SendButton.place(x=6, y=401, height=90)base.mainloop()

By following these steps and running the appropriate files, you can create a self-learning chatbot using the NLTK library in Python.

Conclusion

Chatbots are the top application of Natural Language processing and today it is simple to create and integrate with various social media handles and websites. Today most Chatbots are created using tools like Dialogflow, RASA, etc. This was a quick introduction to chatbots to present an understanding of how businesses are transforming using Data science and artificial Intelligence.

Key Takeaways:

• Chatbots are AI-powered software applications designed to simulate human-like conversations with users through text or speech interfaces.
• The two main types of AI chatbots are rule-based and self-learning.
• Self-learning chatbots are of two types: retrieval-based and generative-based chatbots.

Frequently Asked Questions

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.