A Guide To NLP : A Confluence Of AI And Linguistics

We are living our lives more in bits & bytes than in exchanging emotions. We transact and communicate more over the internet using our super intelligent machines called computer. So we now feel that our machines need to understand how we comprehend while we speak and are trying to give language to them using the filed of Artificial Intelligence Called NLP : Natural Language Processing. Chatbots as an outcome are are becoming a reliable business tools to communicate with humans using this non-human dependent intelligent tool.

I strongly feel :

Until our machine learns to understand behaviour and emotions, jobs of data scientist and engineers are only half done. NLP in collaboration Deep Learning (Field of ML Science) are going to play a pivotal role in giving some sense to language being spoken by computer machines.

What Is NLP :

It’s a stream of Artificial intelligence where machine meets human language giving them words to communicate with humans. It involves intelligent analysis of written language using NLP techniques to get insights from set of textual data like

  1. Sentiment Analysis
  2. Information Extraction & Retrieval
  3. Smart Search etc…

It’s a confluence of artificial intelligence and computational linguisticswhich handles interactions between machines and natural languages of humans in which computers are entailed to analyze, understand, alter, or generate natural language. NLP helps computer machines to engage in communication using natural human language in all forms, including but not limited to speech, print, writing, and signing.

NLP Machine Learning & Deep Learning : How Are They Connected :

NLP is closely related to ML and Deep Learning all these are branched out as shown in image below from Artificial Intelligence : A field of computer science which engages in making machine intelligent. Deep Learning is one of the popular Machine Learning techniques like Regression, K-means etc..

Machine Learning type like unsupervised ML is often used in NLP techniques like LDA(Latent Dirichlet Allocation which is a Topic Modeling Algorithm)

In-order to perform any NLP we need to deeply understand emotional and analytical aspect of how human being process language. Also various data sources of language like social media where human share the content which directly or indirectly is an extension what they are feeling has to be intelligently analysed by machines using NLP. NLP machines need to build a human reasoning system and with the help of ML techniques they can automate NLP process and scale it.

In a nutshell ML, Deep Learning And NLP are interconnected and interdependent to build a intelligent computer machines which can think, speak and act like humans.

John Rehling, an NLP expert at Meltwater Group, said in How Natural Language Processing Helps Uncover Social Media Sentiment.

“By analyzing language for its meaning, NLP systems have long filled useful roles, such as correcting grammar, converting speech to text and automatically translating between languages.”

How NLP Works :

It become really relevant to understand how NLP works so that it becomes easy to grasp the NLP as a whole . Generally NLP has got two main components

  1. NLU : Natural Language Understandings
  2. NLG : Natural Language Generations

Let’s go deep into NLU :

Natural Language Understandings : it deals with the methodology which try to understands how to give relevant meaning to the natural language which is fed to the computer

At the start computer gets an input of natural language ( natural language is any language that has evolved naturally in humans through use and repetition without conscious planning or premeditation. Natural languages can take different forms, such as speech or signing.)

Computer then converts them into Artificial language like speech recognition and or speech to text conversion works. Here we get the data into a textual form which NLU process to understand the meaning of the same.

HMM : Hidden Markov Model (NLU Example) :

                                                             Image source

It is a statistical speech recognition model which converts your speech into a text with the help of prebuilt mathematical techniques and try to infer what you said verbally.

It tries to understand what you said by taking the voice data and breaking it down to small chunk of particular time duration mostly 10–20 ms. These data sets are further compared to pre-fed speech to decode what you said in each unit of your speech. The purpose here is to find phoneme(a smallest unit of speech). Then machine looks at the series of such phonemes and statistically determine the most likely words and sentences to spoke.

Then NLU gets to deeply understand each word where it tries to understand whether it is a Noun or Verb, what is the tense(Past or future) and so on.. This process is defined as POS : Part Of Speech Tagging. NLP has inbuilt lexicon and a set of protocols related to grammar pre-coded into their system which is employed while processing the set of natural language data sets and decode what was said when NLP system processed the human speech.

NLP systems also have a lexicon (a vocabulary) and a set of grammar rules coded into the system. Modern NLP algorithms use statistical machine learning to apply these rules to the natural language and determine the most likely meaning behind what was said. There are several challenges in accomplishing this when considering problems such as words having several meanings (polysemy) or different words having similar meanings (synonymy), but software developers builds their own rules in their NLU systems which can handle this type of problems with proper training and learning


A compared to stage one NLU which does most of hard work on their part to understand what humans have said , NLG has quite an easier job to translates that artificial language of a computers to generate a meaningful text which can further be converted to audible speech using tex-to-speech conversion. The text-to-speech engine analyzes the text using a prosody model, which determines breaks, duration, and pitch. Then, using a speech database, the engine puts together all the recorded phonemes to form one coherent string of speech.

In a nutshell NLP employs NLU & NLG components to process the human natural language specially in speech recognition field and try to understand , decode and infer what was said by delivering string or audible speech as an output.

NLP Applications : IN The Modern Context

Image Source

In this computer age of digital revolution where most of the tasks are being accomplished by humans using machines which are internet connected.NLP has played a vital role in building some robust software tools for industry segments like media, publishing, advertising, healthcare, banking and insurance, helping them to operate efficiently and productively.

Some of the modern usage of NLP :

  1. Chatbots : its a trained software called bots which is capable of handling any scenario of dialog with humans. Some of the popular NLP And ML platform for chatbots which you can used in your business (product or services ) are api.aiMicrosoft Language Understanding Intelligent Service (LUIS) etc…
                                              Image Source

2. Spam Filtering :

Image Source

Most of you must be quite familiar with spams. Being a Gmail user you can Relate spam filtering techniques. Google uses this AI based NLP techniques to keep your inbox clean and spam free. Spam filters have become important as the first line of defense against our ever-increasing problem of unwanted email. A technology that has received much attention is Bayesian spam filtering, a statistical technique in which the incidence of words in an email is measured against its typical occurrence in a corpus of spam and non-spam emails.

3. Machine Translations :

                                         Image Source

NLP is increasingly being used for machine translation programs, in which one human language is automatically translated into another human language. Google is a pioneer in this which translates your text in one language into the text of the desired language.

The challenge with machine translation technologies is not in translating words, but in preserving the meaning of sentences, a complex technological issue that is at the heart of NLP.

4. Named entity extraction:

It is used in separating one item from the set of given item which are of similar nature and attribute. Examples include first and last names, age, geographic locations, addresses, phone numbers, email addresses, company names, etc. Named entity extraction, sometimes also called named entity recognition, makes it easier to mine data.

5. Automatic summarization:

Natural language processing can be used to produce a readable summary from a large chunk of text. For example, one might us automatic summarization to produce a short summary of a dense academic article.

6. Speech Recognition : Text-To-Speech Conversion etc as spoke at the start of the articles. Is what NLP is being extensively used for.

some other utilities of NLP are Sentiment analysis which is used in Algorithmic Stock trading etc..

NLP Algorithms & API’s :

Next in our series of NLP articles, we will go ever more deeper into technicalities of NLP keeping software developer who wants to pursue their career into this AI & ML field of computer science. Till then you can also read some of my other articles about Machine Learning, Deep Learning & AI given below

  1. Deep Learning : What & Why ?
  2. Deep Belief Network Fundamentals
  3. AI Fundamentals : Making Machine Intelligent
  4. Machine Learning : What & Why
  5. Deep Learning : Auto-encoders Fundamentals and types

If you are loving my articles do clap and share it, also If you want to hear more from me and want to consult Click Here and subscribe and i would feel blessed to hear and respond

Thanks for being there with me..

Pramod Chandrayan

Founder & CEO (Mobibit Soft (P) Ltd) | Mobile App Development Consultant | Startup Mentor | Spiritual Seeker

More Posts - Website

Follow Me:
TwitterFacebookLinkedInGoogle Plus

Related posts

Leave a Comment