WHAT ARTIFICIAL INTELLIGENCE CANNOT DO , a grim note to the top 100 intellectuals of this planet , Part 11 - Capt Ajit Vadakayil

THIS POST IS CONTINUED FROM PART 10, BELOW—

https://ajitvadakayil.blogspot.com/2019/12/what-artificial-intelligence-cannot-do.html

CAPT AJIT VADAKAYIL SAYS  AI  MUST MEAN  “INTELLIGENCE AUGUMENTATION “  IN FUTURE ..

Let this be IA

OBJECTIVE AI CANNOT HAVE A VISION,  

IT CANNOT PRIORITIZE,   

IT CANT GLEAN CONTEXT,   

IT CANT TELL THE MORAL OF A STORY ,  

IT CANT RECOGNIZE A JOKE, OR BE A JUDGE IN A JOKE CONTEST  

IT CANT DRIVE CHANGE,    

IT CANNOT INNOVATE, 

IT CANNOT DO ROOT CAUSE ANALYSIS ,  

IT CANNOT MULTI-TASK,   

IT CANNOT DETECT SARCASM,  

IT CANNOT DO DYNAMIC RISK ASSESSMENT ,  

IT IS UNABLE TO REFINE OWN KNOWLEDGE TO WISDOM, 

IT IS BLIND TO SUBJECTIVITY,  

IT CANNOT EVALUATE POTENTIAL,   

IT CANNOT SELF IMPROVE WITH EXPERIENCE,

IT CANNOT UNLEARN

IT IS PRONE TO CATASTROPHIC FORGETTING    

IT DOES NOT UNDERSTAND BASICS OF CAUSE AND EFFECT,   

IT CANNOT JUDGE SUBJECTIVELY TO VETO/ ABORT,    

IT CANNOT FOSTER TEAMWORK DUE TO RESTRICTED SCOPE,  

IT CANNOT MENTOR,   

IT CANNOT BE CREATIVE,  

IT CANNOT THINK FOR ITSELF, 

IT CANNOT TEACH OR ANSWER STUDENTs QUESTIONS,  

IT CANNOT PATENT AN INVENTION, 

IT CANNOT SEE THE BIG PICTURE , 

IT CANNOT FIGURE OUT WHAT IS MORALLY WRONG, 

IT CANNOT PROVIDE NATURAL JUSTICE, 

IT CANNOT FORMULATE LAWS

IT CANNOT FIGURE OUT WHAT GOES AGAINST HUMAN DIGNITY

IT CAN BE FOOLED EASILY USING DECOYS WHICH CANT FOOL A CHILD, 

IT CANNOT BE A SELF STARTER, 

IT CANNOT UNDERSTAND APT TIMING, 

IT CANNOT FEEL

IT CANNOT GET INSPIRED

IT CANNOT USE PAIN AS FEEDBACK,

IT CANNOT GET EXCITED BY ANYTHING

IT HAS NO SPONTANEITY TO MAKE THE BEST OUT OF  SITUATION 

IT CAN BE CONFOUNDED BY NEW SITUATIONS

IT CANNOT FIGURE OUT GREY AREAS,

IT CANNOT GLEAN WORTH OR VALUE

IT CANNOT UNDERSTAND TEAMWORK DYNAMICS 

IT HAS NO INTENTION

IT HAS NO INTUITION,

IT HAS NO FREE WILL

IT HAS NO DESIRE

IT CANNOT SET A GOAL

IT CANNOT BE SUBJECTED TO THE LAWS OF KARMA

ON THE CONTRARY IT CAN SPAWN FOUL AND RUTHLESS GLOBAL FRAUD ( CLIMATE CHANGE DUE TO CO2 ) WITH DELIBERATE BLACK BOX ALGORITHMS,  JUST FEW AMONG MORE THAN 60 CRITICAL INHERENT DEFICIENCIES.

HUMANS HAVE THINGS A COMPUTER CAN NEVER HAVE.. A SUBCONSCIOUS BRAIN LOBE,  REM SLEEP WHICH BACKS UP BETWEEN RIGHT/ LEFT BRAIN LOBES AND FROM AAKASHA BANK,  A GUT WHICH INTUITS,   30 TRILLION BODY CELLS WHICH HOLD MEMORY,   A VAGUS NERVE , AN AMYGDALA ,  73% WATER IN BRAIN FOR MEMORY,  10 BILLION MILES ORGANIC DNA MOBIUS WIRING ETC.

SINGULARITY ,  MY ASS !

Simultaneous Localization and Mapping (SLAM) technology aids in the localization and positioning of a robot or a device in real-time using mathematical and statistical algorithms with different sensors

In navigation, robotic mapping and odometry for virtual reality or augmented reality, simultaneous localization and mapping (SLAM) is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it. 

While this initially appears to be a chicken-and-egg problem there are several algorithms known for solving it, at least approximately, in tractable time for certain environments. Popular approximate solution methods include the particle filter, extended Kalman filter, Covariance intersection, and GraphSLAM.

SLAM algorithms are tailored to the available resources, hence not aimed at perfection, but at operational compliance. Published approaches are employed in self-driving cars, unmanned aerial vehicles, autonomous underwater vehicles, planetary rovers, newer domestic robots and even inside the human body

Self-navigation and communication has significantly improved thanks to technologies like Simultaneously Localisation and Mapping (or SLAM), we are able to visually and sensorily improve augmented reality (AR) experiences. Still, the risks of having human beings in the midst of a robot swarm is fraught with a variety of risks. Not just that different robots need to sense and respond to the location and movement of other robots, they need to respond to “unpredictable” movements and responses of humans.

Just like humans, bots can’t always rely on GPS, especially when they operate indoors. And GPS isn’t sufficiently accurate enough outdoors because precision within a few inches is required to move about safely.

Instead they rely on what’s known as simultaneous localization and mapping, or SLAM, to discover and map their surroundings.

Using SLAM, robots build their own maps as they go. It lets them know their position by aligning the sensor data they collect with whatever sensor data they’ve already collected to build out a map for navigation.

Sounds easy enough, but it’s actually a multi-stage process that includes alignment of sensor data using a variety of algorithms well suited to the parallel processing capabilities of GPUs.

Computers see a robot’s position as simply a timestamp dot on a map or timeline. 

Robots continuously do split-second gathering of sensor data on their surroundings. Camera images are taken as many as 90 times a second for depth-image measurements. And LiDAR images, used for precise range measurements, are taken 20 times a second.

When a robot moves, these data points help measure how far it’s gone relative to its previous location and where it is located on a map.

Motion Estimation

In addition, what’s known as wheel odometry takes into account the rotation of a robot’s wheels to help measure how far it’s travelled. Inertial measurement units are also used to gauge speed and acceleration as a way to track a robot’s position.

All of these sensor streams are taken into consideration in what’s known as sensor fusion to get a better estimate of how a robot is moving.

The mapping calculations described above happen 20-100 times a second, depending on the algorithms. This wouldn’t be possible to perform in real time without the processing power of NVIDIA GPUs. Ideal for robotics

Future development for Isaac on visual odometry will integrate it and elevate it to the level of SLAM. For now, SLAM is used as a check for map recovery of a robot’s location and orientation to eliminate errors in navigation from inaccurate visual odometry results.

Researchers at the Massachusetts Institute of Technology (MIT) presented a project at the International Symposium on Experimental Robotics involving an autonomous drone fleet system that collaboratively mapped an environment under dense forest canopy.

Designed with search and rescue in mind, the drones used lidar, onboard computation and wireless communication, with no requirement for GPS positioning.

Each drone carries laser-range finders for position estimation, localization and path planning. As it flies, each drone creates its own 3-D map of the terrain. A ground station uses simultaneous localization and mapping (SLAM) technology to combine individual maps from multiple drones into a global 3-D map that can be monitored by operators.

The drones were programmed to identify multiple trees’ orientations, as recognizing individual trees in impossible for the technology, and individual trees’ orientation very difficult. When the lidar signal returns a cluster of trees, an algorithm calculates the angles and distances between trees to identify the cluster and determine if it has already been identified and mapped, or is a new mini-environment.

The technique also aids in merging maps from the separate drones. When two drones scan the same cluster of trees, the ground station merges the maps by calculating the relative transformation between the drones, and then fusing the individual maps to maintain consistent orientations.

Dragonfly is a visual 3D positioning/location system based on Visual SLAM:--

A valid alternative to LiDAR and Ultra Wide Band for accurate indoor positioning and location of drones, robots and vehicles.

Based on a patented proprietary technology.

Computer vision and odometry to create an accurate SLAM system.

Just an on-board camera (mono or stereo) required.

The SLAM location engine can run on board of the device or on a remote server (local or cloud).

Below: DRAGON FLY

Logistics Automation: Tracking and locating autonomous mobile robots (AMR) and automated guided vehicles (AGV) to allow indoor navigation. Equipment tracking indoors is also enabled.

Warehouse management: Tracking the location of mobile robots and lift trucks enables automating inventory management, as items are placed on racks.

Forklift Tracking: Forklift location tracking facilitates accident prevention and enables fleet management.

Autonomous Robots: Autonomous self-driven robots, often used in retail and healthcare, can navigate indoors and outdoors using Dragonfly’s centimeter precision location technology and allowing remote monitoring of their position in real time. Dragonfly is a SLAM for ROS technology, as we provide ROS (Robot Operating System) nodes for integration.

Various industries: Drones 3D indoor location and navigation enables inspections and tasks requiring visual identification. Drone positioning and tracking indoors is also possible.

Dragonfly’s accurate indoor location system is based on venue and place recognition:--

Dragonfly navigates and creates a 3D map in real time. Note: the 3D map is not meant to be human intelligible.

The map can be shared among different devices.

Dragonfly comes with an intuitive camera calibration tool for the configuration of the on-board camera.

5 cm accuracy.

WE ARE A NATION WHO ALLOWED A NAXAL RED CORRIDOR TO PARALYSE 20% OF OUR LANDMASS.

WE ARE A NATION WHO ALLOWED KASHMIRI PANDITS TO BE ETHNICALLY CLEANSED AND HUNDREDS OF HINDU TEMPLES TO BE DESTROYED.

AND WE DON’T WANT TO PUNISH THE FOREIGN PAYROLL CULPRITS—INCLUDING ILLEGAL COLLEGIUM JUDGES ?

ARE WE DOOMED TO GET TRUCKLOADS OF UNSOLICITED INFORMATION ABOUT ROTHSCHILDs AGENT KATHIAWARI JEW GANDHI  FROM OUR PM DAILY DAILY ?

NLU and natural language processing (NLP) are often confused. Instead they are different parts of the same process of natural language elaboration. Indeed, NLU is a component of NLP. More precisely, it is a subset of the understanding and comprehension part of natural language processing.

 While speech recognition captures spoken language in real-time, transcribes it, and returns text, NLU goes beyond recognition to determine a user’s intent. Speech recognition is powered by statistical machine learning methods which add numeric structure to large datasets. In NLU, machine learning models improve over time as they learn to recognize syntax, context, language patterns, unique definitions, sentiment, and intent.

NLP — Natural Language “Processing”

NLU — Natural Language “Understanding”

NLG — Natural Language “Generation”

Mathematically the combination of NLU and NLG will result in an NLP engine that works.

How the three of them work in hand in hand:--

NLU takes up the understanding of the data based on grammar, the context in which it was said and decide on intent and entities.

NLP will convert the text into structured data.

NLG generates text generated based on structured data.

What Is Natural Language Processing (NLP)?

Speech recognition is an integral component of NLP, which incorporates AI and machine learning. Here, NLP algorithms are used to understand natural speech in order to carry out commands.

Natural language processing starts with a library, a pre-programmed set of algorithms that plug into a system using an API, or application programming interface. Basically, the library gives a computer or system a set of rules and definitions for natural language as a foundation.

Your development team can customize that base to meet the needs of your product.

Neural networks figure prominently in NLP systems and are used in text classification, question answering, sentiment analysis, and other areas. Processing big data involved with understanding the spoken language is comparatively easier and the nets can be trained to deal with uncertainty, without explicit programming.

NLP, NLU, and NLG all play a part in teaching machines to think and act more like humans. 

So, if you’re Google, you’re using natural language processing to break down human language and better understand the true meaning behind a search query or sentence in an email. You’re also using it to analyze blog posts to match content to known search queries.

NLP is also used whenever you ask Alexa, Siri, Google, or Cortana a question, and anytime you use a chatbot. The program is analyzing your language against thousands of other similar queries to give you the best search results or answer to your question.

The beautiful thing about AI and machine learning is that, with regular use, it learns your language patterns to improve and tailor its results. 

I. NLP, or Natural Language Processing is a blanket term used to describe a machine’s ability to ingest what is said to it, break it down, comprehend its meaning, determine appropriate action, and respond back in a language the user will understand.

II. NLU, or Natural Language Understanding is a subset of NLP that deals with the much narrower, but equally important facet of how to best handle unstructured inputs and convert them into a structured form that a machine can understand and act upon. While humans are able to effortlessly handle mispronunciations, swapped words, contractions, colloquialisms, and other quirks, machines are less adept at handling unpredictable inputs.

III. NLG, or Natural Language Generation, simply put, is what happens when computers write language. NLG processes turn structured data into text.

What Is Natural Language Understanding (NLU)?

Natural language understanding is a smaller part of natural language processing. Once the language has been broken down, it’s time for the program to understand, find meaning, and even perform sentiment analysis.

The program breaks language down into digestible bits that are easier to understand. It does that by analyzing the text semantically and syntactically.

Semantically, it looks for the true meaning behind the words by comparing them to similar examples. At the same time, it breaks down text into parts of speech, sentence structure, and morphemes (the smallest understandable part of a word).

Unlike structured data, human language is messy and ambiguous. As a species, we are rarely straightforward with our communication. Grammar and the literal meaning of words pretty much go out the window whenever we speak.

In fact, “out the window” is a great example. I, of course, didn’t mean that I throw things out a literal window, especially since I was talking about intangible concepts rather than solid objects.

And so it is when you ask your smart device something like “What’s I-93 like right now?”.

If you were being literal, you might get an answer like, “It’s long, gray, and has cars driving on it. It was recently paved between exits 36 and 42.” But you probably wanted to know what the traffic conditions are.

That’s where natural language understanding comes in. It’s taking the slangy, figurative way we talk every day and understanding what we truly mean.

Once a chatbot, smart device, or search function understands the language it’s “hearing,” it has to talk back to you in a way that you, in turn, will understand.

That’s where NLG comes in. It takes data from a search result, for example, and turns it into understandable language. So whenever you ask your smart device, “What’s it like on I-93 right now?” it can answer almost exactly as another human would.

It may say something like, “There is an accident at exit 36 that has created a 15-minute delay,” or “The road is clear.”

NLG is used in chatbot technology, as well. In fact, chatbots have become so advanced; you may not even know you’re talking to a machine.

At its most basic, an algorithm simply tells a computer what to do next with an “and,” “or,” or “not” statement. ... Algorithms can be used to break down and automate sorting tasks.

When chained together, algorithms – like lines of code – become more robust. They're combined to build AI systems like neural networks .  AI often revolves around the use of algorithms. An algorithm is a set of unambiguous instructions that a mechanical computer can execute.

A complex algorithm is often built on top of other, simpler, algorithms. “AI at maturity is like a gear system with three interlocking wheels: data processing, machine learning and business action. ... “The key difference, is that an algorithm defines the process through which a decision is made, and AI uses training data to make such a decision.

An algorithm is a set of instructions — a preset, rigid, coded recipe that gets executed when it encounters a trigger. AI on the other hand — which is an extremely broad term covering a myriad of AI specializations and subsets — is a group of algorithms that can modify its algorithms and create new algorithms in response to learned inputs and data as opposed to relying solely on the inputs it was designed to recognize as triggers.

This ability to change, adapt and grow based on new data, is described as “intelligence.” AI at maturity is like a gear system with three interlocking wheels: data processing, machine learning and business action. It operates in an automated mode without any human intervention.

Data is created, transformed and moved without data engineers. Business actions or decisions are implemented without any operators or agents. The system learns continuously from the accumulating data and business actions and outcomes get better and better with time  .

An algorithm defines the process through which a decision is made, and AI uses training data to make such a decision..  “AI can make life easy by automating actions and making processes more efficient, even learn things from our day to day that we don't necessarily notice. .

Pattern matching in computer science is the checking and locating of specific sequences of data of some pattern among raw data or a sequence of tokens. Unlike pattern recognition, the match has to be exact in the case of pattern matching.. 

Pattern matching in computer science is the checking and locating of specific sequences of data of some pattern among raw data or a sequence of tokens. Unlike pattern recognition, the match has to be exact in the case of pattern matching.

.

Unlabeled data is a designation for pieces of data that have not been tagged with labels identifying characteristics, properties or classifications. Unlabeled data is typically used in various forms of machine learning.

An example of pattern recognition is classification, which attempts to assign each input value to one of a given set of classes (for example, determine whether a given email is "spam" or "non-spam"). However, pattern recognition is a more general problem that encompasses other types of output as well.

Pattern recognition is the ability to detect arrangements of characteristics or data that yield information about a given system or data set. ... In the context of AI, pattern recognition is a sub-category of machine learning (ML).

Pattern recognition and classification is the act of taking in raw data and using a set of properties and features take an action on the data. ... Moving on, we seek to design models and systems that will be able to recognize and furthermore classify these patterns into different categories for further use.

Why do we need to look for patterns? Finding patterns is extremely important. ... Problems are easier to solve when they share patterns, because we can use the same problem-solving solution wherever the pattern exists. The more patterns we can find, the easier and quicker our overall task of problem solving will be..

Labeled data is a group of samples that have been tagged with one or more labels. Labeling typically takes a set of unlabeled data and augments each piece of that unlabeled data with meaningful tags that are informative.

Data classification is the process of sorting and categorizing data into various types, forms or any other distinct class. Data classification enables the separation and classification of data according to data set requirements for various business or personal objectives. It is mainly a data management process.

Clustering is the process of partitioning or grouping a given set of patterns into disjoint clusters. This is done such that patterns in the same cluster are alike and patterns belonging to two dierent clusters are dierent.

Clustering is an important concept when it comes to unsupervised learning. It mainly deals with finding a structure or pattern in a collection of uncategorized data. Clustering algorithms will process your data and find natural clusters(groups) if they exist in the data. You can also modify how many clusters your algorithms should identify. It allows you to adjust the granularity of these groups.

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval, bioinformatics, data compression, and computer graphics.

Cluster analysis itself is not one specific algorithm, but the general task to be solved. It can be achieved by various algorithms that differ significantly in their understanding of what constitutes a cluster and how to efficiently find them.

Popular notions of clusters include groups with small distances between cluster members, dense areas of the data space, intervals or particular statistical distributions. Clustering can therefore be formulated as a multi-objective optimization problem. The appropriate clustering algorithm and parameter settings (including parameters such as the distance function to use, a density threshold or the number of expected clusters) depend on the individual data set and intended use of the results.

Cluster analysis as such is not an automatic task, but an iterative process of knowledge discovery or interactive multi-objective optimization that involves trial and failure. It is often necessary to modify data preprocessing and model parameters until the result achieves the desired properties.

There are different types of clustering you can utilize:-

Exclusive (partitioning)

In this clustering method, Data are grouped in such a way that one data can belong to one cluster only.

Example: K-means

Agglomerative

In this clustering technique, every data is a cluster. The iterative unions between the two nearest clusters reduce the number of clusters.

Example: Hierarchical clustering

Overlapping

In this technique, fuzzy sets is used to cluster data. Each point may belong to two or more clusters with separate degrees of membership.

Here, data will be associated with an appropriate membership value. Example: Fuzzy C-Means

Probabilistic

This technique uses probability distribution to create the clusters

Example: Following keywords

"man's shoe."

"women's shoe."

"women's glove."

"man's glove."

can be clustered into two categories "shoe" and "glove" or "man" and "women."

Clustering Types--

Hierarchical clustering

K-means clustering

K-NN (k nearest neighbors)

Principal Component Analysis

Singular Value Decomposition

Independent Component Analysis

Hierarchical Clustering:

Hierarchical clustering is an algorithm which builds a hierarchy of clusters. It begins with all the data which is assigned to a cluster of their own. Here, two close cluster are going to be in the same cluster. This algorithm ends when there is only one cluster left.

K-means Clustering

K means it is an iterative clustering algorithm which helps you to find the highest value for every iteration. Initially, the desired number of clusters are selected. In this clustering method, you need to cluster the data points into k groups. A larger k means smaller groups with more granularity in the same way. A lower k means larger groups with less granularity.

The output of the algorithm is a group of "labels." It assigns data point to one of the k groups. In k-means clustering, each group is defined by creating a centroid for each group. The centroids are like the heart of the cluster, which captures the points closest to them and adds them to the cluster.

K-mean clustering further defines two subgroups:--

Agglomerative clustering

Dendrogram

Agglomerative clustering:--

This type of K-means clustering starts with a fixed number of clusters. It allocates all data into the exact number of clusters. This clustering method does not require the number of clusters K as an input. Agglomeration process starts by forming each data as a single cluster.

This method uses some distance measure, reduces the number of clusters (one in each iteration) by merging process. Lastly, we have one big cluster that contains all the objects.

Dendrogram:

In the Dendrogram clustering method, each level will represent a possible cluster. The height of dendrogram shows the level of similarity between two join clusters. The closer to the bottom of the process they are more similar cluster which is finding of the group from dendrogram which is not natural and mostly subjective.

K- Nearest neighbors

K- nearest neighbour is the simplest of all machine learning classifiers. It differs from other machine learning techniques, in that it doesn't produce a model. It is a simple algorithm which stores all available cases and classifies new instances based on a similarity measure.

It works very well when there is a distance between examples. The learning speed is slow when the training set is large, and the distance calculation is nontrivial.

Principal Components Analysis:--

In case you want a higher-dimensional space. You need to select a basis for that space and only the 200 most important scores of that basis. This base is known as a principal component. The subset you select constitute is a new space which is small in size compared to original space. It maintains as much of the complexity of data as possible.

Association

Association rules allow you to establish associations amongst data objects inside large databases. This unsupervised technique is about discovering interesting relationships between variables in large databases. For example, people that buy a new home most likely to buy new furniture.

Other Examples:--

A subgroup of cancer patients grouped by their gene expression measurements

Groups of shopper based on their browsing and purchasing histories

Movie group by the rating given by movies viewers

.

Clustering and Association are two types of Unsupervised learning.

Four types of clustering methods are 1) Exclusive 2) Agglomerative 3) Overlapping 4) Probabilistic.

Important clustering types are: 1)Hierarchical clustering 2) K-means clustering 3) K-NN 4) Principal Component Analysis 5) Singular Value Decomposition 6) Independent Component Analysis.

Association rules allow you to establish associations amongst data objects inside large databases.

Clustering – describes an unsupervised machine learning technique for identifying structures among unstructured data. Clustering algorithms group sets of similar objects into clusters, and are widely used in areas including image analysis, information retrieval, and bioinformatics.

Clustering Analysis - is a type of unsupervised machine learning used for exploratory data analysis to find hidden patterns or groupings in datasets. Using metrics like probabilistic or Euclidian distance, clusters group together similar data points.

Clustering, like regression, describes the class of problem and the class of methods.

Clustering methods are typically organized by the modeling approaches such as centroid-based and hierarchal. All methods are concerned with using the inherent structures in the data to best organize the data into groups of maximum commonality.

The most popular clustering algorithms are:--

k-Means

k-Medians

Expectation Maximisation (EM)

Hierarchical Clustering

Clustering Algorithms

The basic idea behind clustering is to assign the input into two or more clusters based on feature similarity. It falls into the category of Unsupervised Machine Learning, where the algorithm learns the patterns and useful insights from data without any guidance (labeled data set).

For example, clustering viewers into similar groups based on their interests, age, geography, etc can be done by using Unsupervised Learning algorithms like K-Means Clustering.Clustering is very similar to classification, but involves grouping chunks of data together based on their similarities. For example, you might choose to cluster different demographics of your audience into different packets based on how much disposable income they have, or how often they tend to shop at your store.

K-Means Clustering

K-means is probably the simplest unsupervised learning approach. The idea here is to gather similar data points together and bind them together in the form of a cluster. It does this by calculating the centroid of the group of data points.

To carry out effective clustering, k-means evaluates the distance between each point from the centroid of the cluster. Depending on the distance between the data point and the centroid, the data is assigned to the closest cluster. The goal of clustering is to determine the intrinsic grouping in a set of unlabelled data. 

The ‘K’ in K-means stands for the number of clusters formed. The number of clusters (basically the number of classes in which your new instances of data can fall into) is determined by the user.

K-means is used majorly in cases where the data set has points which are distinct and well separated from each other, otherwise, the clusters won’t be far apart, rendering them inaccurate. Also, K-means should be avoided in cases where the data set contains a high amount of outliers or the data set is non-linear. 

Clustering is grouping a set of objects in such a manner that objects in the same group are more similar than to those object belonging to other groups. Whereas, association rules is about finding associations amongst items within large commercial databases.

Text analytics and text mining approaches have essentially equivalent performance. Text analytics requires an expert linguist to produce complex rule sets, whereas text mining requires the analyst to hand-label cases with outcomes or classes to create training data

Text Analytics is the process of drawing meaning out of written communication. In a customer experience context, text analytics means examining text that was written by, or about, customers. You find patterns and topics of interest, and then take practical action based on what you learn.

Text analytics is the automated process of translating large volumes of unstructured text into quantitative data to uncover insights, trends, and patterns. Combined with data visualization tools, this technique enables companies to understand the story behind the numbers and make better decisions.

Colossal amounts of unstructured data are generated every minute – internet users post 456,000 new tweets, 510,000 new comments on Facebook, and send 156 million emails – so managing and analyzing information to find what’s relevant becomes a major challenge.

Thanks to text analytics, businesses are able to automatically extract meaning from all sorts of unstructured data, from social media posts and emails to live chats and surveys, and turn it into quantitative insights.

 By identifying trends and patterns with text analytics, businesses can improve customer satisfaction (by learning what their customers like and dislike about their products), detect product issues, conduct market research, and monitor brand reputation, among other things.

Text analytics has many advantages – it’s scalable, meaning you can analyze large volumes of data in a very short time, and allows you to obtain results in real-time. So, apart from gaining insights that help you make confident decisions, you can also resolve issues in a timely manner.

One of the most interesting applications of text analytics in business is customer feedback analysis. This includes analyzing product and service reviews to see how your customers evaluate your company, processing the results of open-ended responses to customer surveys, or checking out what customers say about your brand on social media.

Text Analysis is about parsing texts in order to extract machine-readable facts from them. The purpose of Text Analysis is to create structured data out of free text content. The process can be thought of as slicing and dicing heaps of unstructured, heterogeneous documents into easy-to-manage and interpret data pieces.

Text mining, text analysis, and text analytics are often used interchangeably, with the end goal of analyzing unstructured text to obtain insights. However, while text mining (or text analysis) provides insights of a qualitative nature, text analytics aggregates these results and turns them into something that can be quantified and visualized through charts and reports.

Text analysis and text analytics often work together to provide a complete understanding of all kinds of text, like emails, social media posts, surveys, customer support tickets, and more. For example, you can use text analysis tools to find out how people feel toward a brand on social media (sentiment analysis), or understand the main topics in product reviews (topic detection). 

Text analytics, on the other hand, leverages the results of text analysis to identify patterns, such as a spike in negative feedback, and provides you with actionable insights you can use to make improvements, like fixing a bug that’s frustrating your users.

Text mining, also referred to as text data mining, roughly equivalent to text analytics, is the process of deriving high-quality information from text. ... The overarching goal is, essentially, to turn text into data for analysis, via application of natural language processing (NLP) and analytical methods

NLP and text mining are usually used for different goals. ... Text mining techniques are usually shallow and do not consider the text structure.Usually, text mining will use bag of words, n-grams and possibly stemming over that. In NLP methods usually involve the test structure.

The goal of text mining is to discover relevant information in text by transforming the text into data that can be used for further analysis. Text mining accomplishes this through the use of a variety of analysis methodologies; natural language processing (NLP) is one of them

Text mining enables to quickly extract customers' needs, preferences and requests. It could help managers to make decisions and figure out a lot of measures to respond to customers' discontent. It facilitates gleaning from many unstructured text data and compiles them

Natural language processing (NLP) is a subfield of linguistics, computer science, information engineering, and artificial intelligence concerned with the interactions between computers and human (natural) languages, in particular how to program computers to process and analyze large amounts of natural language data.

The Concept of Text Mining--For example, text categorization, text clustering, concept/entity extraction, sentiment analysis, document summarization, production of granular taxonomies, entity relation modeling

Text mining is required if organisations and individuals are to make sense of these vast information and data resources and leverage value. ... The processed data can then be 'mined' to identify patterns and extract valuable information and new knowledge.

There are different text mining methods as in data mining had been proposed such as clustering, classification, information retrieval, topic discovery, summarization, topic extraction. This phase includes evaluation and interpretation of results in terms of calculating precision and recall, accuracy etc.

Some of the popular Text Mining applications include:--

Enterprise Business Intelligence/Data Mining, Competitive Intelligence.

E-Discovery, Records Management.

National Security/Intelligence.

Scientific discovery, especially Life Sciences.

Search/Information Access.

Social media monitoring.

Natural language processing (NLP ) is a type of artificial intelligence that derives meaning from human language in a bid to make decisions using the information.

NLP algorithms are typically based on machine learning algorithms. Instead of hand-coding large sets of rules, NLP can rely on machine learning to automatically learn these rules by analyzing a set of examples (i.e. a large corpus, like a book, down to a collection of sentences), and making a statical inference

NLP is a sub-area of Artificial Intelligence (AI) research that focusses on the processing of language, either in the form of text or speech. More generally, it can be defined as the modeling of how signs (sound/characters/words) representing some meaning are used in order to fulfill a pre-defined task such as translation, summarization or question answering. In contrast to linguistic studies, NLP is an engineering discipline focusing on achieving a pre-defined task instead of leading to a deeper understanding of language.

Natural language processing is one of the most active research areas in AI and provides a rich target for machine learning research as well

Natural Language Processing (NLP) is the study and application of techniques and tools that enable computers to process, analyze, interpret, and reason about human language.

Natural Language Processing involves the application of various algorithms capable of taking unstructured data and converting it into structured data. If these algorithms are applied in the wrong manner, the computer will often fail to derive the correct meaning from the text.

In order for computers to interpret human language, they must be converted into a form that a computer can manipulate. However, this isn’t as simple as converting text data into numbers. In order to derive meaning from human language, patterns have to be extracted from the hundreds or thousands of words that make up a text document. 

This is no easy task. There are few hard and fast rules that can be applied to the interpretation of human language. For instance, the exact same set of words can mean different things depending on the context. Human language ( especially stupid language English ) is a complex and often ambiguous thing, and a statement can be uttered with sincerity or sarcasm.

NLP technologies are not sophisticated enough to understand all of the nuances of human speech. .

Natural Language Processing involves the application of various algorithms capable of taking unstructured data and converting it into structured data. 

If these algorithms are applied in the wrong manner, the computer will often fail to derive the correct meaning from the text. This can often be seen in the translation of text between languages, where the precise meaning of the sentence is often lost. While machine translation has improved substantially over the past few years, machine translation errors still occur frequently.

Many of the techniques that are used in natural language processing can be placed in one of two categories: syntax or semantics. Syntax techniques are those that deal with the ordering of words, while semantic techniques are the techniques that involve the meaning of words.

Examples of syntax include:--

Lemmatization

Morphological Segmentation

Part-of-Speech Tagging

Parsing

Sentence Breaking

Stemming

Word Segmentation

Lemmatization refers to distilling the different inflections of a word down to a single form. Lemmatization takes things like tenses and plurals and simplifies them, for example, “feet” might become “foot” and “stripes” may become “stripe”.  This simplified word form makes it easier for an algorithm to interpret the words in a document.

Morphological segmentation is the process of dividing words into morphemes or the base units of a word. These units are things like free morphemes (which can stand alone as words) and prefixes or suffixes.

Part-of-speech tagging is simply the process of identifying which part of speech every word in an input document is.

Parsing refers to analyzing all the words in a sentence and correlating them with their formal grammar labels or doing grammatical analysis for all the words.

Sentence breaking, or sentence boundary segmentation, refers to deciding where a sentence begins and ends.

Stemming is the process of reducing words down to the root form of the word. For instance, connected, connection, and connections would all be stemmed to “connect”.

Word Segmentation is the process of dividing large pieces of text down into small units, which can be words or stemmed/lemmatized units.

Semantic NLP techniques include techniques like:==

Named Entity Recognition

Natural Language Generation

Word-Sense disambiguation

Named entity recognition involves tagging certain text portions that can be placed into one of a number of different preset groups. Pre-defined categories include things like dates, cities, places, companies, and individuals.

Natural language generation is the process of using databases to transform structured data into natural language. For instance, statistics about the weather, like temperature and wind speed could be summarized with natural language.

Word-sense disambiguation is the process of assigning meaning to words within a text based on the context the words appear in.

Deep Learning Models For Natural Language Processing--

Regular multilayer perceptrons are unable to handle the interpretation of sequential data, where the order of the information is important. In order to deal with the importance of order in sequential data, a type of neural network is used that preserves information from previous timesteps in the training.

Recurrent Neural Networks are types of neural networks that loop over data from previous timesteps, taking them into account when calculating the weights of the current timestep. Essentially, RNN’s have three parameters that are used during the forward training pass: a matrix based on the Previous Hidden State, a matrix based on the Current Input, and a matrix that is between the hidden state and the output. 

Because RNNs can take information from previous timesteps into account, they can extract relevant patterns from text data by taking earlier words in the sentence into account when interpreting the meaning of a word.

Another type of deep learning architecture used to process text data is a Long Short-Term Memory (LSTM) network. LSTM networks are similar to RNNs in structure, but owing to some differences in their architecture they tend to perform better than RNNs. They avoid a specific problem that often occurs when using RNNs called the exploding gradient problem.

These deep neural networks can be either unidirectional or bi-directional. Bi-directional networks are capable of taking not just the words that come prior to the current word into account, but the words that come after it. While this leads to higher accuracy, it is more computationally expensive.

NLP is  is the acronym for neuro-linguistic programming.  NLP is also an abbreviation used for natural language processing. NLP is a means of training computers to understand human language. This is no easy thing. Human language is fluid; words change over time or with context.

NEURO-LINGUISTIC PROGRAMMING AND NLP IN THIS CONTEXT DOES NOT HAVE ANY CONNECTION WITH NATURAL LANGUAGE PROCESSING.  THE WORD NEURO DOES NOT WORK WITH AI.   

NEURO-LINGUISTIC PROGRAMMING IS A BULLSHIT FRAUDENT  PRACTICE TO HONE COMMUNICATION, FACILITATE PERSONAL DEVELOPMENT AND TO MAKE PSYCHOTHERAPY MORE EFFECTIVE ( SIC).

I HAD WRITTEN A UNFINISHED –ONLY 70% COMPLETE --17 PART POST ON NLP ( NEURO-LINGUISTIC PROGRAMMING ) THROWING SHIT ON IT..

https://ajitvadakayil.blogspot.com/2016/12/ethical-nlp-neuro-linguistic.html

NLP (Natural language processing) is simply the part of AI that has to do with language (usually written)

At its simplest form, NLP will help computers perform commands given to them through text commands.

The next stage of NLP is natural language interaction, which allows humans to communicate with computers using normal, everyday language to perform tasks.

Natural language processing is the linguistically oriented discipline in computer science that is concerned with the capacity of software to understand natural human language – written as well as spoken.

Deep NLP is a branch of both Deep Learning and NLP that deals with using Deep Learning to achieve some of the NLP related tasks. For example, sentiment classification.

Natural Language Processing. NLP is a field in machine learning with the ability of a computer to understand, analyze, manipulate, and potentially generate human language. Information Retrieval(Google finds relevant and similar results). .

NLP powers the voice-based interface for virtual assistants and chatbots. The technology is increasingly being used to query data sets as well.

Because Natural Language Processing involves the analysis and manipulation of human languages, it has an incredibly wide range of applications. Possible applications for NLP include chatbots, digital assistants, sentiment analysis, document organization, talent recruitment, and healthcare.

Chatbots and digital assistants like Amazon’s Alexa and Google Assistant are examples of voice recognition and synthesis platforms that use NLP to interpret and respond to vocal commands. These digital assistants help people with a wide variety of tasks, letting them offload some of their cognitive tasks to another device and free up some of their brainpower for other, more important things. Instead of looking up the best route to the bank on a busy morning, we can just have our digital assistant do it.

Sentiment analysis is the use of NLP techniques to study people’s reactions and feelings to a phenomenon, as communicated by their use of language. Capturing the sentiment of a statement, like interpreting whether a review of a product is good or bad, can provide companies with substantial information regarding how their product is being received.

Automatically organizing text documents is another application of NLP. Companies like Google and Yahoo use NLP algorithms to classify email documents, putting them in the appropriate bins such as “social” or “promotions”. They also use these techniques to identify spam and prevent it from reaching your inbox.

Groups have also developed NLP techniques are being used to identify potential job hires, finding them based on relevant skills. Hiring managers are also using NLP techniques to help them sort through lists of applicants.

NLP techniques are also being used to enhance healthcare. NLP can be used to improve the detection of diseases. Health records can be analyzed and symptoms extracted by NLP algorithms, which can then be used to suggest possible diagnoses. 

One example of this is Amazon’s Comprehend Medical platform, which analyzes health records and extracts diseases and treatments. Healthcare applications of NLP also extend to mental health. There are apps such as WoeBot, which talks users through a variety of anxiety management techniques based in Cognitive Behavioral Therapy.

Natural language is the language humans use to communicate with one another. On the other hand, programming language was developed so humans can tell machines what to do in a way machines can understand. For example, English is a natural language while Java is a programming one.

Natural Language Processing facilitates human-to-machine communication without humans needing to “speak” Java or any other programming language as it allows machines to obtain and process information from written or verbal user inputs.

In essence, developers create NLP models that enable computers to decode and even mimic the way humans communicate.

How Does Natural Language Processing Work?

One of the best things about NLP is that it’s probably the easiest part of AI to explain to non-technical people.

Take one of the most common natural language processing application examples, the prediction algorithm in your email. The software is not just guessing what you will want to say next but analyzes the likelihood of it based on tone and topic. Engineers are able to do this by giving the computer and “NLP training”. 

In other words, they provide the software with a huge amount of data about language including sentences and phrases as well as transcripts from live conversations/emails. This way, over time, computer programs are able to learn how to pair words together; what it is we are trying to convey, and what we need to achieve with that communication.

Naturally, predicting what you will type in a business email is significantly simpler than understanding and responding to a conversation. Still, the decoding/understanding of the text is, in both cases, largely based on the same principle of classification.

Generally, the “understanding” of the natural language (NLU) happens through the analysis of the text or speech input using a hierarchy of classification models.

Unlike common word processing operations, NLP doesn’t treat speech or text just as a sequence of symbols. It also takes into consideration the hierarchical structure of the natural language – words create phrases; phrases form sentences;  sentences turn into coherent ideas. In other words, NLP software doesn’t just look for keywrods. 

It uses pre-programmed or acquired knowledge to decode meaning and intent from factors such as sentence structure, context, idioms, etc. For instance, good NLP software should be able to recognize whether the user’s “Why not?” indicates agreement or a question that requires an answer.

NLP is divided in two key categories:--

Natural Language Understanding (NLU)

Natural Language Generation (NLG)

NLU is an essential sub-domain of NLP and have a general idea of how it works.

It is important to point out that the ability to parse what the user is saying is probably the most obvious weakness in NLP based chatbots today. Human languages are just way too complex. Besides enormous vocabularies, they are filled with multiple meanings many of which are completely unrelated.

To nail the NLU is more important than making the bot sound 110% human with impeccable NLG. Why? If a bot understands the users and fulfills their intent, most won’t care if that response is a bit taciturn… 

It doesn’t work the other way around. A bot that can’t derive meaning from the natural input efficiently can have the smoothest small talk skills and nobody will care. Not even a little!

NLU is the post-processing of text, after the use of NLP algorithms (identifying parts-of-speech, etc.), that utilizes context from recognition devices (automatic speech recognition [ASR], vision recognition, last conversation, misrecognized words from ASR, personalized profiles, microphone proximity etc.), in all of its forms, to discern meaning of fragmented and run-on sentences to execute an intent from typically voice commands.

NLP is short for natural language processing while NLU is the shorthand for natural language understanding. Similarly named, the concepts both deal with the relationship between natural language (as in, what we as humans speak, not what computers understand) and artificial intelligence.

NLU is the post-processing of text, after the use of NLP algorithms (identifying parts-of-speech, etc.), that utilizes context from recognition devices (automatic speech recognition [ASR], vision recognition, last conversation, misrecognized words from ASR, personalized profiles, microphone proximity etc.), in all of its forms, to discern meaning of fragmented and run-on sentences to execute an intent from typically voice commands. 

NLU has an ontology around the particular product vertical that is used to figure out the probability of some intent. An NLU has a defined list of known intents that derives the message payload from designated contextual information recognition sources. 

The NLU will provide back multiple message outputs to separate services (software) or resources (hardware) from a single derived intent (response to voice command initiator with visual sentence (shown or spoken) and transformed voice command message to different output messages to be consumed for M2M communications and actions)

Natural-language understanding (NLU) or natural-language interpretation (NLI) is a subtopic of natural-language processing in artificial intelligence that deals with machine reading comprehension. Natural-language understanding is considered an AI-hard problem.

NLU  describes the quest to build machines with true reading comprehension, so that humans can communicate with them in natural human language and the machine can respond appropriately. Commercial applications of interest include applications ranging from text categorization, where emails are routed to the proper department based on their content, to full comprehension of newspaper articles.

Given that the NLP chatbot successfully parsed and understood the user’s input, its programing will determine an appropriate response and “translate” it back to natural language.

Needless to say, that response doesn’t appear out of thin air.

For the NLP to produce a human-friendly narrative, the format of the content must be outlined be it through rules-based workflows, templates, or intent-driven approaches. In other words, the bot must have something to work with in order to create that output.

Currently, every NLG system relies on narrative design – also called conversation design – to produce that output. This narrative design is guided by rules known as “conditional logic”. These rules trigger different outputs based on which conditions are being met and which are not.

The flip side of NLP is natural language generation (NLG), the AI discipline that enables computers to generate text that is meaningful to humans.

Natural language generation (NLG) is the use of artificial intelligence (AI) programming to produce written or spoken narrative from a dataset.

At its simplest, an NLG platform is a computer process that can generate natural language text and speech from pre-defined data.

In natural language understanding the system needs to disambiguate the input sentence to produce the machine representation language, whereas in Natural Language Generation the system needs to make decisions about how to put a concept into words.

Natural Language Processing (NLP) is what happens when computers read language. NLP processes turn text into structured data. Natural Language Generation (NLG) is what happens when computers write language. NLG processes turn structured data into text

The output of NLG algorithms can either be displayed as text, as in a ., or converted to speech through voice synthesis and played for the user, as smart speakers and AI assistants do.

 NLG can turn charts and spreadsheets into textual descriptions. AI assistants such as Siri and Alexa also use NLG to generate responses to queries.

Google’s AI assistant puts both the capabilities and the limits of artificial intelligence’s grasp of human language. Duplex combines speech-to-text, NLP, NLG and voice synthesis in a very brilliant way, duping many people into believing it can interact like a human caller. 

But Google Duplex is narrow artificial intelligence, which means it is will be good at performing the type of tasks the company demoed, such as booking a restaurant or setting an appointment at a salon. These are domains where the problem space is limited and predictable.

NLP is a tool for computers to analyze, comprehend, and derive meaning from natural language in an intelligent and useful way. This goes way beyond the most recently developed chatbots and smart virtual assistants. In fact, natural language processing algorithms are everywhere from search, online translation, spam filters and spell checking.

So, by using NLP, developers can organize and structure the mass of unstructured data to perform tasks such as intelligent:---

Automatic summarization (intelligently shortening long pieces of text)

Automatic suggestions (used to speed up writing of emails, messages, and other texts)

Translation (translating phrases and ideas instead of word for word)

Named entity recognition (used to locate and classify named entities in unstructured natural languages into pre-defined categories such as the organizations; person names; locations; codes; quantities; price; time; percentages)

Relationship extraction (extraction of semantic relationships among the identified entities in natural language text/speech such as “is located in”, “is married to”, “is employed by”, “lives in”, etc.)

Sentiment analysis (helps identify, for instance, positive, negative and neutral opinion form text or speech widely used to gain insights from social media comments, forums or survey responses)

Speech recognition (enables computers to recognize and transform spoken language into text – dictation – and, if programmed, act upon that recognition – e.g. in case of assistants like Google Assistant Cortana or Apple’s Siri)

Topic segmentation (automatically divides written texts, speech or recordings into shorter, topically coherent segments and is used in improving information retrieval or speech recognition)

Using NLP for simple and straightforward use cases is over the top and completely unnecessary.

In fact, if used in an inappropriate context, natural language processing chatbot can be an absolute buzzkill and hurt rather than help your business. If a task can be accomplished in just a couple of clicks, making the user type it all up is most certainly not making things easier.

On the other hand, if the alternative means presenting the user with an excessive number of options at once, NLP chatbot can be useful. It can save your clients from confusion/frustration by simply asking them to type or say what they want. It’s not much different from coming up to the staff member at the counter in the real world.

NLP is a decades-old field that sits at the cross-section of computer science, artificial intelligence, and, more and more, data mining. It focuses on how we can program computers to process large amounts of natural language data, such as a poem or novel or a conversation, in a way that is productive and efficient, taking certain tasks off the hands of humans and allowing for a machine to handle certain processes – the ultimate “artificial intelligence”.

NLP can refer to a range of tools, such as speech recognition, natural language recognition, and natural language generation. Common NLP algorithms are often manifest in real-world examples like online chatbots, text summarizers, auto-generated keyword tabs, and even tools that attempt to identify the sentiment of a text, such as whether it is positive, neutral, or negative.

Considered a subtopic of NLP, natural language understanding is a vital part of achieving successful NLP. NLU is narrower in purpose, focusing primarily on machine reading comprehension: getting the computer to comprehend what a body of text really means. 

After all, if a machine cannot comprehend the content, how can it process it accordingly? (But, drawing distinct, clear insights from data that is anything but clear or distinct – often governed by only half-rules and exceptions, as is common for language– is tricky;

Natural language understanding can be applied to a lot of processes, such as categorizing text, gathering news, archiving individual pieces of text, and, on a larger scale, analyzing content.

Real-world examples of NLU range from small tasks like issuing short commands based on comprehending text to some small degree, like rerouting an email to the right person based on a basic syntax and decently-sized lexicon. Much more complex endeavors might be fully comprehending news articles or shades of meaning within poetry or novels.

It’s best to view NLU as a first step towards achieving NLP: before a machine can process a language, it must first be understood.

Natural Language Understanding (NLU) or Natural Language Interpretation (NLI),  deals with machine reading comprehension by breaking the elemental pieces of speech. NLU communicates with the untrained and unstructured data in order to understand their insights and meanings, i.e. this technique determines a user’s intent.  

Successful NLP must blend techniques from a range of fields: language, linguistics, data science, computer science, and more.

This is why NLP has been so elusive – an academic advance in a small part of NLP may take years for a company to develop into a successful tool that relies on NLP. Such a product likely aims to be effortless, unsupervised, and able to interact directly with customers in an appropriate and successful manner.  

Context awareness is the ability of a system or system component to gather information about its environment at any given time and adapt behaviors accordingly. Contextual or context-aware computing uses software and hardware to automatically collect and analyze data to guide responses.

NLP helps developers to organize and structure knowledge to perform tasks like translation, summarization, named entity recognition, relationship extraction, speech recognition, topic segmentation, etc.    NLP is a way of computers to analyze, understand and derive meaning from a human languages such as English, Spanish, Hindi, etc.

Natural language refers to language that is spoken and written by people, and natural language processing (NLP) attempts to extract information from the spoken and written word using algorithms.

NLP encompasses active and a passive modes: natural language generation (NLG), or the ability to formulate phrases that humans might emit, and natural language understanding (NLU), or the ability to build a comprehension of a phrase, what the words in the phrase refer to, and its intent. 

In a conversational system, NLU and NLG alternate, as algorithms parse and comprehend a natural-language statement, and formulate a satisfactory response to it. Natural language processing tries to do two things: understand and generate human language. Natural language processing (NLP) is a form of AI that extracts meaning from human language to make decisions based on the information. 

Here are some more applications of natural language processing:Email assistant: Auto-correct, grammar and spell check, as well as auto-complete, are all functions enabled by NLP. Machine translation is a huge application for NLP that allows us to overcome barriers to communicating with individuals from around the world as well as understand tech manuals and catalogs written in a foreign language. 

Google Translate is used by 500 million people every day to understand more than 100 world languages.

Natural Language Processing helps in gaining insights from meaningless and unstructured data. It basically aims to convert human language into a formal representation which is easy for computers or machines to manipulate. 

Besides text recognition, NLP has the ability to recognise meaningful insights from videos and other unstructured materials. This technique helps a machine to understand the sentences and convert them into meaningful information.

NLP includes various approaches such as tokenizer, entity extraction, sentence boundary detection, etc to extract information from free-form text. The use cases of this technique involve summarising text by identifying the entities present in the document, analysing the sentiment in a given text, classifying documents by labelling it as sensitive, spam, etc. 

Conversational AI – describes a branch of artificial intelligence (AI) that focuses on interpreting human (colloquial) language and subsequently communicating back with them. Powered by advanced features such as natural language processing (NLP), conversational AI is the logic that creates virtual conversations. Conversational AI examples of these are the voice assisted devices that are commercially available.

Conversational UI – is the platform that allows a user to communicate with a computer that mimics a ‘human like’ conversation. Powered by natural language processing (NLP), a conversational UI framework allows for an interaction that accounts for the user’s sentiments as well as the context carried during the conversation.

“Common-sense reasoning is a field of artificial intelligence that aims to help computers understand and interact with people more naturally by finding ways to collect these assumptions and teach them to computers. Common Sense Reasoning has been most successful in the field of natural language processing (NLP), though notable work has been done in other areas. 

This area of machine learning, with its strange name, is starting to quietly infiltrate different applications ranging from text understanding to processing and comprehending what’s in a photo. Without common sense, it will be difficult to build adaptable and unsupervised NLP systems in an increasingly digital and mobile world. …

NLP is where common-sense reasoning excels, and the technology is starting to find its way into commercial products. Though there is still a long way to go, common-sense reasoning will continue to evolve rapidly in the coming years and the technology is stable enough to be in business use today. It holds significant advantages over existing ontology and rule-based systems, or systems based simply on machine learning.”

The field of NLP brings together artificial intelligence, computer science, and linguistics with the goal of teaching machines to understand and process human language. NLP researchers and engineers build models for computers to perform a variety of language tasks, including machine translation, sentiment analysis, and writing enhancement. Researchers often begin with analysis of a text corpus—a huge collection of sentences organized and annotated in a way that AI algorithms can understand.

The problem of teaching machines to understand human language—which is extraordinarily creative and complex—dates back to the advent of artificial intelligence itself. Language has evolved over the course of millennia, and devising methods to apprehend this intimate facet of human culture is NLP’s particularly challenging task, requiring astonishing levels of dexterity, precision, and discernment. 

As AI approaches—particularly machine learning and the subset of ML known as deep learning—have developed over the last several years, NLP has entered a thrilling period of new possibilities for analyzing language at an unprecedented scale and building tools that can engage with a level of expressive intricacy unimaginable even as recently as a decade ago. 

Computational linguistics is a field of vital importance in the information age. Computational linguists create tools for important practical tasks such as machine translation, speech recognition, speech synthesis, information extraction from text, grammar checking, text mining and more. confluence of artificial intelligence and computational linguistics which handles interactions between machines and natural languages of humans in which computers are entailed to analyze, understand, alter, or generate natural language. 

Computer science includes the study of formal languages (mathematically defined as production systems). Formal languages can be used to model natural languages, and that is done in computational linguistics, a branch of linguistics The difference is that Computational Linguistics tends more towards Linguistics, and answers linguistic questions using computational tools. Natural Language Processing involves applications that process language and tends more towards Computer Science. 

Computational linguistics is an interdisciplinary field concerned with the statistical or rule-based modeling of natural language from a computational perspective, as well as the study of appropriate computational approaches to linguistic questions.  Traditionally, computational linguistics was performed by computer scientists who had specialized in the application of computers to the processing of a natural language. 

Today, computational linguists often work as members of interdisciplinary teams, which can include regular linguists, experts in the target language, and computer scientists. In general, computational linguistics draws upon the involvement of linguists, computer scientists, experts in artificial intelligence, mathematicians, logicians, philosophers, cognitive scientists, cognitive psychologists, psycholinguists, anthropologists and neuroscientists, among others.

NLP, also known as computational linguistics, is the combination of AI and linguistics that allows us to talk to machines as if they were human. NLP powers predictive word suggestions on our mobile devices and voice-activated assistants like Siri, Bixby and Google's voice search Computational linguists create tools for important practical tasks such as machine translation, speech recognition, speech synthesis, information extraction from text, grammar checking, text mining and more 

The difference is that NLP seeks to do useful things using human language, while Computational Linguistics seeks to study language using computers and corpora. Computational linguistics (CL) is the application of computer science to the analysis, synthesis and comprehension of written and spoken language. 

Computational linguistics is used in instant machine translation, speech recognition (SR) systems, text-to-speech (TTS) synthesizers, interactive voice response (IVR) systems, search engines, text editors and language instruction materials. The interdisciplinary field of study requires expertise in machine learning (ML), deep learning (DL), artificial intelligence (AI), cognitive computing and neuroscience.  computational linguistics is the scientific study of language from a computational perspective.” 

Work in computational linguistics (CL) is concerned with modeling natural language, and draws on a variety of other disciplines, among them cognitive computing and artificial intelligence. The goal of computational linguistics is to develop software able to understand natural language, the everyday language we use to communicate.

The difference is that Computational Linguistics tends more towards Linguistics, and answers linguistic questions using computational tools. Natural Language Processing involves applications that process language and tends more towards Computer Science..

Computational linguistics explores how human language might be automatically processed and interpreted. Research in this area considers the mathematical and logical characteristics of natural language, and develops algorithms and statistical processes for automatic language processing.

Human language is processed by computers in every sector of contemporary society. Smartphones are required to register the meaning of language inputs, machine translation helps us to communicate, and information from large data sets is extracted and summarised. 

Computational linguistics concerns the development and analysis of the methods which facilitate these applications and others like them. Analysis might therefore focus on anything from fundamental linguistic issues such as modelling the meaning of the word and recognising the grammatical structure of sentences, to complex applications such as machine translation or the assessment of statements for factual accuracy. 

Analysis is conducted using statistical and computational processes such as neural networks or processes borrowed from logic. Computational linguistics therefore makes an important contribution to the further development of artificial intelligence and serves as a driver of innovation in this field.

THIS POST IS NOW CONTINUED TO PART 12 , BELOW--

https://ajitvadakayil.blogspot.com/2020/02/what-artificial-intelligence-cannot-do.html

CAPT AJIT VADAKAYIL
..