How Businesses Are Using Natural Language Processing To Talk To Data

CEO of Crux Intelligence and four-time founder/board member. Putting AI in the hands of every business user.

Today’s business users rely on a collection of reports and dashboards to better understand the data underlying their operations. These tools are most often designed by IT organizations, which use coding languages like SQL to ask questions of their database and report the findings back to business users.

Because of this, the modern enterprise has access to a huge collection of data, offering feedback on everything from sales and marketing campaigns to supply chain operations and logistics. However, not every organization has the tools and skills to maximize the value of this data.

When IT employees are tasked with querying the company’s databases, a barrier is erected that limits the usefulness of the company’s business intelligence platform or strategy. Here’s the issue: A business user doesn’t have the technical ability to ask questions of the data; an IT user doesn’t have the business acumen to most effectively form the query and derive insights. While using some available data is better than using none at all, a business intelligence tool is significantly less valuable if it can’t be operated by every business user.

Speaking Different Languages

Why is it so difficult for business users to communicate with their AI-backed tools? At their core, humans and computers process information and communicate in fundamentally different ways. When humans ask questions, they gather their thoughts into phrases and sentences.

When computers are given a question, they break down these long phrases into individual entities, which must then be classified into different categories for the computer to make sense of them. If you use the example of a customer buying paper for their offices (How long ago did Company X purchase reams of printer paper?), the computer will break it down into “how long ago” (question referring to time), “Company X” (business entity), “purchase” (action), “reams” (unit) and “printer paper” (product). By understanding these entities and how they relate to each other, the computer can then work toward answering the question.

Unfortunately, the way humans think and speak makes life inherently more challenging for computers, particularly if the questioner hasn’t been formally trained in the ways computers process information. Each individual has several idiosyncrasies that apply to the way they speak, whether that’s the way they structure their sentences, the different words they choose to express thoughts and possible slang or colloquialisms that may not be part of the computer’s lexicon. These unique aspects of communication, which make human-to-human conversations more varied and interesting, only serve to muddy the waters for a computer built to think in black and white.

Natural Language Processing 

For artificial intelligence tools — and business intelligence platforms in particular — to be useful, business employees must be able to directly ask questions of the data. The key solutions to this problem are text and language processing-based, which enable computers to understand human questions and their semantic structures, meanings and how they can be applied to the data.

A natural language interface is necessary to deliver maximum value and to provide a utility that can adapt to the needs of the business. Businesses are taking different approaches to natural language interfaces, whether building their own in-house or adopting a pre-built solution from a vendor. Each of these options comes with its own set of challenges. 

Some of the challenges when building NLP solutions in-house, include:

• Data availability. To implement NLP and machine learning solutions, availability of raw data is essential. Sometimes, for example, the same words can have different meanings based on the context. Humans can understand the meanings of different words based on the context of the sentence but, while developing NLP solutions, you need a lot of data to train the language model. Do you have the necessary data? 

• Domain-specific language understanding. Different businesses and industries often use very different vocabulary or language which is specific to the business. An NLP processing model needed for healthcare would be very different than one used to process insurance queries. Developing domain-agnostic NLP solutions is even more complex. 

• Time-consuming implementation. Building such a system takes a lot of effort. Such NLP-ML solutions are able to come up with hidden insights without being explicitly programmed where to look, but it’s time-consuming. Make sure your team is aware of this challenge and that you have the proper resources before proceeding on this kind of build. 

On the other hand, some of the challenges when adopting existing NLP solutions are: 

• Ability of the system to understand business queries. You need to decide between keyword matching solutions and semantic search solutions — that is, the ability to understand the words in the sentence, its meanings and the context. Make sure you understand these differences to know which is more optimal for your business. 

• Ability to ask in business queries in natural language. Many solutions require the user to query the data with complex query language, complex filtering or complex structured queries. Ask vendors whether it’s possible for business users to ask questions in natural language in spoken and text format and decide whether this functionality is crucial. 

Make-Or-Break Moment

The past 10 years have resembled the Wild West for artificial intelligence: companies sitting on gold mines of data trying desperately to extract value from their resources. While some don’t know where to look, others use the wrong tools to dig into their information. Today’s companies are separating themselves based on the choices they made in the early days of data analytics.

This may be the last chance for organizations to implement the type of AI-backed solutions that could transform their operations. Both late entries to the field and those who previously backed the wrong horse will need to choose the new tools that will lead them forward. A natural language interface empowers every employee to ask questions of the data. Choosing the wrong natural language option is akin to sending unarmed soldiers into battle, and no business leader will be happy with the results.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?