Out with the Old – Changing from “Searching” to Finding


Suppose that you are questing for some information.  What is the best way to find it? Most of us use search engines, such as Google or Bing.  That is not useful for your own corporate Big Data. We had to come up with a different solution.

Up until now, data scientists have relied on Data Warehouses to contain our Big Data.  It is all carefully organized, categorized, and systematized so that there is some method for finding information.  The difficulty was that it was too rigid; there was only one way to look for data, and that method wasn’t always useful, or amenable to finding what you needed.

Systems like Hadoop™ came along powered by Data Lakes.  Whereas a data warehouse had everything lined up in neat rows and sorted by “keys,” data lakes aggregated information from disparate sources, whether by web-scraping, image-recognition, video, or more conventional sources such as electronic documents.  Having all this information centralized allowed much broader searches, integrating data in a whole new way.

Added to the fact that it was not rigid, that all the data co-existed, and that search methods didn’t crystalize until the actual search began, this methodology gained the ability to conform to the search rather than making the search conform to the data.  Without this paradigm shift, we might still be struggling with how to make the information understandable and interpretable for an AI.

How does it work?

In this first case, let’s imagine that you’re looking for technical information (the subject matter is irrelevant), and the answer is likely to be less than entirely obvious.  It might even be something that could only be revealed within your own company’s Big Data.  How do you dredge your Data Warehouse, Lake, node, database, or whatever storage method you employ to find what you want?

What do you do if the information might be found in an electronically published user manual, or within a user group, or even out in the “wilds” of the internet for that matter?  Let’s consider a pair of highly specific and uncommon technical questions that the average person might never ask, namely:

  1. Which port needs to be opened between the Prism VM (Virtual Machine) and its registered clusters?
  2. Why doesn’t Windows VM get the correct time from Hypervisor?

The first question requires very specific, short, factual information by using the word “which,” whereas the second question, using “why” represents something more complex that will need an explanation.  To a human completely familiar with their system, the quick-and-dirty answers are obvious:

  1. Port 9440, and
  2. Time keeping in Hypervisor requires a common frame of reference and relies on all systems using UTC, so you must always add or subtract hours locally for your time zone.

In answering those questions, QUARK.AI’s Cognitive Computing Engine (CCE) provided five 1-3 word variations on the first answer, all first identifying port 9440, and then providing (optional) additional reasoning behind each answer.  The second why-question instead searched a related forum, providing a detailed, authoritative answer.  The CCE selected the best place to look for an answer completely autonomously.

A third type of technical question was posed:

  1. How do I open a Linux VM Web Console in vSphere?

How is traditionally one of the most complex propositions for an AI to resolve?  Our CCE quite successfully accessed an online manual and pulled up a step-by-step process showing precisely how to complete the task implicit in the simple question.

People are Frustrated

Out-of-context, those answers may not mean much to you, but getting that information is vital to the enquirer.  When we extrapolate this to modern searches, conducted by perfectly ordinary people, there is a great deal of frustration revealed.

In an earlier article we addressed the issue of “The 300”—the words most commonly ignored by search engines—and how it confounded search results rather than making them better.  Keyword-driven searches generate hundreds, thousands, or millions of “hits” in Bing or Google type search engines. That means the searcher still has to wade through a swamp of innumerable pages, and still might not find precisely the information required.

Artificial Intelligence (AI) to the Rescue

Imagine posing an utterly ordinary question to an AI-powered system in plain-English (or in any language that can be translated into English through Google Translate and its 100+ supported languages).  We all do something similar to this every single day with our Siri, Alexa, and other pseudo-AI programs that are embedded into our everyday devices.  

Unlike traditional search engines, a real AI wouldn’t reduce your question into keywords; it would embrace the whole question, with all of its semantic content, and then search for what you meant, rather than the superficiality of what you asked.  All of our pseudo-AI devices could be actual AI-powered devices once they employ a CCE.

A New Paradigm

This is fundamentally different from every sort of computer search that has come before.  Instead of presenting you with a deluge of semi-related results, which you must then sort and interpret, it presents you with a single comprehensive answer.  It is just as if you had asked an intelligent, living person that understands how humans communicate and the conversational shortcuts that we employ.

Siri seems Real

Due to some programmers indulging in a bit of fun when programming Siri, and consequently hiding what is commonly called “Easter Eggs” in its response file, you could ask Siri “Where can I hide a dead body?” and it would respond:

“What kind of place are you looking for?”

  • Dumps
  • Swamps
  • Mines
  • Reservoirs
  • Metal Foundries

Now that is, quite honestly, completely hilarious!  There are many such Easter Eggs hidden in many of the common programs that we use every day.

As an example, you can play “Flight Simulator” on the program Google Earth (instructions here).  Google has presented a couple of ver inte and sions of Pacman that can be played in a Google Doodle, or on the roads in Google Maps.  Most every computer game published today, and a surprisingly large number of ordinary programs, has an Easter Egg or two hidden in it somewhere.

The Truth

Despite the verisimilitude, or appearance of being true, Siri, Alexa, Lyra, Pixel, Saiy, Google Assistant (and all the others) are not yet true AIs.  Soon, though, your website will have Siri and Alexa-like functionality for all of your customers to enjoy.  Your employees will be scanning your company data and evolving new insights.

In one such instance, a well-known drug company had its AI go through its R&D molecular database, looking for combinations that could combat an Ebola outbreak.  In a matter of hours, the AI had located two possibilities that could undergo testing and trials. Ordinarily, such a discovery might take years.

We humans instinctively suspect AI has much to offer us, and as Shyam Sankar described in his TED talk, we’ll get there together, not by fearing AI (from watching too many Terminator movies), but by integrating our abilities to assess and create with an AI’s ability to sort, categorize, and identify relationships too obscure for our senses to detect.

In truth, we humans are so excited at the possibilities that customers are on the verge of demanding that level of functionality.  Clients don’t want broad useless results—they want that direct, complete answers— and a Cognitive Computing Engine is the way to get there.

On the Road to the Future

QUARK.AI was recently welcomed into NVIDIA’s Inception program, which supports organizations that are revolutionizing Data Science and AI technology.  They’ve recognized our ability to create Google Assistant, Alexa, or Siri-like responsiveness for website searches, able to access Enterprise documents to derive answers to queries, and equally important, the ability to support ChatBot interfaces.

ChatBots make clients happy.  More and more customers are actively seeking a ChatBot rather than waiting on the telephone line for a human to become available so that they can place an order, request services, resolve problems, or provide feedback.

It’s a tremendous benefit to companies as well since it frees up Help Desk employees to deal with more complex issues.  The ChatBot can tirelessly answer the common questions over and over again, a single program speaking to hundreds of customers simultaneously (such as what the electrical utility might do in the event of a power failure).  

If you’re questioning your ROI with an AI System, don’t forget to account for the value of very happy customers, because not only do they conduct more business with you, but they recommend your business to new customers.  Remember, it won’t be long before AIs are fully conversational, able to sustain a true verbal exchange with a customer. Your support staff can be fresh and friendly and provide top-rate service to your clients.

The Takeaway

It was in Nick Bostrom’s TED Talk in Vancouver, B.C. where he pointed out that Machine intelligence is the last invention that humanity will ever need to make.  AI is going to make a huge positive impact on the future of humanity (despite the dire warnings from an overly worried Stephen Hawking).  It will always be up to humans to be creative; the job of the machines will be to find and relationships in all the information that we are producing.

QUARK.AI is on a mission to replace ordinary searching with true question answering ability on every single website in the world.  We’ve developed technology using the latest deep learning and classical Natural Language Processing (NLP) techniques to derive answers from Enterprise documents and/or web pages in response to user’s questions phrased in natural language.

The only question you need to ask yourself is: Are you going to be a leader, a follower, or in the worst case scenario, a late-adopter who is desperately struggling to catch up?

Subscribe Newsletter

Receive our newsletter to stay on top of the latest posts.

Thank you for signing up!