Read My Mind – AI eliminates “guesswork”

Blog

True Artificial Intelligence (AI) is going to replace Siri, Pixel, Lyra, and Alexa’s limited repertoire very soon.  Why? Despite their occasional brilliance due to good programming, they are not real AIs by any stretch of the imagination.  They are quite remarkable at what they do and, if you are judicious in your questioning, you can have what amounts to a sustained “real” conversation.

The difficulty is that they all possess a limited set of responses for specific circumstances (which can be quite valuable in a defined environment), but they certainly do not approach human’s intelligence, analytical skills, or ability to make colloquial associations.  The keyword approach to information retrieval in no way resembles how humans associate or recall information within their brains.

Imagine if these programs were actually driven by a functional AI program!  Gone would be the nonsensical associations, and you would get only the real information you needed.  Instead of thousands of pages with some frustratingly vague relation to some of the words in your query, you would get a concise, specific answer to your exact question—and who wouldn’t want that?

Consider that when Daltrey, Townsend, Entwistle, and Moon were still with us and performing, it would have been difficult to look them up on a search engine, had one existed at the time.  Two words composed the rock band’s name, and both are ignored by search engines. Nowadays, if you search for “The Who”, it will take you right to their website, but only as a “special case.”

The single greatest difficulty encountered with traditional search engines is the use of “stop words” to delimit searches.  When you search Google for “The Who” it provides 6,820,000,000 results in 0.56 seconds. By ignoring stop words, it can return results much more quickly; it does away with repeated words, the most common words in English, apostrophes (hasnt, couldnt) and periods (ie, etc, eg).  

The 300

Although there is no definitive universal list, here are the words most commonly ignored by search engines. Have a quick look and then try to imagine how many times you’ve received completely unsatisfactory results because the keyword matching software skipped something vital to an appropriate answer.  

List of stopwords

Typical stop words

Notice all the spelled-out numbers, (one, two, three, etc.) are ignored, as are twenty, forty, sixty, and hundred.  Good luck trying to find “twenty-four top computer systems since 2015 until now”. The only term used from that search string would be “2015” which is going to get you a lot of irrelevant data and useless answers.

Now, strictly speaking, the algorithms have gotten much, much better.  If the algorithm determines that almost all of the content is stop words, it will read more of the words.  Googling “twenty-four top computer systems since 2015 until now” provides 269,000,000 results in 0.86 seconds.  As you can see, it is not in the billions of results like our “The Who” search, and it took nearly twice as long, but it provided several pages of relevant results.

It’s still not enough

Why Websites Should Replace Conventional Searching

AI can do the job so much better.  Current AI is quite advanced when compared to previous techniques.  It is still only at the beginning of its evolution and will become phenomenally competent as we progress, but truthfully, despite its potential, we’re only just starting to realize its benefits.

AI’s strength is currently in heavy duty data sorting.  Humans can only manage a few simultaneous tasks.  For example, consider Keith Moon, mentioned earlier, or Buddy Rich and Gene Krupa.  These are three of the best drummers in the world, using all four appendages, doing different things, simultaneously in perfect syncopation.  This (Buddy & Gene drum battle) would be impossible for the average person.

The reason for this is quite simple.  Nerve impulses in the human brain, and throughout the body, travel at about 100 miles per hour.  In some cases (pain receptors) it can be as little as 1.1 mph, and within the brain itself, up to 120 mph.  That is a very wide range but still pretty slow compared to the speed of information transfer in a computer chip, nearly at the speed of light, which is 186,282 miles per second.  Exceptional humans have specific and more efficient connections.

When you combine computer speed with the ability to perform trillions of operations per second, it’s pretty clear that computers can outperform humans when it comes to reviewing documents.  Even if an AI program needs to perform a million operations to reach the same conclusion that a human could by making logical leaps, it is still 1,000,000 times faster, so it wins the race with ease.  When you add the further consideration that an AI isn’t restricted to looking at documents sequentially like a human, there is simply no contest.

Harnessing that Power

Current ChatBots are notoriously limited in their ability to respond realistically.  Within the confines of their designed operations, they can sometimes replace a human invisibly (to their correspondent), but the responses are driven by FAQs and clever programming, not innate intelligence or reasoning ability.  Or at least that was so until now…

QUARK AI to the rescue

QUARK AI is not a ChatBot company, though we are comfortable integrating our system with your ChatBot, or one of your choosing.  We can even create a custom ChatBot interface that precisely fits your needs.

What we have done is develop a Cognitive Computing Engine (CCE) utilizing the latest developments in Deep Learning (DL).  This is supplemented by Natural Language Understanding (NLU), so a person can phrase their inquiry in plain English.  It also utilizes Natural Language Processing (NLP) to enhance understanding when homonyms (words sounding or spelled the same) are used contextually, such as differentiating between polish and Polish.

Speed is of the Essence

People, particularly busy researchers, need to be able to access the most recent information promptly so that their investigations remain relevant and not redundant.  Even ordinary people have been known to spend more than two hours searching for precise information on a particular subject.  

Almost three-quarters of people surveyed have reported “search fatigue” while seeking information on conventional websites, and 75% of those people report abandoning the search without obtaining the information they needed.  It has never been completely satisfactory before, and now it is unnecessary.

Making it Work

Through a combination of AI planning techniques and more than ten years of research into automated human/computer dialog (both multi-agent and completely autonomous systems), along with studying spoken word conversational interfaces, our Cognitive Computing Engine has become a reality.  The key was to eliminate the entire keyword search scenario from the equation because it is both inefficient and produces a lot of unrelated hits.

We’re all familiar with IBM’s efforts beating human champions in chess and in Jeopardy; now Google’s AlphaGo is a winner in the ancient Chinese game of GO.  While the last one is a stunning achievement, it is Jeopardy that stands out for most people.

The IBM WATSON computer which accomplished this was very limited—in that its sole capability was defeating a human Jeopardy champion.  It required thousands of hours of programming and millions of dollars to be able to interpret the puns, wordplay, and double entendre nature of Jeopardy questions to accomplish this task, and the result was an idiot savant or one-trick-pony.

WATSON, in its current and more broadly-functional form, is now a suite of programs which provides very human-like responses when it is equipped with the appropriate data-model for the situation.  This is the sort of technology and technique that is required to assess vast numbers of documents and understand them in a relational way, eliminating the dependence on keywords.

Similarly, our Cognitive Computing Engine does not rely on keywords either.  It interprets and understands the whole sentence of the question.  The differences between our AI and WATSON are two-fold.  

First, WATSON is (essentially) the prototype, meaning it is top-heavy with development effort.  Second they have spent more than a billion dollars to get it to its current status—and that is a lot of capital that they need to earn back.  These two factors mean that they are going to improve what they have rather than sweep it all away to begin again with newer techniques.

Benefiting from their experience, newcomers like us can start afresh, without the inertia of a massive monetary investment steering development.  And by beginning with faster more stable infrastructure, modern processors, and tighter code, we can bypass problems inherent in 2017 and earlier systems; we can stick to up-to-date Best Practices, premium hardware, and the latest coding techniques, meaning that our system will not only be slimmer and better, but also less expensive to implement and maintain.

Searchers are Frustrated

If a person were to ask “What are you up to?” you would understand it immediately, yet Google responds with 897,000,000 results (0.59 seconds) defining the phrase in English—not a very useful response!  Our engine might be more pragmatic, explaining its current operations, or even recognize it as a greeting with low semantic content and say “Not much—How about you?”

This, in and of itself, makes a good case for getting away from searches driven by any keyword-matching paradigm.  By empowering personal tools like Alexa or Google Assistant with a Neural Network (NN) and Deep Learning backed Cognitive Computing Engine, results will be much more explicit, and eliminate the need to sift through dozens (or thousands) of pages of results.

That is the fundamental difference—Deep Learning isn’t possible without a substantial and functional Neural Network.  These networks emulate the interconnectivity of a human brain. Of course, it is currently impossible to replicate the number of connections in an organic brain.  We have 100 billion neurons in our brain, and each has about 1,000 synapses or 100 trillion in total.

Any number of synapses might be responsible for the smell of cinnamon, but some of those same ones might also be part of a chain that defines the taste of “apple,” or helps you sense the passage of time.  One synapse doesn’t equal one thought, memory or data bit. Each time the chain fires in a different order, it is a different thought or effect. The connection possibilities are almost limitless.

That immense complexity may be discouraging to data scientists working on AI.  On the other hand, we don’t need that level of interconnectedness—computers compensate for their simplicity with unimaginable speed, as mentioned above.  They can be trillions of times less complex because they are trillions of times faster!

The Takeaway

The one thing you shouldn’t do is be intimidated by the technology, or by AI itself.  Those that climb aboard this DeLorean headed for the future will garner immense benefits; those that don’t will be left struggling and might never catch up.

Human knowledge is currently doubling every year and a half.  All the information we have created, since this moment, all the way back to the beginning of human history, comprises just half of what we will have in 18 months.  

Open Access has made it possible for people and scholars to examine documents and discoveries without a paywall standing in the way.  The problem is that although we can understand all that information, our lifetimes are far too short to be able to read it all.

We need to deploy an AI-driven Cognitive Computing Engine so that it can read all this new information we are creating, trillions of times faster than we can do it.  Then it can assimilate it, becoming able to answer our questions posed in plain, everyday language.

Without AI, we’ll be like a Monarch with a treasury full of wealth, but who is so absorbed in bookkeeping that it never gets used.  If you want to monetize this information, to create new revenue streams, you need to get on board with AI and eliminate keyword searching, because keywords are now an evolutionary dead end.

Subscribe Newsletter

Receive our newsletter to stay on top of the latest posts.

Thank you for signing up!