On the state of Artificial Intelligence – Part 1

As broad industry trends have developed, we’ve found it surprising how our philosophies, definitions, and approaches to building artificial intelligence differ from so many others. We maintain a very traditional view of what constitutes AI, and we believe that biology has a lot to teach us. This is definitely not the norm today. Our conception of intelligent machines has required us to consider many specific computational issues and questions that seem largely ignored in the tech industry. Moreover, what is often promoted as AI by marketing teams seems to have very little to do with intelligence. And despite excellent work and truly impressive examples of ingenuity by researchers in both universities and tech companies, we find that the vast majority of this effort is carried out within very narrow boundaries. This follows in the wake of some high-profile successes and the utter dominance of statistical machine learning, the vogue in AI today. Today’s approaches are without doubt very powerful for certain problems. It’s just not clear that the current language, thinking, and machinery of machine learning necessarily provide the best way to frame questions and seek answers to the problems that arise in the design of intelligent machines.

Intelligence as a philosophical subject and AI as a focus in computer science both have long and interesting histories. From the dawn of the computer age, people were fascinated by the possibility of building thinking machines — machines that could do the sorts of things that humans do with ease. The well-known Turing test for AI comes to us from this era. Turing really wanted to ask if computers could think, but more practically, settled on the question of whether a computer could behave as an intelligent human would. He posed this as playing an “imitation game”. You (the examiner) get five minutes to ask questions of the player (the machine or human). The point of the game is to come up with a set of clever questions that a person could answer, but would trip up the computer; hopefully, this would illuminate the sort of thought processes that require intelligence. This was really meant to be a theoretical game, so Turing allowed a great deal of leeway about what what might be reasonable for (some future) computer to compute (nearly anything, turns out). The test has been the subject of many philosophical debates over decades but makes one thing clear: it is not easy to define intelligence in a broad sense.

Philosophy aside, the Turing test illustrates that one we can try to define intelligence is by its external appearances. We all do this to some extent, based on what we have experienced with computers. Computers are good at some things, but notoriously bad at others. Thus, the de facto definition for AI becomes: something a computer does that you don’t expect a computer to be able to do. Certainly this is the basis for how AI has been marketed — -not wholly unreasonable approach, but an odd way to define intelligence. This definition is obviously a moving target, and one that confuses consumers, investors, and researchers alike about the nature of AI. Where we take particular issue with marketing trends is the dogged determination to label any solution to any potentially difficult problem as “AI”.

In fact, most of what gets called AI today is really automation. What is the difference? Automation means that we can take a problem and break it down into small steps that a computer is designed to execute. As long as a task can be broken down into these instructions, there are few practical limits to what can be done, given enough time, memory, etc. Certainly, we have been automating highly complex tasks long before AI was a marketing term: connecting cell phone calls amidst hundreds of millions of others, managing nuclear reactors, and controlling rockets, to name a few. The key point is that in order to automate a process, we must anticipate all possible situations that may arise. When we don’t anticipate all the situations, or we do not program the computer to respond properly to a situation, it fails the task (we have a bug). We often say that the computer can make decisions based on data, but the conditions for choosing to do one thing over another must also be fully anticipated and specified accordingly.

We can take this example of automation a bit further. The thermostat could “learn” about your habits, rather than requiring you to specifically set the times and temperatures. It could keep a record of the times that you turn the heat off and on. Once it gathers enough data it can start making adjustments to the heat to mimic what you usually do. So now it reduces the temperature setting at 11pm and you didn’t have to tell it to. In fact, it would likely be a bit more accurate, determining that you actually only need the heat until 10:38 pm. Very nice. We have added some good functionality, but we are still just automating a few more items. The thermostat still doesn’t understand that you are cold, it just recorded and repeated some usage patterns. It doesn’t distinguish between changing the temperature on the thermostat and the house being warmer.

Consider that when you return to the house in the evening, you might have preferred to turn up the heat 30 minutes earlier (if you had been able to) so that a pre-warmed house would await you. Anticipating how nice this pre-warm function might be, you could certainly program that into the system. But again that would just be another successive approximation to what we anticipate what an intelligent being would do. We are always playing catch up.

For other problems that we would like to solve, anticipating and specifying all the possible situations is impossible. For instance, imagine that you want recognize animals in a set of digital images as cats or dogs. The animals in the images might be very different sizes, pose differently, have different fur colors. There may be lighting differences and all sorts of backgrounds. Even in a small image, there are literally billions of potential combinations of colors of the pixels in the image that could correspond to a cat or a dog. One would think that there is no way to account for them all. Nonetheless, we as humans can effortlessly identify the animal and decide the species. The reason we can is because there are statistical regularities in the patterns of pixels, such as in edges or color blobs. Biological visual systems have solved this problem of identifying such regularities and interpreting them.

For purposes of automation, this problem can also be solved well with statistical machine learning. Much like the thermostat learned the patterns of desired temperatures, machine learning allows us to identify patterns in example data, even if these patterns are highly varied and complex. In the past handful of years, considerable progress has been made in “deep learning,” which is a catch-all phrase for using neural networks to solve machine learning problems. Neural network computing is an area of AI research that, as the name implies, takes inspiration from networks of biological neurons in brain. There is nothing necessarily deep about what’s being learned — just that the networks are composed of many “layers” of artificial neurons rather than just one or two as in the past.

Historically, it has been difficult to train neural networks with more than a couple of layers. Not surprisingly deep networks require lots of memory and a very large number of examples for training. In the well-known ImageNet competition the standard training set is 1.2 million images. Moreover, people “augment” the example set further by doing things like mirroring the images: A cat is still a cat when you see it in the mirror, but the pattern of pixel colors is not identical. Using the term “deep learning” emphasizes that we are in a new era of neural networks where previous issues have been overcome. Mainly due to to cheap, fast computer hardware with lots of memory, very large data sets, and in some cases algorithmic improvements, training such networks today is a less daunting task. There are online courses, excellent open-source software libraries, and an increasing number of companies that provide APIs to handle deep learning tasks. These all make it possible to provide a basic level of deep learning functionality without much experience or technical understanding. Thus, the past few years have seen an explosion in the number of companies that market deep learning solutions.