Understanding learning and models
The way that humans learn has been studied for many decades now. There are a handful of psychological theories that try to explain how we acquire knowledge, use it, and generalize it in order to apply what we know to completely new scenarios. Taking one step back, we could ask ourselves: what does it mean to learn? We could say that, once we learn something, we are able to repeat it in a more or less detailed way. In reality, learning implies much more than just copying a behavior or memorizing a piece of poetry. In fact, we understand what we learn and are able to generalize that knowledge, which helps us to react correctly to new people, places, and situations.
The need to create a machine that somehow mimics our human behavior and intelligence has been desired for a very long time. Hundreds of years ago, kings were amazed by chess-playing machines, musical instruments that did not require a human player, and mysterious boxes that answered all kinds of questions. These, many times fake, inventions show that one of the greatest dreams of humans is to create an intelligent being, which is able to replicate easy or difficult tasks that are usually performed by people, even when intelligence is an elusive and not easily-defined thing.
Many years have passed, and technology has evolved in such a way that we can now create machines that think, or at least seem to. In fact, most of the systems that we call intelligent are just able to perform repetitive tasks or react to external inputs according to whatever we have showed them by way of example. As we progress through the chapter, we will see that some of the defined characteristics of human learning and intelligence are already part of modern machine learning systems and some are still the subject of science fiction novels.
By definition, machine learning means to teach a machine or an algorithm to perform tasks. We have been doing this for many years now – it is called programming. We give a computer a set of instructions, the order in which they should be executed, and a number of options of how to react to a limited number of inputs. If the input is not known, or if we ask the computer to do something that is not contained in the program, then it will fail, showing an error. The difference between this traditional paradigm and machine learning is that we will never tell the computer exactly what to do. We will either let it discover patterns or show it samples of what we want. We will use programming, of course, but just to define algorithms that learn in the sense that was described previously. From finding the straight line that better represents a set of points to driving a car, everything a machine can do is learned in this way.
As babies, we start exploring the world around us. Since we are too young to understand words or examples, we basically experience the world through our senses. We learn the difference between hard and soft, rough and smooth, hot and cold. We can call for attention when we need something, and we can even gain an understanding of the patience levels of our parents and pets. In most cases, nobody sits next to us to explain what we see, hear, feel, taste, and smell. This is an example of what we call unsupervised learning.
In unsupervised learning, the training data is "unlabeled". Without our help or intervention, the algorithm/s (or program/s) will find the required connections or unsuspected patterns in the data and learn the details and properties of the dataset.
Later, as we grow up, we understand words and start naming things. Our parents tell us when we see a dog or a cat, we learn our names and theirs, and we learn to identify our toys from among other children's toys (and fight over them). Without even realizing it, we relate some characteristics of objects, animals, and people to their names. These are examples of what is known as supervised learning. In the case of a computer, the algorithm is shown as a set of variables that are representative of the properties of the problem and then it learns how these features relate to the name of the label.
Science has shown us the immense complexity of the world that surrounds us. Every branch of scientific knowledge needs advanced mathematical calculations and even completely new ways of looking at data. However, the vast majority of what we can explain is only a fraction of the real world. Whenever we describe a physical phenomenon, an economic or financial event, or try to understand the behavior of individuals and groups, we rely on simplified versions of the real problem. These are called models and they make it possible for us to build a mental representation of whatever we are trying to explain. If the model is accurate enough, we will be able to predict some future event, or get some approximate value for a certain outcome. As you should have realized by now, this is incredibly powerful. For example, if an artillery soldier is capable of calculating, with accurate precision, where the cannonball is going to hit, then his army has a clear advantage over the enemy in battle. A model is a simplified version of reality that is used to understand a problem and eventually make predictions. Understanding something that your opponent ignores always represents an advantage.