What is a neural network?

It is extremely hard to draw a parallel between neural networks and any other existing algorithmic mannerism for problem-solving that we have thus far. Linear regression, for example, simply deals with calculating a line of best fit with respect to the mean of squared errors from plotted observation points. Similarly, centroid clustering just recursively separates data by calculating ideal distances between similar points iteratively until it reaches an asymptotic configuration.

Neural networks, on the other hand, are not that easily explicable, and there are many reasons for this. One way of looking at this is that a neural network is an algorithm that itself is composed of different algorithms, performing smaller local calculations as data propagates through it. This definition of neural networks presented here is, of course, not complete. We will iteratively improve it throughout this book, as we go over more complex notions and neural network architectures. Yet, for now, we may well begin with a layman's definition: a neural network is a mechanism that automatically learns associations between the inputs you feed it (such as images) and the outputs you are interested in (that is, whether an image has a dog, a cat, or an attack helicopter).

So, now we have a rudimentary idea of what a neural network is—a mechanism that takes inputs and learns associations to predict some outputs. This versatile mechanism is, of course, not limited to being fed images only. Indeed, such networks are equally capable of taking inputs such as some text or recorded audio, and guessing whether it is looking at Shakespeare's Hamlet, or listening to Billie Jean, respectively. But how could such a mechanism compensate for the variety of data, in both form and size, while still producing relevant results? To understand this, many academics find it useful to examine how nature can solve this problem. In fact, the millions of years of evolution that occurred on our planet, through genetic mutations and environmental conditions, has produced something quite similar. Better yet, nature has even equipped each of us with a version of this universal function approximator, right between our two ears! We speak, of course, of the human brain.