Weights and biases

In case of error in the simulation of the phenomenon, it's necessary to make an appropriate correction. To do this, each input must be assigned a weight—that is, a numerical value that modulates the impact that this input has on the total sum to determine the potential of the neuron. In other words, each input contributes more or less to determine the exceeding of the threshold value and the triggering of the potential.

The weight of a connection is a numerical value for which we multiply the value of the input. In this way, the input will have a greater or lesser effect on the total sum of the inputs according to the weight entity. The sum of the input values will now become the weighted sum of the input values. We can represent this process with the following equation:

Moreover, we can think of the input and the corresponding weight as vectors of the type INPUT = (Input₁, Input₂,.., Input_n) and W = (w₁, w₂,.., w_n). The total input signal to the neuron, what we have identified as the sum of the input values, will then be the scalar product of these two vectors (INPUT · W). Each component of the INPUT vector is multiplied with the corresponding component of the vector W and all of the obtained products are added together. The result will be a scalar representing the weighted sum of the input values, as shown in the following formula:

Geometrically, the scalar product of two vectors can be considered as a measure of their similarity. If the vectors have the same direction, the scalar product is maximum; if the vectors have opposite directions (180°), the scalar product is null. In the weighted sum, each entry participates in the sum proportionally to the weight entity.

Among the weights is a special weight named bias. It's not tied to any other unit of the network and it's as if it always had input equal to 1. The bias serves to make a translation on the axis of the abscissas to the output function. A new unitary input will be associated to the input values and an additional element b_i will be present in the weight vector, which will represent the weight referred to as the bias. On the basis of what has been said, the previous equation assumes the following form:

The effect of the bias is to control the translation of the activation threshold with respect to the origin of the signals. Formally, the bias plays a role similar to that of the weights that act as regulators of the intensity of the signal emitted (or received).