Utilisateur:Arbg0002/brouillons/réseaux de neurones
Neural networks
Pre-processing
- Data exploration (clean-up)
- Data transformation
- Outlier detection and removal
- Data normalization, one of:
- Min-Max normalization: linear scaling into a data range, typically [0,1]
- Zscore normalization: input variable data is converted into zero mean and unit variance
- Sigmoidal normalization: nonlinear transformation of the input data into the range [-1,1], with a sigmoid function
- Other normalization: e.g. if a variable is exponentially distributed, take the logarithm
- Data analysis
- Validation of results
Source: [1]
Global behavior
The global behavior of each layer of a neural network is to produce a linear combination of the weights its nodes produced. These weights model the connections between neurons, using an activation function (as detailled in the subsection below). Then all incoming weights to a node are summed, hence resulting in a linear combination of its inputs, through the propagation function (as detailled in the subsection below). Often, positive contributions are labelled "excitatory" and negative ones, "inhibitory".
Activation function
The activation function is alike <math>\phi(v_i)=U(v_i)</math>, where <math>U</math> is the Heaviside step function.
- normalizable sigmoid activation function: <math>\phi(v_i)=U(v_i)\tanh(v_i)</math>, where the hyperbolic tangent function can be replaced by a sigmoid function
- gaussian function: <math>\phi(v_i)=\exp\left(-\frac{\|v_i-c_i\|^2}{2\sigma^2}\right)</math>
- multiquadratic function: <math>\phi(v_i)=\sqrt{\|v_i-c_i\|^2 + a^2}</math>
- inverse multiquadratic function: <math>\phi(v_i)=(\|v_i-c_i\|^2 + a^2)^{-1/2}</math> where <math>c_i</math> is the vector representing the function "center" and <math>a</math> and <math>\sigma</math> are parameters controlling the spread of the radius
More on wikipedia: [2] - permanent link: [3]
Propagation function
The input <math>p_j(t)</math> to the neuron <math>j</math> from the outputs <math>o_i(t)</math> of the connected neurons in the preceeding layer can be expressed as
- <math> p_j(t) = \sum_{i} o_i(t) w_{ij}+ w_{0j} </math> , where <math>w_{0j}</math> is a bias.