Neural networks

Pre-processing

Data exploration (clean-up)
Data transformation
Outlier detection and removal
Data normalization, one of:
- Min-Max normalization: linear scaling into a data range, typically [0,1]
- Zscore normalization: input variable data is converted into zero mean and unit variance
- Sigmoidal normalization: nonlinear transformation of the input data into the range [-1,1], with a sigmoid function
- Other normalization: e.g. if a variable is exponentially distributed, take the logarithm
Data analysis
Validation of results

Source: [1]

The activation function is alike <math>\phi(v_i)=U(v_i)</math>, where <math>U</math> is the Heaviside step function.

normalizable sigmoid activation function <math>\phi(v_i)=U(v_i)\tanh(v_i)</math>, where the hyperbolic tangent function can be replaced by a sigmoid function
gaussian function: <math>\,\phi(v_i)=\exp\left(-\frac{\|v_i-c_i\|^2}{2\sigma^2}\right)</math>
multiquadratic function: <math>\,\phi(v_i)=\sqrt{\|v_i-c_i\|^2 + a^2}</math>
inverse multiquadratic function: <math>\,\phi(v_i)=(\|v_i-c_i\|^2 + a^2)^{-1/2}</math> where <math>c_i</math> is the vector representing the function "center" and <math>a</math> and <math>\sigma</math> are parameters affecting the spread of the radius

Download: [2]