The main symbols and abbreviations used throughout the text are listed as follows.
absolute value of a real number
expectation value of a random variable
first-order derivative of the function
second-order derivative of the function
gradient of the function with respect to
vector or matrix transposition
-dimensional real Euclidean space
correlation coefficient between random variables and
Gaussian distribution with mean vector and covariance matrix
uniform distribution over interval
chi-squared distribution with degree of freedom
Shannon entropy of random variable
-order Renyi entropy of random variable
-order information potential of random variable
survival information potential of random variable
-entropy of discrete random variable
mutual information between random variables and
KL-divergence between random variables and
-divergence between random variables and
Fisher information rate matrix
kernel function for density estimation
Gaussian kernel function with width
reproducing kernel Hilbert space induced by Mercer kernel
feature space induced by Mercer kernel
weight vector in feature space
NLMS normalized least mean square
MLE maximum likelihood estimation
FLOM fractional lower order moment
ADALINE adaptive linear neuron
RKHS reproducing kernel Hilbert space
KAPA kernel affine projection algorithm
KMEE kernel minimum error entropy
KMC kernel maximum correntropy
PDF probability density function
GGD generalized Gaussian density
DPI data processing inequality
MCC maximum correntropy criterion
QIP quadratic information potential
CRE cumulative residual entropy
SIP survival information potential
QSIP survival quadratic information potential
KLID Kullback–Leibler information divergence
EDC Euclidean distance criterion
MinMI minimum mutual information
MaxMI maximum mutual information
AIC Akaike’s information criterion
BIC Bayesian information criterion
MDL minimum description length
FIRM Fisher information rate matrix
MIH minimum identifiable horizon
ITL information theoretic learning
BIG batch information gradient
FRIG forgetting recursive information gradient
SIG stochastic information gradient
SIDG stochastic information divergence gradient
SMIG stochastic mutual information gradient
FP-MEE fixed-point minimum error entropy
RFP-MEE recursive fixed-point minimum error entropy
EDA estimation of distribution algorithm