Statistical Signal Processing

Basic concepts of statistical decision theory; Bayes decision theory; HMMs and their state estimation (Forward--backward), Kalman as special case, approximate state estimation (particle filtering, Extended Kalman Filter), unknown parameters; Inference under logarithmic loss, mutual information as a fundamental measure of statistical relevance, properties of mutual information: data processing, chain rules. Directed information. Prediction under logarithmic loss; Context Tree Weighting algorithm; Sequential decision making in general: prediction under general loss functions, causal estimation, estimation of directed information. Non-sequential inference via sequential probability assignments. Universal denoising; Denoising from a decision theoretic perspective: nonparametric function estimation, wavelet shrinkage, density estimation; Estimation of mutual information on large alphabets with applications such as boosting the Chow-Liu algorithm. Estimation of the total variation distance, estimate the fundamental limit is easier than to achieve the fundamental limit; Peetre's K-functional and bias analysis: bias correction using jackknife, bootstrap, and Taylor series; Nonparametric functional estimation. Prerequisites: Familiarity with probability theory and linear algebra at the undergraduate level.