Fast Kernel ICA using an Approximate Newton Method
by Hao Shen, Stefanie Jegelka, and Arthur Gretton
The goal of ICA is to separate linearly mixed sources to minimise the statistical dependence between the estimated unmixed sources. Kernel ICA algorithms use kernel measures of statistical independence as their optimisation criteria (BacJor02, GreHerSmoBouSch05). Fast Kernlel ICA (FastKICA) employs an approximate Newton method to perfom the optimization efficiently for larger-scale problems.
The kernel independence criterion we use here is the Hilbert-Schmidt norm of the covariance operator in feature space (see GreBouSmoSch05). Another interpretation of this criterion is as a characteristic function based independence measure (Feu93), as used previously in ICA by CheBic05 and EriKoi03.
The functions 'chol_gauss' and 'amariD' are based on code from
The derivative is computed as described in JegGre07 (for incomplete Cholesky
The code may be downloaded here.
Unzipping the downloaded archive will create a directory called 'fastKICA', which should contain the following files:
|README.txt||Instructions on use|
|utils\chol_gauss.c||Incomplete Cholesky decomposition (from Francis Bach).|
|utils\dChol2.c||Mex code for derivative (Windows).|
|utils\dChol2Lin.c||Mex code for derivative (Linux).|
|utils\Kmn.c, dKmnKin.c||Mex code, also for derivative (Win/Linux).|
|utils\getKern.c||Mex code to compute Gaussian kernel.|
|utils\compDerivChol.m||Matlab interface for the gradient.|
|utils\dChol.m, dCholLin.m||Needed for the gradient (Win/Linux).|
|utils\hessChol.m||Computes the Hessian.|
|demo.m||Code for demo of kernel ICA.|
|amariD.m||Amari divergence (based on the code by Francis Bach).|
|[SheJegGre09]||Shen, H., S. Jegelka and A. Gretton: Fast Kernel-Based Independent Component Analysis. IEEE Transactions on Signal Processing. download||[SheJegGre07]||Shen, H., S. Jegelka and A. Gretton: Fast Kernel ICA using an Approximate Newton Method. AISTATS 2007. download||[BacJor02]||Bach, F. and M. I. Jordan: Kernel Independent Component Analysis, Journal of Machine Learning Research 3, 1-48, 2002.||[GreHerSmoBouSch05]||Gretton, A., R. Herbrich, A. Smola, O. Bousquet and B. Schoelkopf: Kernel Methods for Measuring Independence. Journal of Machine Learning Research 6, 2075-2129 (2005). download||[GreBouSmoSch05]||Gretton, A., O. Bousquet, A. Smola and B. Schoelkopf: Measuring Statistical Dependence with Hilbert-Schmidt Norms. ALT 2005, 63-78. download||[Feu93]||Feuerverger, A.: A Consistent Test for Bivariate Dependence, International Statistical Review 61(3), 419-433, 1993.||[CheBic05]||Chen, A. and P. Bickel: Consistent independent component analysis and prewhitening, IEEE Trans. Signal Process. 53(10), 3625-3632, 2005.||[EriKoi03]||Eriksson, J. and V. Koivunen: Characteristic-function-based independent component analysis, Signal Processing 83(10), 2195-2208, 2003||[JegGre07]||Jegelka, S. and A. Gretton: Brisk Kernel ICA. In Large Scale Kernel Machines, MIT Press, Cambridge, Massachusetts (to appear, 2007)|