About

This package contains a Matlab implementation of the Fast Kernel ICA algorithm, as described in SheJegGre09, SheJegGre07.

The goal of ICA is to separate linearly mixed sources to minimise the statistical dependence between the estimated unmixed sources. Kernel ICA algorithms use kernel measures of statistical independence as their optimisation criteria (BacJor02, GreHerSmoBouSch05). Fast Kernlel ICA (FastKICA) employs an approximate Newton method to perfom the optimization efficiently for larger-scale problems.

The kernel independence criterion we use here is the Hilbert-Schmidt norm of the covariance operator in feature space (see GreBouSmoSch05). Another interpretation of this criterion is as a characteristic function based independence measure (Feu93), as used previously in ICA by CheBic05 and EriKoi03.

The functions 'chol_gauss' and 'amariD' are based on code from Francis Bach. The derivative is computed as described in JegGre07 (for incomplete Cholesky decomposition).

Code

The code may be downloaded here.

Unzipping the downloaded archive will create a directory called 'fastKICA', which should contain the following files:

fastkica.m main routine
README.txt Instructions on use
utils\chol_gauss.c Incomplete Cholesky decomposition (from Francis Bach).
utils\dChol2.c Mex code for derivative (Windows).
utils\dChol2Lin.c Mex code for derivative (Linux).
utils\Kmn.c, dKmnKin.c Mex code, also for derivative (Win/Linux).
utils\getKern.c Mex code to compute Gaussian kernel.
utils\compDerivChol.m Matlab interface for the gradient.
utils\dChol.m, dCholLin.m Needed for the gradient (Win/Linux).
utils\hsicChol.m Computes HSIC.
utils\hessChol.m Computes the Hessian.

Datasets

 The following data and code are also provided in the zip archive, for demonstration purposes.
demo.m Code for demo of kernel ICA.
amariD.m Amari divergence (based on the code by Francis Bach).
source2.wav Data file
source3.wav Data file
source4.wav Data file

References
[SheJegGre09] Shen, H., S. Jegelka and A. Gretton: Fast Kernel-Based Independent Component Analysis. IEEE Transactions on Signal Processing. download
[SheJegGre07] Shen, H., S. Jegelka and A. Gretton: Fast Kernel ICA using an Approximate Newton Method. AISTATS 2007. download
[BacJor02] Bach, F. and M. I. Jordan: Kernel Independent Component Analysis, Journal of Machine Learning Research 3, 1-48, 2002.
[GreHerSmoBouSch05] Gretton, A., R. Herbrich, A. Smola, O. Bousquet and B. Schoelkopf: Kernel Methods for Measuring Independence. Journal of Machine Learning Research 6, 2075-2129 (2005). download
[GreBouSmoSch05] Gretton, A., O. Bousquet, A. Smola and B. Schoelkopf: Measuring Statistical Dependence with Hilbert-Schmidt Norms. ALT 2005, 63-78. download
[Feu93] Feuerverger, A.: A Consistent Test for Bivariate Dependence, International Statistical Review 61(3), 419-433, 1993.
[CheBic05] Chen, A. and P. Bickel: Consistent independent component analysis and prewhitening, IEEE Trans. Signal Process. 53(10), 3625-3632, 2005.
[EriKoi03] Eriksson, J. and V. Koivunen: Characteristic-function-based independent component analysis, Signal Processing 83(10), 2195-2208, 2003
[JegGre07] Jegelka, S. and A. Gretton: Brisk Kernel ICA. In Large Scale Kernel Machines, MIT Press, Cambridge, Massachusetts (to appear, 2007)

Contact

arthur.gretton@gmail.com