Independent Component Analysis

Videos

  1. Stanford ML
    1. ICA, SEE
  2. GaTech ML
    1. ICA 1, Udacity
    2. ICA 2, Udacity
    3. ICA Example, Udacity
    4. Matrix, Udacity
    5. PCA vs ICA, Udacity
  3. Blind Source Separation ICA with Python
    1. Sklearn, Shogun install
    2. FastICA with Sklearn
    3. ICA Jade Algo with Shogun, ipynb
    4. Applying Jade to Images, ipynb
  4. Videolectures.net
    1. Intro to ICA by Aapo Hyvärinen, University of Helsinki (2005) 5:44:22
    2. ICA by Jean-François Cardoso (2003) 5:35:17
  5. Optimization 2012 CMU
    1. Lectures 10, 11
  6. Youtube playlist
  7. ICA in FSL playlist

Notes

  1. Wikipedia
  2. Andrew Ng (Stanford) ML Notes
  3. Razvan Bunescu (Ohio) Notes
  4. Blind Source Separation (MIT) Notes
  5. Tutorial by Jonathon Shlens
  6. Independent Component Analysis: A Tutorial by Aapo Hyvärinen and Erkki Oja (pdf)
  7. ICA at NLPCA

Implementations

  1. Sklearn – FastICA, python code
    1. Blind Source Separation using ICA: Example, code, ipynb, ipynb-noise
    2. FastICA on 2D Point Clouds: Example, code, ipynb
    3. Faces dataset decompositions: Example, code, ipynb
    4. Decomposing Signals in components (Matrix Factorization problems)
  2. Sklearn – Learning Gabor Filters: page, gist, ipynb
  3. MLPack ICA C++ (Radical) usage, class ref, source
  4. IT++ ICA (C++)
  5. Cuda: BSS page
  6. MDP Toolkit (python)
  7. libICA (C++)
  8. Fast ICA (C++): github
  9. Matlab
    1. ICA:DTU Toolbox
    2. Medical Image Analysis Lab
    3. Neurophysiological Biomarker Toolbox
    4. PCA and ICA package
    5. FastICA Toolbox
    6. cICA by E. Bingham (code, paper, python port)
    7. EEGLab Toolbox
    8. ICALab Gui Toolbox
    9. kICA – 2 lines of code
    10. cICA and ICAtoolbox page (2 and 4)
    11. JADE (complex, demo, real, page, python real port or here)

Whitening

  1. Wikipedia
  2. Stanford Learning

Books to buy

  1. Book by A. Hyvärinen, J. Karhunen, E. Oja (Group ICA page, Fast ICA page) (online pdf, Amazon)
  2. Book: Independent Component Analysis: A Tutorial Introduction by James Stone (code) (Amazon)

Align two 3D vectors using Euler Angles ($\alpha_x,\alpha_y,\alpha_z$)

Assume there are two 3D vectors $p$ and $q$ and $p$ needs to be aligned (point in the same direction as) to $q$.

First, let’s extract the axis-angle representation $k$ and $\theta$:

$
\begin{eqnarray}
k &=& p \times q \\
k &=& \frac{k}{||k||} \\
p &=& \frac{p}{||p||} \\
q &=& \frac{q}{||q||} \\
\theta &=& cos^{-1}(p \cdot q)
\end{eqnarray} $

Convert axis-angle ($k$ and $\theta$) to a matrix $R$

$\begin{eqnarray}
K &=&
\left[ \begin{array}{ccc}
0 & k(3) & k(2) \\
k(3) & 0 & -k(1) \\
-k(2) & k(1) & 0
\end{array} \right] \\
R &=& e^{\theta K} \\
R &=& I +sin(\theta)K + (1-cos(\theta))K^2
\end{eqnarray} $

Convert matrix $R$ to Euler angles ($\alpha_x,\alpha_y,\alpha_z$)

$\begin{eqnarray}
\alpha_x &=& atan2(R(3,2), R(3,3)) \\
\alpha_y &=& atan2(-R(3,1), \sqrt{R(3,2)^2 + R(3,3)^2}) \\
\alpha_z &=& atan2(R(2,1), R(1,1))
\end{eqnarray} $

Verify Results

$\begin{eqnarray}
M_x &=&
\left[ \begin{array}{ccc}
1 & 0 & 0 \\
0 & cos(\alpha_x) & -sin(\alpha_x) \\
0 & sin(\alpha_x) & cos(\alpha_x)
\end{array} \right] \\
M_y &=&
\left[ \begin{array}{ccc}
cos(\alpha_y) & 0 & sin(\alpha_y) \\
0 & 1 & 0 \\
-sin(\alpha_y) & 0 & cos(\alpha_y)
\end{array} \right] \\
M_z &=&
\left[ \begin{array}{ccc}
cos(\alpha_z) & -sin(\alpha_z) & 0 \\
sin(\alpha_z) & cos(\alpha_z) & 0 \\
0 & 0 & 1
\end{array} \right] \\
p_{rot} &=& M_zM_yM_xp \\
p_{rot} &=& \frac{p_{rot}}{||p_{rot}||} \\
q &=& \frac{q}{||q||} \\
metric &=& atan2(||p_{rot} \times q||, p_{rot} \cdot q)
\end{eqnarray} $

Metric should be zero.