pca - How to implement ZCA Whitening? Python -

im trying implement zca whitening , found articles it, bit confusing.. can shine light me?

any tip or appreciated!

here articles read :

http://courses.media.mit.edu/2010fall/mas622j/whiten.pdf http://bbabenko.tumblr.com/post/86756017649/learning-low-level-vision-feautres-in-10-lines-of

i tried several things of them didnt understand , got locked @ step. right have base start again :

dtype = np.float32 data = np.loadtxt("../inputdata/train.csv", dtype=dtype, delimiter=',', skiprows=1) img = ((data[1,1:]).reshape((28,28)).astype('uint8')*255) 

as pointed out in r.m's comment, andfoy's zca whitening function contains small, crucial mistake: np.diag(s) should removed. numpy returns s m x 1 vector , not m x m matrix (as common other svd implementations, e.g. matlab). hence zcamatrix variable becomes m x 1 vector , not m x m matrix should (when input m x n). (also, covariance matrix in andfoy's answer valid if x pre-centered, i.e mean 0).

here python function generating zca whitening matrix:

def zca_whitening_matrix(x):     """     function compute zca whitening matrix (aka mahalanobis whitening).     input:  x: [m x n] matrix.         rows: variables         columns: observations     output: zcamatrix: [m x m] matrix     """     # covariance matrix [column-wise variables]: sigma = (x-mu)' * (x-mu) / n     sigma = np.cov(x, rowvar=true) # [m x m]     # singular value decomposition. x = u * np.diag(s) * v     u,s,v = np.linalg.svd(sigma)         # u: [m x m] eigenvectors of sigma.         # s: [m x 1] eigenvalues of sigma.         # v: [m x m] transpose of u     # whitening constant: prevents division 0     epsilon = 1e-5     # zca whitening matrix: u * lambda * u'     zcamatrix = np.dot(u, np.dot(np.diag(1.0/np.sqrt(s + epsilon)), u.t)) # [m x m]     return zcamatrix 

and example of usage:

x = np.array([[0, 2, 2], [1, 1, 0], [2, 0, 1], [1, 3, 5], [10, 10, 10] ]) # input: x [5 x 3] matrix zcamatrix = zca_whitening_matrix(x) # zcamatrix zcamatrix # [5 x 5] matrix xzcamatrix = np.dot(zcamatrix, x) # project x onto zcamatrix xzcamatrix # [5 x 3] matrix 

hope helps!

nb: can see full answer, in python, stanford ufldl zca whitening exercise here.