ex7 pca

Author: Mark Linderman

ML Notes.md

@@ -507,11 +507,11 @@ Keep in mind that the goal is to translate a point in 2 dimensional space (x,y)
 
 So the procedure is that first, you compute the "covariance" matrix (called Sigma, which, confusingly, is not a sum operation though it looks a lot like one and is also *not* the standard deviation). And then use that matrix called Sigma to compute the "eigenvectors" of that matrix.  Say what?  He glossed over the details but computing eigenvectors is the same as a "single value decomposition".  You can do this in octave using the svd function.  There are equivalents in other languages (like Python, I'd assume).  The octave svd function, returns 3 vectors, U, S and V - or maybe that's a matrix...
 
-So, to get into the details, to compute the covariance matrix, you take each sample *column*, or feature (not rows this time), and multiply it by it's transposed version (producing an n x n matrix) and then add them all together which still results in an n x n matrix. (Don't forget: n is the feature index and m the samples index.)  Averaging that over the number of samples ($\frac{1}{m}$) just gives you a different n x n matrix.  That's $\Sigma$.    
+So, to get into the details, to compute the covariance matrix, you take each sample *column*, or feature (not rows this time), and multiply it by it's transposed version (producing an n x n matrix) and then add all the matrices that result (1 per feature)together.  The result is still an n x n matrix. (Don't forget: n is the feature index and m the samples index.)  Averaging that over the number of samples ($\frac{1}{m}$) just gives you a different n x n matrix.  That's $\Sigma$.    
 
-$$ \Sigma = \frac{1}{m}\sum_{1-1}^m(x^{(i)})(x^{(i)})^T $$
+$$ \Sigma = \frac{1}{m}\sum_{i-1}^m(x^{(i)})(x^{(i)})^T $$
 
-That's from Ng.  But it can't be right since he claims it'll result in an n x n matrix. But if $x^{(i)}$ is a single example, a row, as it's always been throughout the course, then this will result in 1 x 1 vector.  But it doesn't matter since the vectorized version below does look like it will return an n x n matrix.  So, moving on, the next step (in octave):
+That's from Ng.  But it can't be right since he claims it'll result in an n x n matrix.   But if $x^{(i)}$ is a single example, a row, as it's always been throughout the course, then this will result in 1 x 1 vector.  (I think he must mean to reverse those two terms, transposed one first.) It doesn't matter since the vectorized version below does look like it will return an n x n matrix.  So, moving on, the next step (in octave):
 
 $$ [U,S,V] = svd(\Sigma) $$
 

machine-learning-ex7/ex7/ex7.m

@@ -164,7 +164,7 @@
 % indices in idx. 
 
 % We can now recover the image from the indices (idx) by mapping each pixel
-% (specified by its index in idx) to the centroid value
+% (specified by its index in idx) to the centroid value (1 of 16 colors)
 X_recovered = centroids(idx,:);
 Y_recovered = centroidsB(idxB,:);
 

machine-learning-ex7/ex7/pca.m

@@ -20,10 +20,9 @@
 %       number of examples).
 %
 
-
-
-
-
+% calling it covariance here - also commonly known as Sigma
+covariance = (1/m) * transpose(X) * X;
+[U, S, _] = svd(covariance);
 
 
 % =========================================================================

machine-learning-ex7/ex7/projectData.m

@@ -18,7 +18,7 @@
 %                    projection_k = x' * U(:, k);
 %
 
-
+Z = X * U(:, K);
 
 
 % =============================================================

machine-learning-ex7/ex7/recoverData.m

@@ -21,7 +21,7 @@
 %               Notice that U(j, 1:K) is a row vector.
 %               
 
-
+X_rec = Z * transpose(U(:, K));
 
 % =============================================================
 

machine-learning-ex7/ex7/token.mat

@@ -1,4 +1,4 @@
-# Created by Octave 4.4.1, Fri Feb 15 21:20:49 2019 EST <[email protected]>
+# Created by Octave 4.4.1, Sun Feb 17 21:26:38 2019 EST <[email protected]>
 # name: email
 # type: sq_string
 # elements: 1