The technique of principal component analysis enables us to create and use a reduced set of variables, which are called principal factors.  A reduced set is much easier to analyze and interpret. To study a data set that results in the estimation of roughly 500 parameters may be difficult, but if we could reduce these to 5 it would certainly make our day. We will show in what follows how to achieve this reduction. 

While these principal factors represent  or replace one or more of the original variables, it should be noted that they are not just a one to one transformation, so inverse transformations are not possible.  To shed a light on the structure of principal components analysis, let us consider a multivariate sample variable, X. It is a matrix with n rows and p columns. The p elements of each row are scores or measurements on a subject, such as height, weight and age. 

Next standardize the X matrix so that each column mean is 0 and each column variance is 1. Call this matrix Z. Each column is a vector variable, z_i,  i = 1, . . . , p. The main idea behind principal component analysis is to derive a linear function y for each of the vector variables z_i,  This linear function possesses an extremely important property, namely,  its variance is maximized. 

This linear function is referred to as a component of z. It is specified algebraically by y =V'z , where V is a p x n coefficient matrix that carries the p-element variable z into the derived n-element variable y. 

The mean of y is m_y = V'm_z =  0, because m_z = 0. 

The dispersion of y is 

                                        D_y = V'D_zV = V'RV 

Now, it can be shown that  the dispersion D_z  of a standardized variable is a correlation matrix. Thus R is the correlation matrix for z. 

At this junction you may be tempted to say: "so what?". To answer this let us look at the intercorrelations among the elements of a vector variable.  The number of parameters to be estimated for a p-element variable is 

          p means 
          p variances 
          (p² - p)/2 covariances 
          for a total of 2p + (p²-p)/2 parameters. 

        If p = 2 there are 5 parameters 
        If p = 10, there are 65 parameters 
        If p = 30 there are 495 parameters 

All these parameters must be estimated and interpreted. That is a herculean task, to say the least. Now, if we could transform the data so that we obtain an uncorrelated vector variable, life becomes much more bearable, since there are no covariances.