Can somebody please explain me what is the physical significance of two random variables being uncorrelated. Mathematically, it means their covariance is zero, but what does this tell about the behavior or joint pdf of the two variables. I am confused since I had the impression that uncorrelatedness is the same as statistical independence, but now I realized that uncorrelatedness is just a special case of statistical independence.
Statistical independence is much stronger that uncorrelatedness. Independence implies uncorrelatedness, but the converse is not true in general. Nevertheless, for gaussian R.V.s, uncorrelatedness implies independence.
An example: suppose a R.V. X uniformly distributed in [-1,1]. Define the R.V. Y=X^2 .
X and Y are uncorrelated according to the definition. But they are clearly not independent.
Correlatedness/Uncorrelatedness doesn't tell much about the joint pdf of the two variables (except for gaussian RVs). It has to do only with moments up to order 2.