The most famous free codes that I know is "GSL" - The GNU Scientific Library. The unix version is free but the windows version starts charging (if you get on emule, you may still get free windows codes). I have used both of them and they are really great. GSL can solve a lot of mathematics problems and it certainly has a matrix solver. All matrix functions start with gsl_matrix.
When you start using it, you better find some examples first as it took me some time to get used to it.