Let y = Hx be your system equation where x is the input and y is the output.
If you have inv(H), then your system equation is easily solved by doing x = inv(H)y. However, we cannot have inv(H) directly in many of the real applications. That is where the pseudo inverse comes from.
Although you don't have inv(H), you can change y = Hx to H'y = H'Hx which in turn x = ((H'H)^-1)Hy. In deed, x is the famous 'least square approximation'. Therefore, to be precise, x_ls = ((H'H)^-1)Hy where x_ls is the least square solution and ((H'H)^-1)H is the pseudo inverse of H.