GPUs have a fixed computational/programming model
FPGAs do not, and their logic can be customized exactly for an application's needs.
FPGAs require (usually) some hardware design experience
GPUs does not
GPU is really a "software acceleration" that can accelerate certain class of compute-intensive applications. The only interface to the GPU (nVidia or AMD) is PCI Express.
FPGA, on the other hand, can be used for a broader range of accelerations. It has customizable IOs, so it can interface with any chip (with compatible signal levels, speed, number of IOs)
You can compare:
Xilinx highest-end FPGAs, XC6VLX760(~$20k) OR XC6VSX475T(~$10k)
Nvidia TESLA C2070 (~$3k) Or Tesla M2090(~$3k)