The problem is hiss sound in IF output of balanced mixer. It comes from one or both IQ channels. While one channel can be very clear, other one is hissing. Changing diodes does not help. Can it be because of wong sizes of microstrip mixer elements?
Here is about LO leakage of the mixer. Single-Balance mixers has not very good LO to other ports isolation. Their isolation depends very much by return loss at each port (RF, LO, IF).
Usually the RF port of single-balanced mixers requires not only good return loss at RF frequency, but at LO frequency also, otherwise the circuit balance can be affected. The same is valid for LO port return loss at RF frequency.