The capacitor just stops you wasting DC current so it is better to leave it in place. Some Gunn diodes exhibit strange characteristics as you change the voltage across them, their impedance changes and it can upset the circuit driving them. All the resistor and capacitor do is help to keep the impedance more constant.
I see you are using an ERA-3 MMIC as an output amplifier. Remember that the output from the mixer diode is the difference between Gunn frequency and a signal arriving from outside the waveguide so the frequency is generally quite low. When used for proximity detection using the Doppler shift, the frequency is usually no more than a few Hz. If you are using this as a transceiver, the frequency will be the difference between your Gunn oscillation and the frequency from the transmitter sending to you so it's likely to be no more than a few MHz. For example, if your oscillator is at 10.525GHz and the other source is at 10.550GHz, the output frequency will only be 25MHz.
Brian.