There are two kinds of SPDTs: one is simply a PIN device replacing a mechanical relay contact. When "open", such device presents a high reflective coefficient. Due to such mismatch, it is often not suitable in applications like yours.
The other SPDT switch is "matched", instead of "open" state it introduces a matched 50-Ohm termination. Such device I think is preferable for your application.
Low-noise amplifiers like MMICs have often their S22 not too good, some also S11. Then any imperfect match in connected components like a SPDT switch can cause even an oscillation.
In your case, I would suggest to sacrifice 3-6 dB of gain and use 3-6 dB attenuation pads in SPDT switch lines. I hope this will improve your system response.
There are also MMIC "switched-gain" amplifiers with a structure similar to your circuit, in which MMIC designers solved the mismatch problem on chip.