floating ground circuit
I don't think it's a good idea to supply the inverter ground from a current source.
You didn't tell, if you are designing the circuit from discrete parts or targetting to IC design. Usual logic IC don't allow an input voltage below ground, so an analog SPDT switch IC (xx3157 or similar) would be a suitable solution for the inverter. Floating ground voltage should have a bypass cap or must be driven by a low impedance buffer (a voltage rather than the current source in your circuit).
If it's an IC design problem, you have all options of connecting individual transistors. But the low impedance point is still valid to my opinion.