There is no problem in your design, I can judge both 5V and 3V have enough phase margin just from your gain plot. You can give a step input to verify the circuit stability.
There's no problem in your circuit, I think you can try another tool to simulate AC respose. Maybe it's just a bug in your tool. And you can try other different Vdd in your tool.
ok, i see.
The OPA circuit stucture is just normal, and should be OK. Some suggestions:
1. Check the DC operating point of the circuit when Vdd=3V and Vdd = 5V, to see if there is any difference, pay attention to the floating bias and the output MOSFET gate voltage. But if the circuit can work under 3V, 5V should be OK. just try.
2. Try to disconnect the load, and simulate to see if it is OK for Vdd=5V.
I think the circuit structure is OK, but the W/L size of the transistors might be with problems. Would like to know how you determine the W/L size? If the floating bias circuit was designed for 3V application, then there might be problem under 5V, because there should be cretain relationship between the foating bias circuit and the output MOSFET. If i am wrong, just point out.
I guess the problem comes from either the constant Gm block or the biasing of the cascodes. I've been using this architecture for a while and with no much effor you reach gains of more than 100dB. The gains shown in the figures are a poor 70dB.
Please upload a schematic showing all transistors. Also indicate the common mode input voltage. Maybe the circuit ois not operating in its linear region. I'm prety sure the problem comes from some bad biasing.
From the gain plot of 5V, I can calculate the phase margin.
PM=180-90-arctg(0.1)+arctg(0.01)=84.6
arctg(0.1) for the contribution of second pole, arctg(0.01) for zero.
I think the poor gain is because the cascode transistor works in linear region, in 5V condition. Because both 3V and 5V have the same GBW, but their -3dB frequency is different. So I guess the main pole small signal resistor is reduced when the circuit works in 5V condition in your current model, my suggest is you should check your bias circuit, there should be some problem in it. But if you don't care the gain, the circuit will work stably for the enough phase margin.