The concept of loop gain comes when there is a feedback from output to input. Also stability is an issue for only systems with feedback. Any system without feedback like feedforward systems need not bother about stability. (at very high frequencies, all the gate drain parasitics act as feedback and you need to consider them)
When an opamp is built, it is assumed that a feedback loop will be used around it (unless used as a comparator). If the specific feedback factor is not known, it is designed for the worst case feedback factor of 1, i.e unity gain feedback.
Stability analysis needs to be done on the open loop amplifier, considering all the effects of the closed loop, i.e, the loop needs to be broken at some point and the loading conditions must not change when the loop is broken.
The two poles present in your amplifier will make the system go close to instability not because they are present but because when connected in feedback will create a situation where in the closed loop gain can go to infinity(when |Af|=1,phase=180deg). It is feedback which is causing instability.
In your case, do a bode plot analsyis on A(s), You will see the gain dropping according to the presence of the two poles. You have to ensure that the unity gain bandwidth is between the first and second poles, which ensures at least 45 deg phase margin.
Check out without using miller caps. You will see that the two poles occur before the unity gain frequency. Miller caps will split the two poles.