Gain and phase margin are quantities defined for the loop gain. In a negative feedback system you want to keep away from the point where the feedback becomes positive - that is your loop gain has a value of 1 or 0dB and your excess phase is 180deg. Since it is a negative feedback, you already have an inversion for the loop gain even at DC. With frequency, the loop adds extra delay and shifts the phase. If you accumulate more phase compared to the initial 180deg you start running into troubles. That additional phase accumulation is the excess phase. If it becomes 180deg, then your total phase shift along the loop becomes 360deg and your feedback becomes positive. If at that frequency you also have loop gain of 0dB, then you meet the conditions for oscillation.
So, you look at the frequency where the loop gain crosses 0db and check the phase for that same frequency. The phase margin is a measure of how far the phase is from the 360deg (or 0deg as some simulators show it). If PM is for example 90deg, then your closed loop amplifier behaves like a 1st order system with nice exp settling to a step. The lower you go with the PM, the more ringing you see in the step response. As a rule of thumb people take 45deg as min. PM, but you already have quite a bit of ringing for it. Now it depends on your application. For switch cap circuits you generally shoot for about PM=75deg. If you process sine signals, it is customary to aim at 60deg. For LDO, you may go to 45deg perhaps.
Gain margin is another metric of goodness for your loop.It says how much your loop gain has fallen below 0dB when the PM=0deg - in other words how far you are from the dangerous point of 0dB. Usually you want to have double digits GM.