Right, unlike modern DSOs that have X-Y mode selectable under horizontal settings, 54520A has it under math functions.
Your table is correct and the conversion between volt and dBm is valid only for a 50 ohm terminated system. The measurement with the scope in high impedance you should have the voltage doubled with respect to the scope terminated on the same impedance of that of the generator. If your generator is 50 ohm impedance and you see 1 Vpp with the scope terminated onto 1 Mohm then you will see 50 ohm/(1Mohm+50 ohm)*1Vpp≈0.5 Vpp.
Could you post the values you have read: Vpp on the scope and dBm from FFT ?
No, 3 dB difference seems to me too much. In case of miscalibration I would expect a difference between generator setting and scope measurement, but good agreement between trace and FFT.
Could you say what about the frequency of your signal, the RBW of the FFT, the horizontal set of the scope (total acquisition time and sampling time) and the windowing type (flat top, Blackmann Harris, etc.)