Hello Julew,
I assume your spectrum analyzer with S11 adapter will only measure the modulus of S11.
About conjugate match. A power amplifier is designed to give a certain amount of power into a predefined load (for example 50 Ohms). This does not mean that the output impedance of the amplifier will be (around) 50 Ohms.
A PA design based for maximum gain will operate under conjugate match condition, but a design for better efficiency may show complete other output impedance. When using "tuning for maximum output" this will very likely result in more power, but reduced efficiency and not meaningful result.
So I think you should measure the output impedance with 50 Ohms load. As you don't have means to measure phase, you can use a slotted line to measure output impedance.
You need an attenuator and two signal sources. The attenuator is connected to the slotted line and the output of the amplifier goes to the other input of the slotted line. Source 1 you use to drive the power amplifier. The attenuator is to protect source 2 against excessive reverse power.
Source 2 you connect to the attenuator and is tuned about 100 kHz to 1 MHz away from frequency of source 1. Adjust Source 2 so that about 0.1% of the PA's output power is sent towards the PA. So if your PA gives 100mW, you need to send about 0.1 mW towards the PA.
Now connect the spectrum analyzer to the sliding voltage probe and tune the analyser to the frequency of source 2. Make sure that the peak from the PA itself doesn't saturate the spectrum analyser. Note that you may see side bands that are generated by the PA. If these sideband are above -15 dB (w.r.t. carrier of source 2), reduce power of source 2.
When you move along the slotted line, you may notice the VSWR pattern along the line. From this pattern you can calculate the complex output impedance of the PA. You need reasonable knowledge of cable transmission theory (as you also have to calibrate your setup), and it is time consuming, but this method can give reasonable results, Without the need for a VNA or VSA.
A microstrip line (or ground plane backed CPW) you can use as slotted line. The probe can be just a capacitive pick-up wire that probes the E-field along the microstrip. The CPW has the advantage of having a ground present on the same side as the signal carrying trace.