Hello,
Assuming that supply voltage and reference oscillator is spectrally clean, the phase comparator's output voltage is determined by the phase noise of the VCO and the df/dVvco ratio. You should ignore the out of band very narrow spikes that are inherent to phase/frequency comparators.
If your VCO has large phase noise within the bandwidth of the PLL, the phase comparator will correct the phase difference because of the phase noise, hence you get a time varying VCO control voltage. A low df/dVvco ratio will result in more control voltage "noise".
If you have some means to measure the spectrum of the VCO, you may measure the VCO alone (with a constant (DC) control voltage). When you close the loop, the phase noise should reduce, hence the PLL is doing its job.
You may measure the spectrum of the control voltage (FFT on oscilloscope) to check for any loop instability. Make sure the narrow spikes are removed before FFT to avoid aliasing.