Well, I hope you realize that the 32 MHz signal, after is is divided by 32, will "look" like it has lower jitter, but it is exactly the same jitter just divided by 32! Conversely, your master clock at 1 MHz will have to be much better than 32 times as good as the 32 MHz oscillator for you to be able to lock the two together and get any improvement.
Another way of saying this is if your 32 MHz oscillator has a phase noise of -120 dBc/Hz at 1 KHz offset, to improve this by locking to a 1 MHz oscillator, that 1 MHz oscillator had better have phase noise MUCH better than -150 dBc/Hz at 1 KHz offet. This is because the pll will act as a frequency multiplier X32, which is a 30.1 dB degradation of phase noise (20 * LOG 32 = 30.1 dB).
So, IF your 1 MHz oscillator is indeed much more than 30 dB better in phase noise than your 32 MHz oscillator, you can try to improve things by phase locking the two together. Then you have to play around with the PLL control loop bandwidths to get things just right.
You sould try to add a big capacitor, like a 47 uF tantalum, right at the 32 MHz VCXO's Vdd pin, and see what happens.
A common use of the PLL and its bandwidth is to improve the noise of a VCO close to the carrier (perhaps within +/- 20 KHz of the carrier frequency) by locking the higher frequency VCO to a very stable low frequency XTAL oscillator. But at higher offset frequencies, where the VCO's free running phase noise will be better than the XTAL's phase noise when degraded by the 20LogN multiplication effect, you close out the PLL control bandwidth and let the VCO run free. So, at typically 200 KHz, your PLL is having no effect on the VCO any more.