measuring vco phase noise
This is a common problem--you design a PLL and it does not work. The solution can be fairly complex, though, as any number of things may be going wrong. Here are a few ideas about how to get started:
* You know what frequency it should be at (you programmed the divisors and know the reference frequency). What is the corresponding VCO tune voltage that goes with that one frequency? Say it is +2.5 Volts. Hook up a voltmeter to the tune line and see what the voltage is. If it is near 0 volts or near 5 volts, obviously it is not even locked. Check to see if the chip is programmed right, if the correct pwer supply voltages are present to drive the chip and VCO. Try programming the chip with a wide range of different divisors, and see if any of them lock up (some divisors are illegal in some chips).
* If the voltage is approximately the right tuning voltage, then MAYBE it is locked (or trying to lock). Change the reference frequency slightly up or down (i.e. replace the reference clock with a tunable synthesizer). Does the VCO track up and down too? If so, you either have a high noise source (reference clock is not good enough, voltage regulator is oscillating, op amp is oscillating, etc), OR you have an unstable loop. Try other loop parameters for the loop filter, typically narrower bandwidth. If you can see a very distinct sine wave (in the 10 KHz to 5 MHz range) on an oscilloscope hooked up to the VCO tune line, that is a pretty good indication that the loop is unstable and oscillating.
* If you can get the thing to lock up with any of the above, do the math to figure out if it is at the right frequency. If you are dividing the VCO by 1000, and dividing the clock by 10, then your VCO should be 100.00000 times the clock frequency. If it is 100.013, then it is misprogrammed. It is easy to get the divisor ratios off by one bit.
* After you have the thing at least locked up and tracking the clock at the right frequency, then you can play around trying to get things better. You can try a lab standard crystal oscillator in place of the clock you are using to see if the noise gets better. You can try different phase detector frequencies and divisior ratios, and different loop bandwidths, trying to get it to have better phase noise.
* One trick to measure stability would be to inject a known disturbance, and see how the loop reacts to it. AC coulple in a square wave of say 50 mV onto the VCO tune line. Hook an oscilloscope probe to the tune line and see how the loop responds to this step disturbance. You want to see a step in voltage, followed by an exponential ramp back to the correct VCO tune voltage, with little or no overshoot. If it rings for many cycles, you have a marginally stable loop--fix your loop filter accordingly.