lmx2326 filter loop
If you are using a 10 MHz reference at the phase detector, then you should expect some sidebands at +/- 10 MHz from the VCO carrier. This is because there is an small pulse coming outof the phase detector at a 10 MHz rate, and a small portion of that gets to the VCO tune line. You can try to lowpass or notch filter it out, but have to be careful to not lose any phase margin in the control loop.
IF you divide the reference clock down to some lower frequency before it hits the phase detector, then you would expect the sidebands at +/- that lower frequency.
As far as locking, I gave some bench test ideas for debugging in this thread: