Let me take a crack at it. Your intended system usage dictates what you need to do.
VCO's determine their frequency with a resonator circuit. Usually some sort of L-C network, where the C is partially composed of a varactor diode. Where ever the reactance of the total L equals that of the total C, the VCO will oscillate.
If you have a nice resistive 50 ohm load, that is the end of the story. But if you have an unfriendly load, like a 0/180 degree switching phase modulator, then the varying load will somewhat load down the VCO's resonator circuit. A surprisingly little variation in this external reactance will cause a very big change in the VCO's operating frequency. Usually the phase locked loop does not have a big enough bandwidth to adequately supress this effect, and you end up with all sorts of phase errors, etc, in the communications link.
So, during your design of the vco, you determine how happy the vco will be with the load presented, and then usually conclude that you need a buffer amp with a lot of reverse isolation (s12) to keep the system happy.