Well, you can write books about this subject.
The main point is that with two antennas spaced more than x wavelengths apart, the probability that either one of them momentarily has a better signal is higher as you say yourself. The "gain" can be expressed in BER, or in some kind of equivalent dB figure that would result in the same BER as the diversity system provides.
Diversity is more than just the antenna, one should include the receiver system too.
Principally it is possible to combine or do diversity switching in RF or somewhere else in the analog path, but by far the most straightforward, flexible and best method is to have totally seperate receiver branches and combine in the digital domain.
If you have eg. 3 receiving antennas with each their own receiver and demodulator, and sum up the received signal, you effectively reduce the apparent noise level. I know it can all be proven with mathematics and probabilities, but still I think it is kind of magic.
With two antennas you can have a reduction in BER comparable to a signal increase of 5-7 dB, and with three 6-9 dB depending on environment, frequency, path loss, modulation method and a couple of other parameters.
Switching antennas is not a good idea and complicated too, because also switching causes bit errors.
Bert