Hi,
The images are of simulations I did to see how that P-Channel MOSFET might work for you - looks good r.e. the 50mV limit.
The simulations use LV devices as that's what's available in the simulator, but the LMV339 is the same as the LM339 except the LM339 operates from 2 to 30V, so that's not an issue. I was only able to emulate battery 1 (3.2 to 4.2V) because the LMV339 is only up to 5V supply. TLV431 is 1.24V reference, in comparison TL431 is 2.5V reference; there are plenty of ~1.25V references out there. As mentioned, LM339 is 1 package with 4 comparators.
FDD4141 cost around 1USD/1GBP/€1.50 each, whatever currency is in your country. But... smallest amount I saw was a pack of 10. They only seem to come in DPAK-3 (SMD), but that's easy to solder.
I haven't simulated this with the batteries in series, so guess the resistor dividers would need to be changed, perhaps the whole of that side of the circuit but I was just trying to give an idea of one way to do what I think I have understood you are trying to do.
One nuisance may be having to get a load of resistors you may only use once, maybe the relays are a simpler choice, however, the MOSFETs will last longer, worthwhile bearing in mind.
- - - Updated - - -
If what you want to do is this:
It won't work. The MOSFETs need a reference value of their own for the gate voltage to work, which comes from Source to Drain voltage. These will all have different voltages so either the top one or the bottom MOSFET will not turn off when sent the same buzzer signal as the other two... Relays don't care what the voltage on the other side of the coil is, MOSFETs do, unfortunately.
Again, for that battery stack I think what's needed is differential measurement of each battery voltage, followed by a comparator, then the MOSFET. The more I think about it, the more I think that if the relays work, they just take up more space than liked, I'd stick with them as from the little I know it might turn out to be a rather complicated circuit (that ironically emulates the 96 cent device in function).
Sorry, TorC, I've tried to think of a semiconductor solution, but can't come up with anything better.