With poor selectivity, your system is hard to reject blockers and their intermodulation terms. It is not only resulted from insufficinet IF filtering, but also many related characteristics like RF filtering(for heterdyne), image rejection, LO phase noise, dynamic range, spurious free dynamic range, etc. Another case of insufficient selectivity resulted from LO leakage or Tx leakage, these two leakage produce blockers (interferrer) once your transceiver is powered on.
Sure, the BER is effected by insufficient selectivity. But the extreme case is that the whole receiver totally fails while there are some in-band and out-band blockers.