Freqzt,
Remember for 2:1 VSWR match with a 50 Ohm line , which is close to 10dB return loss or -10dB |S11|, you can have approximately 25-100 Ohm input impedance swing. Frankly, antenna can be matched according to the definition (i.e. 10dB return loss) even if it does not have 50 Ohm input impedance.
You have asked whether is it needed to bother about input impedance if S11 is fine. My answer is no, unless you are concerned about normalized transfer functions of the antenna.
Afterall, the matching of a UWB antenna is the least of the worries, go ahead and analyze the radiation patterns.