The SMA connector is designed to achieve coplanar matching with the outer conductor. In other words, the center contact, dielectric, and outer conductor all lay in the same plane.
The dissimilar coaxial line sizes of the SMA female connectors produce a capacitive discontinuity at the reference plane that increases the VSWR at high frequencies (up to 1.2 at 20GHz).
Here is a trick to fix this discontinuity at very high frequencies.
Offsetting the female contact on the board can provide an inductive compensation for this capacitance. The resulting inductive discontinuity would create a mismatch in the "standard" junction, improving the VSWR.