The infinite ground plane is for purely theoretical stuff only - I only use it to show people the "perfect" monopole pattern.
Since in the end you probably want to build and test the antenna, I'd strongly suggest you assign PerfE to your ground plane - because the size of it has a big impact on the whole behavior.
Of course, if you use sheets (instead of solid boxes), then also the patch is PerfE. The pins should be PEC (at least for first impressions, to minimize computational time). Normally the SMA connectors are built of two cylinders (the inner is the pin, therefore PEC, the outer is the isolation, normally Teflon, eps=2.1) where the lateral surface of the outer cylinder is also PerfE.
Hope that helps. Otherwise some more clarification on your problem would be nice.