Thanks PlanarMetamaterials for your comment.
I tried adding straight and uniform sections of my waveguide at the ends of my device, but it seems that the length of these "extensions" was not enough. I still cannot have similar performance in comparison to the simulation of a straight and homogeneous CPW case. (S11 parameter)
After this I have tried using a lumped port excitation scheme (with a vertical, normal to the structure, type of lumped port; similar to the type of excitation using when probing the device with GSG probes). For this case, I have better performance in terms of both reflection(S11 below -20dB) and transmission but how can I explain these differences?
I know that the lumped port excitation uses a voltage difference applied between the terminal and the reference conductors (ground electrodes for a CPW design), which tells me that the simulation is actually not solving in terms of the modal parameters (S modal parameters) but for the voltage-current defined S parameters.
How could I reconcile the concept of mode propagation through the structure with the results obtained using lumped ports excitation? Could I assume that the applied voltage results in a linear combination of "modes" (propagating, leaky and evanescent) ? And if so, why would the S parameters for both simulation schemes (wave port and lumped port) differ in their performance?
If you have any idea, thanks for sharing it.