I am designing a CMF(common-mode filter) in HFSS, and I want to get the
mixed-mode S-parameters of the CMF.But I has no knowledge how to get them
in HFSS.
you can simulate for conventional S parameters in HFSS and then convert them in differential-common mode parameters, there are equations how to do that.