For the same operating frequency, ridged WG is much smaller than regular rec.WG and it is sensitive, especially for your Ku-band case.
Thus to realise the transfomer, you probably need several steps (at least 3), and ridged structure should also be included in the first few transformer steps.
e.g, the first step of your transformer: bigger WG and lower ridge
the second step of your transformer: biggerbigger WG and lowerlower ridge
.
.
.
.
the last step of your transformer: regular Ku-band (WR75) Rec. WG and no ridge.
The length of the ridge could be optimized for impedance matching. Theoritically quarter wavelength transformer should work, but you need to calculate the impendance of your each transformer step by HFSS first.
Calculation must have some errors, then you have to use optimization. I know there is a pain by HFSS....