Hysteretic Converter Output Voltage Ripple issue

I'm trying to simulate a purely hysteretic converter in LTSpice (no one-shots used).
I used an Op-Amp block w feedback resistors to make a hysteretic comparator, the hysteresis is about 20mV.
The OpAmp drives a PMOS and 0.4V shottky is used for the catch-diode.
On VOUT there is inductor and cap and feedback resistors provide divided version of output voltage back to the hyst-comparator.

When I sumulate this circuit, I see huge ripple (200mV to 1V) on the output under most loads.
I tried varying the capacitor and inductor values (1uH-10uH, 2uF-100uF) with minimal improvements.
The lowest I could get the ripple to drop was 80mVpp when load current was reduced to smaller range.

(see attached screen captures)

Do you know what I could be doing wrong?

I did some more digging and seems the high-ripple I see on VOUT is due to control-loop dynamics.
While throwing the schematic together I totally ignored the loop gain-VS-freq behaviour.

By adding couple of caps experimentally (w/o calculating transfer-function or phase-margin) around the Op-amp and FB resistors I was able to reduce the ripple to ~25mV.