Creating CP with a single feed can be done, but it's usually done by:
a) Exciting both polarizations with the single source
b) Creating some sort of phase delay in one polarization with respect to the other
For example, a slot or truncated corner in a single-feed patch antenna.
To do this with a horn would be a bit of effort, an equivalent feed might be by inserting the pin at 45-degrees into a square waveguide, and then chamfering two corners of the waveguide to create the phase difference (I haven't seen this done, it's just a guess). I can't say it's been done before.
More simply, you could also use a 3-dB 90-degree coupler to take one input and divide it into two with a phase shift.