This is in fact a switched capacitor CMFB Circuit. I think that the switches are added so that you intially charge your circuit to VREF. In one phase, the sampling capacitor charges to VREF and in the other phase to the VOUTP/VOUTM. Then the difference is summed to give rise to the common mode correction voltage.
Refer to an IEEE paper titled "Analysis of Switched-Capacitor Common Mode Feedback Circuit" for good insight.