I2C uses only two signals for bidirectional communication, whereas SPI uses three. Start and stop conditions are indicated by a transition on the SDA (data) line, with the clock line HIGH. SPI does not need that, since the clock actually shifts the data, both in and out of the master/ slaves.
I2C allows addressing, SPI does not. In fact, SPI requires additional signals to select the different slaves.
I2C is a multimaster bus, featuring collision detection. SPI uses a single master which shifts data in/ out of shift registers in the master and the active slave.
I2C allows acknowlegde, SPI does not.
I2C signals are open-drain with pullups, to allow multiple access. In SPI, the master transmits the clock and MOSI and the slaves transmit MISO. Thus, all signals are unidirectional, driven hard in both directions.
In most cases, SPI is faster. Part of the reason is that I2C uses pullups, which slow it down in conjunction with the inherent bus capacitance.