These capacitors provide a low impedance supply voltage to key areas of the chip. In general the inner layout is comprised of individual distinct cores or other complex circuits, but the wafer metalization process do not ensure sufficient charge capacity as an external component can do, which can grow up its capacitance with its height profile; a capacitor made on silicon is much more expensive if compared to any other technologies.
I have been repairing some industrial PCs form a manufacturer which do not assemble heatsinks with these 4 spacers on each corner as shown in the 2nd picture, but are more similar to the 1st picture. They took in consideration the gap between the CPU and Capacitors. I have been saving some mainboards from scrap with minor defects, but still usable, but it is a little annoying when I was trying to mount my own 'homemade' computer with them. I asked myself why didn't they simply covered them with some resin; perhaps due to termal issues.