Shunt capacitances at input and output stages in power circuits(either linear or smps) are generally used for lowpass filtering that filters out undesired ripple. Furthermore, I have experienced that some shunt capacitances around a LDO regulator may have important role in internal control mechanisms of the LDOs. If you dont put that cap there, you will see output voltage shifts to somewhere being far from your expectation.
As KerimF mentioned, you need to investigate your LDO's datasheet to understand roles of the caps. However, manufacturers may not give detailed information but there must be recommended application circuits, stick those schematics.