Tantalum and electrolytic caps have a high ESL (especially the
older radial lead through-hole types) which makes them ineffective
at returning the switching current spikes whence they came.
The low value small body ceramic caps can be had with much
higher SRFs, low ESL/ESR and have their part to play.
You may find modern surface mount tantalums with ESR/ESL
approaching ceramics if you are willing to dig and spend. You
need to look at the details, often not even on the datasheet
but maybe found in app notes or product family marketing
literature, to determine whether you have a decent cap.
Figure your desired / expected risetime equates to a quarter
cycle of some frequency, and you want the SRF of the cap to
be above this point (like, 1nS risetime is 1/4 of a 250MHz sine
and you'd like a cap that is not lossy, much, at that frequency.
Loss rolls on fast after SRF.