In case of a chemical battery (which would cover all batteries typically associated to that word), chemical reactions take place that consumes electrons at the cathode and deposit it at the anode. This continues until an equilibrium is reached, where the reverse reaction happens at the same rate. If you remember from chemistry class, equilibrium occurs because the reagents for the forward and reverse reaction are in such proportions that they occur at the same rate. For the reaction happening inside a battery, this always includes the electron. In electrical terms, the relative abundance (or lack) of electrons is referred to as voltage*. E.g. the reaction inside an AA battery reaches equilibrium when there is a 1.5V potential across the terminals.
Hence, the reason shorting both ends of an AA alkaline battery to ground produces no noteworthy result is because in such a case, the supply of electrons become limited by the rate of the chemical reaction, even when the chemical reaction has all the electrons it needs thanks to the shorted circuit. Whereas shorting a car battery will create sparks, because the reaction happens at a MUCH faster rate.
So what you really have is a chemical system that tries to maintain a constant voltage as charge is drawn from it, which is very different from a capacitor. While a battery will still lose voltage, the process is fundamentally different from C=Q/V.
So generally speaking, one cannot expect it to behave like a capacitor. Although, maybe at a specific frequency, the battery behaves closely enough to a capacitor that it CAN resonate.