I know that for negative supplies a LM79xx is needed. As I don't have any of those I was thinking to
connect two LM7805 outputs in Series. Each one powered by a separate battery. This should then be similar to two bench power supplies with floating outputs connected in series.
It should be possible to have -5 - 5V or 0-10V supply. Is my thinking correct? And if not, what is the problem and why does it work with bench supplies?
Why do you need a dual-polarity power supply?
ANY AC opamp circuit will work with a single supply voltage if its (+) input is biased at about half the supply voltage and the signal input, output and feedback to ground parts are capacitor-coupled.
Here is an example: