Below is the LTspice simulation of a circuit that should work for you.
It uses a TLV431 low-power programmable reference (about U$0.60) as a comparator with a built in 1.25V reference voltage.
The output goes high when the Ref voltage drops below 1.25V.
The U2 pot is shown adjusted to give an output high when the 9V battery drops below 6V.