Sounds like your motor may not have enough torque. I've modded several servos into gear-motors to be used in useless machine and the specs on them were around 3-5 Kg/cM.
The motor that I'm selling in my Useless kits stall at around 3Lbs/inch.
But, it all depends on how stiff your toggle is, and how long an arm you use (longer arms need more torque). And you can also adjust things by using 2, 3 or 4 batteries. The original design suggested using 4 AA's which in most cases was overkill. It made for a really strong/fast action...too fast in my opinion. By dropping down to 3 or 2 cells, there usually was enough torque to flip the toggle.
The only real way to know is to test things out.