Just remember if you boost voltage, you're going to cut current. A decent AA battery should be able to produce about 1A at 1.5V, but if you want 12V, you'll only get about 100mA, depending on efficiency, etc. WE can't tell you why your joule thief didn't work. Did you actually MEASURE current? Did you measure the current out of the battery? Also, although the RUNNING current of the motor may be 100mA, you need a LOT more to get it started; that might be your problem. A capacitor across you power supply output might help.
You need to look at your circuit efficiency, the capacity of the battery(s) you want to use.