What type of processor are you using?
I'm basing this reply on PIC processors as these are the ones I'm most familiar with. There are often other features of the micro which have to be turned off, for example, ADCs or comparator modules, you should preserve their state in software if necessary then disable them before going into sleep mode. If you have an independent watchdog timer that might also be drawing current.
As for the IO ports, there is no point in using pull-up or pull-down resistors when the port is an output so they are normally only enabled when in input mode, you may have inadvertently increased the current by making the pins into inputs during sleep. What you need to do is decide the optimum logic levels on the peripherals to ensure they do not draw current from the processor then set the pins to that level before sleeping. For example (I have not checked this, I'm only suggesting) if the flash memory had pull-down on it's inputs, it would be better to drive it low than to sink current through your pull-up then into the flash's pull-down. Obviously you have to take care that you don't put the peripheral into an undesired state by leaving the pins at levels that have bad consequences.
Brian.