Indeed, @ 1A that's (12-4V * 1A) = 8W, rather a big waste for a simple 1A regulated supply. Would require a decent size heatsink.However, you should know that, while the circuit is simple, it is very inefficient. At 4V 1A output, the efficiency is [(4*1)/(12*1)]*100% = 33.33%. So, you waste 2/3 of your input power / energy as heat. This will make the LM317 HOT! You will need to provide proper heatsinking to ensure you don't burn the LM317.
Do not overlook the idea of using a diode to subtract 0.7 volts or so from the 5V supply. Diodes are cheap and can handle the amps.
You can stack multiple diodes to obtain whatever drop you wish (as long as you can accept plus or minus .35 V deviation).
Is it important to get exactly 4V? Zeners are typically within a 5% tolerance of the nominal value. 3.9 is one of the standard values, and in reality may yield anywhere between 3.7 and 4.1V.
If you need exactly 4V, then it's probably best to go with an adjustable method.