In a current measurement (100uA range), when I stroke the outside surface of this analog multimeter with my palm, the finger run towards large scale.
Does human magnetic field affect it?
How can I avoid this kind of casual interference?
Not the magnetic field ! what you are almost certainly seeing is a static charge building on the meter window which is attracting the finger. The finger only takes a very slight 'push' to move it and the static charge probably has sufficient force to pull it against it's return spring.
True, many times, in situations like one described the static charge could develop potentials exceeding thousands of volts that could easily affect the correct position of finger. (This static voltage plays the role of attracting the finger but does not directly enter the circuit, except for pickup).
Back in the day, (b4 digital readouts) it was common practice to remove the plastic cover from the meter and coat the inside surface with a solution of water and dish washing liquid soap, I suppose any soap dissolved in water would work. It works by preventing the build up of a static charge. Try it, you will like it.