Simplest description:
You wire it across the supply line and place something (typically a resistor) in series with it to limit the current that can flow. It is wired so it normally doesn't conduct (positive to the cathode end) but the property of a Zener diode is it starts to conduct when a threshold voltage is reached, called the 'knee voltage'. You choose a suitable Zener diode that starts to conduct at the voltage you want to run your PIC from, say 4.7V or 5.1V and because it starts to conduct at that voltage and the current it can draw is limited, it stabilizes at the knee voltage. Less than that voltage stops it conducting, more than that voltage draws current and prevents the voltage going higher.
Brian.