Connection : one end connects to Vdd etc. The other end connects to gnd.
Purpose :
Low freq. : to reduce noise, smooth ripple to obtain a smooth dc. e.g 220uF to remove ripple
High freq. : to short particular frequency to gnd e.g. 10pF connect between signal line and gnd shorts ~1.8Ghz to gnd
Bypass cap. :
Connection : normally connects in series
Purpose : allow ac signal pass through while blocks dc signal.