Pull Up, Pull Down
It is a bad idea to leave a controller input disconnected - just hanging in the air. At all times, it should be connected to either a logical 1 (+5 Volts) or a logical 0 (ground = 0 Volts). If you leave an input pin not connected ("floating"), it may read in as a 0, a 1, or just flap up and down in the breeze.
Simple electrical switches just open and close. If you hook a switch up to an input, and the switch is open, the input will float and can not be trusted. You have to find a way to make sure that you always present a solid 1 or 0.
A common way to impose a default input is with pull-up or pull-down resistors. These resistors can be used to shove a 0 ("pull-down" to ground) or a 1 ("pull up" to +5 volts) into input pins.
The 0 or 1 provided by the resistor can be safely overpowered by the switch when it is activated. You should use the resistor that does the opposite of what your switch will do when activated:
if activating the switch pulls the input low (to ground), use a pull-up resistor
if activating the switch pulls the input high (to +5 Volts), use a pull-down resistor