If they are across the supply lines and ground yes, you can replace them with ceramics but polyester ones are stable and have virtually unlimited life so I doubt they are responsible for your problem. Electrolytic capacitors on the other hand, contain liquid electrolyte which tends to dry out over time so they should be replaced.
Don't go nuts trying to kill the noise completely, audio circuits in computers of that era were nowhere near as well designed as modern ones and the sound was often added as a novelty feature for sound effects in games, they were never intended to be high quality.
Brian.