Yes well it depends on what type of distro are U using (one that uses rpm - red hat package managment or deb - debian packages) RPM's and DEB's are simular to windows msi packages as they contain the compiled binary files and a database of all the files in the package.
When you need to uninstall it the software managment app eg. rpm queries the software database on what file came with what soft package, and on request it removes it.
There are a lot of diferent Linuxes in this world some great some not, and most of them do hava administration software, like the controll panel in windows, but most of these softwares are different on different linuxes:
Suse has -> Yast2;
Mandriva -> drakX;
edc...