I have setup my product to allow side by side installations, if I install the latest version on a clean machine and then deinstall it works fine the following are tidied up correctly:
- Files & Folders
- Desktop icon
- Start menu entry
- Registry details
If I install the latest version side by side with a previous release and then try and uninstall it all goes horribly wrong. All of the above is left behind, each version installs to a versioned location so there is no crossover between the versions.