By default they don't. The default location for the downloaded files are the same folder where the installer .exe is stored and the installer files are extracted "somehwere else" (I guess a temp folder).
If the download location is not set by the user, it's indeed automatically set to the same location as the .EXE.
After later testing and investigating it seems deleting the downloaded packages of global prerequisites
is not currently supported. By keeping the "Do not delete extracted MSI and CAB files" option disabled, only global prerequisites included in the package will be deleted.
I discussed the issue with the development team, they are aware of it and undergoing a solution for a future release.
Meanwhile, I'm afraid there's no other workaround than to use feature-based prerequisites.
The custom action solution I posted the first time is not valid because it has much more implications than I initially anticipated. One of them, as you noticed yourself, is setting the download location.
All the best,