Sorry for the wait, I had to try some other features as well.
After more investigation, I found the following limitation:
- the ".NET InstallerClass" file is kept in use by the installer during "Install Execution Stage" action group and that's why it cannot be deleted
- in this case the file can be deleted only after "Finish Dialogs Stage" action group
- if the file is created in "[ProgramFilesFolder]\..." you need administrator privileges to delete it
This looks very helpful and I'm thankful... but it's contradictory to all my findings so far. I have :
- still in use in the Finish Dialogs stage (or at least, I can't delete it at that point).
- I can delete all other files in the same folder, even if it's Program Files and even if they're used after the installerclass, like my Start Service batch file. Buuuuuut... see below.
1. Go in the Install Parameters page and check the "Run as administrator" option (in this case you will have an EXE package instead of MSI), then place your VBScript custom action after the "Finish Dialogs Stage" action group
Nope, didn't work

Although the only configuration I tried so far is with the InstallerClass as Deferred/No Impersonation and the DeleteFolder as Immediate.
2. Set the path for your "DeploymentItems_Dir" folder so that it does not require administrator privileges (i.e. "C:\folder"), then place your VBScript custom action after the "Finish Dialogs Stage" action group
Yes! Finally, it works somehow! I currently still have the EXE Run As Admin configuration from my last tries, and deploying to C/Temp/ProgramName. More testing needed to see how much flexibility I have over this.
So the big breakthrough I was waiting for, thanks a lot for the help! Will post again if I get more details on this.
EDIT1 : so it worked, I started playing with some settings and could only reproduce it randomly. Found cause #1 for not working : I ask the user for a property (textbox) and in a SetProperty action I set it to empty {} before the dialog. If the user leaves it empty, for some reason my folder was not deployed at all and the script crashed. Weird. But beside that, there still seems to be something totally random that makes this fail : it works, I change an option, it crashes, I set it back, it still crashes. This seems to happen mostly with Install Parameter : Fast Installation.
EDIT2 : something at runtime seems to affect this somehow. If I have a working EXE and keep it, create a new EXE that fails, and re-run the first EXE, it will now fail instead of succeeding again. I have no idea what's going on. I did find that failed installers sometimes stay opened in the task manager (msiexec) which keeps the files locked, but I go delete them every time. I also noticed the EXE's size randomly changes between 1.8M and 2.3M -EDIT- that's most likely the encryption which changes when you play with the Fast Installation option.
EDIT3 : still randomly quirky. Just did a test that used the Program Files folder and the file was deleted successfully, which contradicts the reason for the first successes ([ProgramFilesFolder]\[Manufacturer]\[ProductName]). Totally expected that to fail. Still don't understand what's going on. On another note, I'm now suspecting the Media window's encryption (LZMA vs CAB) might be up to something, I think it changes sometimes with the Fast Installation and other options. Oh and I had some successes without Run As Admin too, but never with MSIs. Still investigating. Oh and sometimes I can set the DeleteFolder action to Deferred, sometimes it's disabled and I can only choose Immediately. Weird.
EDIT4 : Required : EXE output file, Fast Installation (Install parameters window), CAB encryption (Media window), action must be in Finish Dialogs and Immediate. My Installerclass action is Deferred with No Impersonation. And if you change any of those, it's very possible that even if you put those options back it still won't work. If that happens, try checking-unchecking options or closing AI and restarting it. I don't know why that "playing around" is needed. Oh, and don't forget to not leave any property empty (see comment in EDIT1 section).