I think I figured this out on my own. In case anyone has this question in the future, here was my problem and the solution.
In Visual Studio, the Advanced Installer MSI project cannot have "New Items" or "Existing Items" added to it. So, its not like any other Visual Studio projects that I am used to working with. You just can't add any items to it other than the ones it adds itself. It basically has just a single item (the .aip file) and one or more references. It appears that is all that will ever show up for the project in Solution Explorer.
I needed another item (the Add/Remove Programs icon file) added to the project, so on the file system, I created a folder named "Media" and put my icon file in it. Then, I used the Advanced Installer Visual Studio extension's "Product Details" tab to browse to the icon file. This works for building the file locally, but not on TFS because the file is not in source control yet.
The secret was to look in the "Excluded Changes - Additions" section on the Pending Changes page of Team Explorer. In there was the icon file, which I selected and "Promoted". This caused it to be added to TFS Source control, even though you still don't see it anywhere in Solution Explorer in Visual Studio. (Seems like the Advanced Installer Visual Studio extension should at least show it in Solution Explorer since it is an artifact of the project, but it doesn't).
Then, my TFS build succeeded because Advanced Installer could find the icon file since it had been uploaded to source control.
In general, I think the Advanced Installer documentation and YouTube videos are a little lacking when it comes to using Advanced Installer's Visual Studio extension. Maybe Advanced Installer is not used that much with Visual Studio and that is why there isn't a lot of documentation.
In my case, I got sick of trying to figure out a previous developer's Wix-based installer and am trying to migrate to Advance Installer because it has a GUI. It does have good documentation on the main Advanced Installer software, but not enough for the Visual Studio extension. So, some better documentation would have been helpful. I think I have most of it figured out by trial and error now, though.