After much time, trial and error, I was finally able to get Visual Studio's remote debugging features to work.  In my travels around the Internet, no one seems to have compiled all the steps to make the process work successfully and seemlessly into one page, this is my attempt.

Setting up

Complete all the steps listed on the How to: Set Up Remote Debugging article on MSDN.

Permissions

Don't try and fight the cross-domain permissions battle, its just not worth it.  If the machine your trying debug is not on a domain, then don't run VS from a machine thats on the domain. Make sure the same user with the same password exists on both machines.

PDB's

Put the PDB files on the remote machine (target) in the same folder as your app.

Source Code

In our environment, our build server produces an installer, and it also produces the PDB's we used in the above step. If we want Visual Studio to automatically pick up the right source files when we are debugging, you need to store the source code on your host machine in the same location as the machine that built your PDB's.

For example, in our world, on our build server the code lives on d:\code\projectname\code\ so on your host machine, you would store our source code on d:\code\projectname\code

Results

If you have followed all these steps, and if the stars align just right, you should now be able to step through your source code when remote debugging.

Questions/Comments

Questions, comments or just can't make it work for some reason? Leave a comment and fill out your actual email address nd I'll try to address it!