After a quick search I found that I ran into a known uninstall issue. The hanging in removing the Exchange files is caused by a final Powershell script that won't run so instead of ending in failure it just hangs and gives the appearance that it is still processing the removal.
All you need to do is open task manager and kill powershell.exe. Doing this will automatically spawn a new Powershell and allow the removal to complete. In my situation, the removal completed within 10 seconds of the new Powershell launching.