Running Internet Explorer 9, the icons display correctly:
However if you switch Internet Explorer 10 into compatibility mode, the icons also display correctly. To enable compatibility mode click the following page icon next to the address bar.
When it turns blue in colour, this means compatibility mode is enabled and the RD Web Access icons will reappear in RD Web Access.
A bug has been logged with Microsoft on this issue.