In this case the folder is the desktop. That would be a problem because
Windows would probably not let you delete it unless you are logged in as a
different user and that user is an admin and the personal folders of your
existing username are not marked private.

If the filename is truly invalid open My Computer, right-click on C:, click on
Properties, go to the Tools tab and hit the Check Now button. Windows might
be able to fix thast file name and then you can delete it.