Exception while retrieving REMOTE_USER server variable

We have a ashx http handler that retrieves the REMOTE_USER variable from the ServerVariables during the ProcessRequest.
        public void ProcessRequest(HttpContext context)
        {
            if (context.Request.ServerVariables["REMOTE_USER"].Contains("\\")) << exception is thrown here
            ...

The authentication is handled by IIS using basic authentication (windows accounts on domain controller).
For some users this starts failing (after working fine for a few hours) and throwing the following exception:
[IdentityNotMappedException: Some or all identity references could not be translated.] System.Security.Principal.SecurityIdentifier.Translate(IdentityReferenceCollect ...