Avoid mutex lock wait in socket close failure