sunrpc: fix peername failed on closed listener