selftest: use MALLOC_CHECK_=3 to give a diagnostic on failure