drs_util: Improve memory usage when joining large DB
drs_Replicate.replicate() could consume a large amount of memory when
replicating a large DB. This is not a leak - the memory gets freed when
the function returns (i.e. once the partition is fully replicated).
However, while the partition is in the process of being replicated, it
accumulates memory for each replication chunk it receives. This can have
considerable overhead with 1000s of objects/links in the partition.
This was exhausting memory when joining a VM with 1Gb RAM to a DC with
25K users (average ~15 group memberships per user).
It seems that by storing a reference to something that's on the ctr's
talloc tree, it doesn't free up the memory for each ctr message (until
the function actually returns and req is destroyed).
With 10K users (and average 15 group memberships per user), .replicate()
consumed 211Mb of memory, according to talloc.report_full(). With this
patch, it goes down to just the current ctr message (1-2Mb).
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Oct 17 08:56:42 CEST 2018 on sn-devel-144