dm mpath: eliminate use of spinlock in IO fast-paths
authorMike Snitzer <snitzer@redhat.com>
Thu, 17 Mar 2016 22:38:17 +0000 (18:38 -0400)
committerMike Snitzer <snitzer@redhat.com>
Thu, 5 May 2016 19:25:52 +0000 (15:25 -0400)
commit2da1610ae20e995e53658c3b10166d2ad74e30bd
tree4bee08cc145727a6366b2d516e41d2f703e3090c
parent20800cb3450ee44ec1827d7e8bbfd5a9dc02e6cd
dm mpath: eliminate use of spinlock in IO fast-paths

The primary motivation of this commit is to improve the scalability of
DM multipath on large NUMA systems where m->lock spinlock contention has
been proven to be a serious bottleneck on really fast storage.

The ability to atomically read a pointer, using lockless_dereference(),
is leveraged in this commit.  But all pointer writes are still protected
by the m->lock spinlock (which is fine since these all now occur in the
slow-path).

The following functions no longer require the m->lock spinlock in their
fast-path: multipath_busy(), __multipath_map(), and do_end_io()

And choose_pgpath() is modified to _not_ update m->current_pgpath unless
it also switches the path-group.  This is done to avoid needing to take
the m->lock everytime __multipath_map() calls choose_pgpath().
But m->current_pgpath will be reset if it is failed via fail_path().

Suggested-by: Jeff Moyer <jmoyer@redhat.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Tested-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
drivers/md/dm-mpath.c