media: v4l: async: Prepare for async sub-device notifiers

media: v4l: async: Prepare for async sub-device notifiers

Refactor the V4L2 async framework a little in preparation for async
sub-device notifiers. This avoids making some structural changes in the
patch actually implementing sub-device notifiers, making that patch easier
to review.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Acked-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>