1 .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-or-later WITH no-invariant-sections
8 The Request API has been designed to allow V4L2 to deal with requirements of
9 modern devices (stateless codecs, complex camera pipelines, ...) and APIs
10 (Android Codec v2). One such requirement is the ability for devices belonging to
11 the same pipeline to reconfigure and collaborate closely on a per-frame basis.
12 Another is support of stateless codecs, which require controls to be applied
13 to specific frames (aka 'per-frame controls') in order to be used efficiently.
15 While the initial use-case was V4L2, it can be extended to other subsystems
16 as well, as long as they use the media controller.
18 Supporting these features without the Request API is not always possible and if
19 it is, it is terribly inefficient: user-space would have to flush all activity
20 on the media pipeline, reconfigure it for the next frame, queue the buffers to
21 be processed with that configuration, and wait until they are all available for
22 dequeuing before considering the next frame. This defeats the purpose of having
23 buffer queues since in practice only one buffer would be queued at a time.
25 The Request API allows a specific configuration of the pipeline (media
26 controller topology + configuration for each media entity) to be associated with
27 specific buffers. This allows user-space to schedule several tasks ("requests")
28 with different configurations in advance, knowing that the configuration will be
29 applied when needed to get the expected result. Configuration values at the time
30 of request completion are also available for reading.
35 The Request API extends the Media Controller API and cooperates with
36 subsystem-specific APIs to support request usage. At the Media Controller
37 level, requests are allocated from the supporting Media Controller device
38 node. Their life cycle is then managed through the request file descriptors in
39 an opaque way. Configuration data, buffer handles and processing results
40 stored in requests are accessed through subsystem-specific APIs extended for
41 request support, such as V4L2 APIs that take an explicit ``request_fd``
47 User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC`
48 for the media device node. This returns a file descriptor representing the
49 request. Typically, several such requests will be allocated.
54 Standard V4L2 ioctls can then receive a request file descriptor to express the
55 fact that the ioctl is part of said request, and is not to be applied
56 immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that
57 support this. Configurations set with a ``request_fd`` parameter are stored
58 instead of being immediately applied, and buffers queued to a request do not
59 enter the regular buffer queue until the request itself is queued.
64 Once the configuration and buffers of the request are specified, it can be
65 queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor.
66 A request must contain at least one buffer, otherwise ``ENOENT`` is returned.
67 A queued request cannot be modified anymore.
70 For :ref:`memory-to-memory devices <codec>` you can use requests only for
71 output buffers, not for capture buffers. Attempting to add a capture buffer
72 to a request will result in an ``EACCES`` error.
74 If the request contains configurations for multiple entities, individual drivers
75 may synchronize so the requested pipeline's topology is applied before the
76 buffers are processed. Media controller drivers do a best effort implementation
77 since perfect atomicity may not be possible due to hardware limitations.
81 It is not allowed to mix queuing requests with directly queuing buffers:
82 whichever method is used first locks this in place until
83 :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is
84 :ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier
85 a buffer was queued via a request or vice versa will result in an ``EBUSY``
88 Controls can still be set without a request and are applied immediately,
89 regardless of whether a request is in use or not.
93 Setting the same control through a request and also directly can lead to
96 User-space can :ref:`poll() <request-func-poll>` a request file descriptor in
97 order to wait until the request completes. A request is considered complete
98 once all its associated buffers are available for dequeuing and all the
99 associated controls have been updated with the values at the time of completion.
100 Note that user-space does not need to wait for the request to complete to
101 dequeue its buffers: buffers that are available halfway through a request can
102 be dequeued independently of the request's state.
104 A completed request contains the state of the device after the request was
105 executed. User-space can query that state by calling
106 :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file
107 descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a
108 request that has been queued but not yet completed will return ``EBUSY``
109 since the control values might be changed at any time by the driver while the
110 request is in flight.
112 .. _media-request-life-time:
114 Recycling and Destruction
115 -------------------------
117 Finally, a completed request can either be discarded or be reused. Calling
118 :ref:`close() <request-func-close>` on a request file descriptor will make
119 that file descriptor unusable and the request will be freed once it is no
120 longer in use by the kernel. That is, if the request is queued and then the
121 file descriptor is closed, then it won't be freed until the driver completed
124 The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it
125 available again. No state is retained by this operation: the request is as
126 if it had just been allocated.
128 Example for a Codec Device
129 --------------------------
131 For use-cases such as :ref:`codecs <codec>`, the request API can be used
132 to associate specific controls to
133 be applied by the driver for the OUTPUT buffer, allowing user-space
134 to queue many such buffers in advance. It can also take advantage of requests'
135 ability to capture the state of controls when the request completes to read back
136 information that may be subject to change.
138 Put into code, after obtaining a request, user-space can assign controls and one
143 struct v4l2_buffer buf;
144 struct v4l2_ext_controls ctrls;
147 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd))
150 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;
151 ctrls.request_fd = req_fd;
152 if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls))
155 buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
156 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD;
157 buf.request_fd = req_fd;
158 if (ioctl(codec_fd, VIDIOC_QBUF, &buf))
161 Note that it is not allowed to use the Request API for CAPTURE buffers
162 since there are no per-frame settings to report there.
164 Once the request is fully prepared, it can be queued to the driver:
168 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE))
171 User-space can then either wait for the request to complete by calling poll() on
172 its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will
173 want to get CAPTURE buffers as soon as possible and this can be done using a
174 regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`:
178 struct v4l2_buffer buf;
180 memset(&buf, 0, sizeof(buf));
181 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
182 if (ioctl(codec_fd, VIDIOC_DQBUF, &buf))
185 Note that this example assumes for simplicity that for every OUTPUT buffer
186 there will be one CAPTURE buffer, but this does not have to be the case.
188 We can then, after ensuring that the request is completed via polling the
189 request file descriptor, query control values at the time of its completion via
190 a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`.
191 This is particularly useful for volatile controls for which we want to
192 query values as soon as the capture buffer is produced.
196 struct pollfd pfd = { .events = POLLPRI, .fd = req_fd };
199 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;
200 ctrls.request_fd = req_fd;
201 if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls))
204 Once we don't need the request anymore, we can either recycle it for reuse with
205 :ref:`MEDIA_REQUEST_IOC_REINIT`...
209 if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT))
212 ... or close its file descriptor to completely dispose of it.
218 Example for a Simple Capture Device
219 -----------------------------------
221 With a simple capture device, requests can be used to specify controls to apply
222 for a given CAPTURE buffer.
226 struct v4l2_buffer buf;
227 struct v4l2_ext_controls ctrls;
230 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd))
233 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;
234 ctrls.request_fd = req_fd;
235 if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls))
238 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
239 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD;
240 buf.request_fd = req_fd;
241 if (ioctl(camera_fd, VIDIOC_QBUF, &buf))
244 Once the request is fully prepared, it can be queued to the driver:
248 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE))
251 User-space can then dequeue buffers, wait for the request completion, query
252 controls and recycle the request as in the M2M example above.