1 .. This file is dual-licensed: you can use it either under the terms
2 .. of the GPL 2.0 or the GFDL 1.1+ license, at your option. Note that this
3 .. dual licensing only applies to this file, and not this project as a
6 .. a) This file is free software; you can redistribute it and/or
7 .. modify it under the terms of the GNU General Public License as
8 .. published by the Free Software Foundation version 2 of
11 .. This file is distributed in the hope that it will be useful,
12 .. but WITHOUT ANY WARRANTY; without even the implied warranty of
13 .. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14 .. GNU General Public License for more details.
18 .. b) Permission is granted to copy, distribute and/or modify this
19 .. document under the terms of the GNU Free Documentation License,
20 .. Version 1.1 or any later version published by the Free Software
21 .. Foundation, with no Invariant Sections, no Front-Cover Texts
22 .. and no Back-Cover Texts. A copy of the license is included at
23 .. Documentation/media/uapi/fdl-appendix.rst.
25 .. TODO: replace it to GPL-2.0 OR GFDL-1.1-or-later WITH no-invariant-sections
27 .. _media-request-api:
32 The Request API has been designed to allow V4L2 to deal with requirements of
33 modern devices (stateless codecs, complex camera pipelines, ...) and APIs
34 (Android Codec v2). One such requirement is the ability for devices belonging to
35 the same pipeline to reconfigure and collaborate closely on a per-frame basis.
36 Another is support of stateless codecs, which require controls to be applied
37 to specific frames (aka 'per-frame controls') in order to be used efficiently.
39 While the initial use-case was V4L2, it can be extended to other subsystems
40 as well, as long as they use the media controller.
42 Supporting these features without the Request API is not always possible and if
43 it is, it is terribly inefficient: user-space would have to flush all activity
44 on the media pipeline, reconfigure it for the next frame, queue the buffers to
45 be processed with that configuration, and wait until they are all available for
46 dequeuing before considering the next frame. This defeats the purpose of having
47 buffer queues since in practice only one buffer would be queued at a time.
49 The Request API allows a specific configuration of the pipeline (media
50 controller topology + configuration for each media entity) to be associated with
51 specific buffers. This allows user-space to schedule several tasks ("requests")
52 with different configurations in advance, knowing that the configuration will be
53 applied when needed to get the expected result. Configuration values at the time
54 of request completion are also available for reading.
59 The Request API extends the Media Controller API and cooperates with
60 subsystem-specific APIs to support request usage. At the Media Controller
61 level, requests are allocated from the supporting Media Controller device
62 node. Their life cycle is then managed through the request file descriptors in
63 an opaque way. Configuration data, buffer handles and processing results
64 stored in requests are accessed through subsystem-specific APIs extended for
65 request support, such as V4L2 APIs that take an explicit ``request_fd``
71 User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC`
72 for the media device node. This returns a file descriptor representing the
73 request. Typically, several such requests will be allocated.
78 Standard V4L2 ioctls can then receive a request file descriptor to express the
79 fact that the ioctl is part of said request, and is not to be applied
80 immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that
81 support this. Configurations set with a ``request_fd`` parameter are stored
82 instead of being immediately applied, and buffers queued to a request do not
83 enter the regular buffer queue until the request itself is queued.
88 Once the configuration and buffers of the request are specified, it can be
89 queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor.
90 A request must contain at least one buffer, otherwise ``ENOENT`` is returned.
91 A queued request cannot be modified anymore.
94 For :ref:`memory-to-memory devices <codec>` you can use requests only for
95 output buffers, not for capture buffers. Attempting to add a capture buffer
96 to a request will result in an ``EACCES`` error.
98 If the request contains configurations for multiple entities, individual drivers
99 may synchronize so the requested pipeline's topology is applied before the
100 buffers are processed. Media controller drivers do a best effort implementation
101 since perfect atomicity may not be possible due to hardware limitations.
105 It is not allowed to mix queuing requests with directly queuing buffers:
106 whichever method is used first locks this in place until
107 :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is
108 :ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier
109 a buffer was queued via a request or vice versa will result in an ``EBUSY``
112 Controls can still be set without a request and are applied immediately,
113 regardless of whether a request is in use or not.
117 Setting the same control through a request and also directly can lead to
120 User-space can :ref:`poll() <request-func-poll>` a request file descriptor in
121 order to wait until the request completes. A request is considered complete
122 once all its associated buffers are available for dequeuing and all the
123 associated controls have been updated with the values at the time of completion.
124 Note that user-space does not need to wait for the request to complete to
125 dequeue its buffers: buffers that are available halfway through a request can
126 be dequeued independently of the request's state.
128 A completed request contains the state of the device after the request was
129 executed. User-space can query that state by calling
130 :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file
131 descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a
132 request that has been queued but not yet completed will return ``EBUSY``
133 since the control values might be changed at any time by the driver while the
134 request is in flight.
136 .. _media-request-life-time:
138 Recycling and Destruction
139 -------------------------
141 Finally, a completed request can either be discarded or be reused. Calling
142 :ref:`close() <request-func-close>` on a request file descriptor will make
143 that file descriptor unusable and the request will be freed once it is no
144 longer in use by the kernel. That is, if the request is queued and then the
145 file descriptor is closed, then it won't be freed until the driver completed
148 The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it
149 available again. No state is retained by this operation: the request is as
150 if it had just been allocated.
152 Example for a Codec Device
153 --------------------------
155 For use-cases such as :ref:`codecs <codec>`, the request API can be used
156 to associate specific controls to
157 be applied by the driver for the OUTPUT buffer, allowing user-space
158 to queue many such buffers in advance. It can also take advantage of requests'
159 ability to capture the state of controls when the request completes to read back
160 information that may be subject to change.
162 Put into code, after obtaining a request, user-space can assign controls and one
167 struct v4l2_buffer buf;
168 struct v4l2_ext_controls ctrls;
171 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd))
174 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;
175 ctrls.request_fd = req_fd;
176 if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls))
179 buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
180 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD;
181 buf.request_fd = req_fd;
182 if (ioctl(codec_fd, VIDIOC_QBUF, &buf))
185 Note that it is not allowed to use the Request API for CAPTURE buffers
186 since there are no per-frame settings to report there.
188 Once the request is fully prepared, it can be queued to the driver:
192 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE))
195 User-space can then either wait for the request to complete by calling poll() on
196 its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will
197 want to get CAPTURE buffers as soon as possible and this can be done using a
198 regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`:
202 struct v4l2_buffer buf;
204 memset(&buf, 0, sizeof(buf));
205 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
206 if (ioctl(codec_fd, VIDIOC_DQBUF, &buf))
209 Note that this example assumes for simplicity that for every OUTPUT buffer
210 there will be one CAPTURE buffer, but this does not have to be the case.
212 We can then, after ensuring that the request is completed via polling the
213 request file descriptor, query control values at the time of its completion via
214 a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`.
215 This is particularly useful for volatile controls for which we want to
216 query values as soon as the capture buffer is produced.
220 struct pollfd pfd = { .events = POLLPRI, .fd = req_fd };
223 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;
224 ctrls.request_fd = req_fd;
225 if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls))
228 Once we don't need the request anymore, we can either recycle it for reuse with
229 :ref:`MEDIA_REQUEST_IOC_REINIT`...
233 if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT))
236 ... or close its file descriptor to completely dispose of it.
242 Example for a Simple Capture Device
243 -----------------------------------
245 With a simple capture device, requests can be used to specify controls to apply
246 for a given CAPTURE buffer.
250 struct v4l2_buffer buf;
251 struct v4l2_ext_controls ctrls;
254 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd))
257 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL;
258 ctrls.request_fd = req_fd;
259 if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls))
262 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
263 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD;
264 buf.request_fd = req_fd;
265 if (ioctl(camera_fd, VIDIOC_QBUF, &buf))
268 Once the request is fully prepared, it can be queued to the driver:
272 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE))
275 User-space can then dequeue buffers, wait for the request completion, query
276 controls and recycle the request as in the M2M example above.