5 The RxRPC protocol driver provides a reliable two-phase transport on top of UDP
6 that can be used to perform RxRPC remote operations. This is done over sockets
7 of AF_RXRPC family, using sendmsg() and recvmsg() with control data to send and
8 receive data, aborts and errors.
10 Contents of this document:
14 (*) RxRPC protocol summary.
16 (*) AF_RXRPC driver model.
24 (*) Example client usage.
26 (*) Example server usage.
28 (*) AF_RXRPC kernel interface.
30 (*) Configurable parameters.
37 RxRPC is a two-layer protocol. There is a session layer which provides
38 reliable virtual connections using UDP over IPv4 (or IPv6) as the transport
39 layer, but implements a real network protocol; and there's the presentation
40 layer which renders structured data to binary blobs and back again using XDR
56 (1) Part of an RxRPC facility for both kernel and userspace applications by
57 making the session part of it a Linux network protocol (AF_RXRPC).
59 (2) A two-phase protocol. The client transmits a blob (the request) and then
60 receives a blob (the reply), and the server receives the request and then
63 (3) Retention of the reusable bits of the transport system set up for one call
64 to speed up subsequent calls.
66 (4) A secure protocol, using the Linux kernel's key retention facility to
67 manage security on the client end. The server end must of necessity be
68 more active in security negotiations.
70 AF_RXRPC does not provide XDR marshalling/presentation facilities. That is
71 left to the application. AF_RXRPC only deals in blobs. Even the operation ID
72 is just the first four bytes of the request blob, and as such is beyond the
76 Sockets of AF_RXRPC family are:
78 (1) created as type SOCK_DGRAM;
80 (2) provided with a protocol of the type of underlying transport they're going
81 to use - currently only PF_INET is supported.
84 The Andrew File System (AFS) is an example of an application that uses this and
85 that has both kernel (filesystem) and userspace (utility) components.
88 ======================
89 RXRPC PROTOCOL SUMMARY
90 ======================
92 An overview of the RxRPC protocol:
94 (*) RxRPC sits on top of another networking protocol (UDP is the only option
95 currently), and uses this to provide network transport. UDP ports, for
96 example, provide transport endpoints.
98 (*) RxRPC supports multiple virtual "connections" from any given transport
99 endpoint, thus allowing the endpoints to be shared, even to the same
102 (*) Each connection goes to a particular "service". A connection may not go
103 to multiple services. A service may be considered the RxRPC equivalent of
104 a port number. AF_RXRPC permits multiple services to share an endpoint.
106 (*) Client-originating packets are marked, thus a transport endpoint can be
107 shared between client and server connections (connections have a
110 (*) Up to a billion connections may be supported concurrently between one
111 local transport endpoint and one service on one remote endpoint. An RxRPC
112 connection is described by seven numbers:
115 Local port } Transport (UDP) address
122 (*) Each RxRPC operation is a "call". A connection may make up to four
123 billion calls, but only up to four calls may be in progress on a
124 connection at any one time.
126 (*) Calls are two-phase and asymmetric: the client sends its request data,
127 which the service receives; then the service transmits the reply data
128 which the client receives.
130 (*) The data blobs are of indefinite size, the end of a phase is marked with a
131 flag in the packet. The number of packets of data making up one blob may
132 not exceed 4 billion, however, as this would cause the sequence number to
135 (*) The first four bytes of the request data are the service operation ID.
137 (*) Security is negotiated on a per-connection basis. The connection is
138 initiated by the first data packet on it arriving. If security is
139 requested, the server then issues a "challenge" and then the client
140 replies with a "response". If the response is successful, the security is
141 set for the lifetime of that connection, and all subsequent calls made
142 upon it use that same security. In the event that the server lets a
143 connection lapse before the client, the security will be renegotiated if
144 the client uses the connection again.
146 (*) Calls use ACK packets to handle reliability. Data packets are also
147 explicitly sequenced per call.
149 (*) There are two types of positive acknowledgment: hard-ACKs and soft-ACKs.
150 A hard-ACK indicates to the far side that all the data received to a point
151 has been received and processed; a soft-ACK indicates that the data has
152 been received but may yet be discarded and re-requested. The sender may
153 not discard any transmittable packets until they've been hard-ACK'd.
155 (*) Reception of a reply data packet implicitly hard-ACK's all the data
156 packets that make up the request.
158 (*) An call is complete when the request has been sent, the reply has been
159 received and the final hard-ACK on the last packet of the reply has
162 (*) An call may be aborted by either end at any time up to its completion.
165 =====================
166 AF_RXRPC DRIVER MODEL
167 =====================
169 About the AF_RXRPC driver:
171 (*) The AF_RXRPC protocol transparently uses internal sockets of the transport
172 protocol to represent transport endpoints.
174 (*) AF_RXRPC sockets map onto RxRPC connection bundles. Actual RxRPC
175 connections are handled transparently. One client socket may be used to
176 make multiple simultaneous calls to the same service. One server socket
177 may handle calls from many clients.
179 (*) Additional parallel client connections will be initiated to support extra
180 concurrent calls, up to a tunable limit.
182 (*) Each connection is retained for a certain amount of time [tunable] after
183 the last call currently using it has completed in case a new call is made
186 (*) Each internal UDP socket is retained [tunable] for a certain amount of
187 time [tunable] after the last connection using it discarded, in case a new
188 connection is made that could use it.
190 (*) A client-side connection is only shared between calls if they have have
191 the same key struct describing their security (and assuming the calls
192 would otherwise share the connection). Non-secured calls would also be
193 able to share connections with each other.
195 (*) A server-side connection is shared if the client says it is.
197 (*) ACK'ing is handled by the protocol driver automatically, including ping
200 (*) SO_KEEPALIVE automatically pings the other side to keep the connection
203 (*) If an ICMP error is received, all calls affected by that error will be
204 aborted with an appropriate network error passed through recvmsg().
207 Interaction with the user of the RxRPC socket:
209 (*) A socket is made into a server socket by binding an address with a
212 (*) In the client, sending a request is achieved with one or more sendmsgs,
213 followed by the reply being received with one or more recvmsgs.
215 (*) The first sendmsg for a request to be sent from a client contains a tag to
216 be used in all other sendmsgs or recvmsgs associated with that call. The
217 tag is carried in the control data.
219 (*) connect() is used to supply a default destination address for a client
220 socket. This may be overridden by supplying an alternate address to the
221 first sendmsg() of a call (struct msghdr::msg_name).
223 (*) If connect() is called on an unbound client, a random local port will
224 bound before the operation takes place.
226 (*) A server socket may also be used to make client calls. To do this, the
227 first sendmsg() of the call must specify the target address. The server's
228 transport endpoint is used to send the packets.
230 (*) Once the application has received the last message associated with a call,
231 the tag is guaranteed not to be seen again, and so it can be used to pin
232 client resources. A new call can then be initiated with the same tag
233 without fear of interference.
235 (*) In the server, a request is received with one or more recvmsgs, then the
236 the reply is transmitted with one or more sendmsgs, and then the final ACK
237 is received with a last recvmsg.
239 (*) When sending data for a call, sendmsg is given MSG_MORE if there's more
240 data to come on that call.
242 (*) When receiving data for a call, recvmsg flags MSG_MORE if there's more
243 data to come for that call.
245 (*) When receiving data or messages for a call, MSG_EOR is flagged by recvmsg
246 to indicate the terminal message for that call.
248 (*) A call may be aborted by adding an abort control message to the control
249 data. Issuing an abort terminates the kernel's use of that call's tag.
250 Any messages waiting in the receive queue for that call will be discarded.
252 (*) Aborts, busy notifications and challenge packets are delivered by recvmsg,
253 and control data messages will be set to indicate the context. Receiving
254 an abort or a busy message terminates the kernel's use of that call's tag.
256 (*) The control data part of the msghdr struct is used for a number of things:
258 (*) The tag of the intended or affected call.
260 (*) Sending or receiving errors, aborts and busy notifications.
262 (*) Notifications of incoming calls.
264 (*) Sending debug requests and receiving debug replies [TODO].
266 (*) When the kernel has received and set up an incoming call, it sends a
267 message to server application to let it know there's a new call awaiting
268 its acceptance [recvmsg reports a special control message]. The server
269 application then uses sendmsg to assign a tag to the new call. Once that
270 is done, the first part of the request data will be delivered by recvmsg.
272 (*) The server application has to provide the server socket with a keyring of
273 secret keys corresponding to the security types it permits. When a secure
274 connection is being set up, the kernel looks up the appropriate secret key
275 in the keyring and then sends a challenge packet to the client and
276 receives a response packet. The kernel then checks the authorisation of
277 the packet and either aborts the connection or sets up the security.
279 (*) The name of the key a client will use to secure its communications is
280 nominated by a socket option.
285 (*) MSG_WAITALL can be set to tell sendmsg to ignore signals if the peer is
286 making progress at accepting packets within a reasonable time such that we
287 manage to queue up all the data for transmission. This requires the
288 client to accept at least one packet per 2*RTT time period.
290 If this isn't set, sendmsg() will return immediately, either returning
291 EINTR/ERESTARTSYS if nothing was consumed or returning the amount of data
297 (*) If there's a sequence of data messages belonging to a particular call on
298 the receive queue, then recvmsg will keep working through them until:
300 (a) it meets the end of that call's received data,
302 (b) it meets a non-data message,
304 (c) it meets a message belonging to a different call, or
306 (d) it fills the user buffer.
308 If recvmsg is called in blocking mode, it will keep sleeping, awaiting the
309 reception of further data, until one of the above four conditions is met.
311 (2) MSG_PEEK operates similarly, but will return immediately if it has put any
312 data in the buffer rather than sleeping until it can fill the buffer.
314 (3) If a data message is only partially consumed in filling a user buffer,
315 then the remainder of that message will be left on the front of the queue
316 for the next taker. MSG_TRUNC will never be flagged.
318 (4) If there is more data to be had on a call (it hasn't copied the last byte
319 of the last data message in that phase yet), then MSG_MORE will be
327 AF_RXRPC makes use of control messages in sendmsg() and recvmsg() to multiplex
328 calls, to invoke certain actions and to report certain conditions. These are:
330 MESSAGE ID SRT DATA MEANING
331 ======================= === =========== ===============================
332 RXRPC_USER_CALL_ID sr- User ID App's call specifier
333 RXRPC_ABORT srt Abort code Abort code to issue/received
334 RXRPC_ACK -rt n/a Final ACK received
335 RXRPC_NET_ERROR -rt error num Network error on call
336 RXRPC_BUSY -rt n/a Call rejected (server busy)
337 RXRPC_LOCAL_ERROR -rt error num Local error encountered
338 RXRPC_NEW_CALL -r- n/a New call received
339 RXRPC_ACCEPT s-- n/a Accept new call
340 RXRPC_EXCLUSIVE_CALL s-- n/a Make an exclusive client call
341 RXRPC_UPGRADE_SERVICE s-- n/a Client call can be upgraded
342 RXRPC_TX_LENGTH s-- data len Total length of Tx data
344 (SRT = usable in Sendmsg / delivered by Recvmsg / Terminal message)
346 (*) RXRPC_USER_CALL_ID
348 This is used to indicate the application's call ID. It's an unsigned long
349 that the app specifies in the client by attaching it to the first data
350 message or in the server by passing it in association with an RXRPC_ACCEPT
351 message. recvmsg() passes it in conjunction with all messages except
352 those of the RXRPC_NEW_CALL message.
356 This is can be used by an application to abort a call by passing it to
357 sendmsg, or it can be delivered by recvmsg to indicate a remote abort was
358 received. Either way, it must be associated with an RXRPC_USER_CALL_ID to
359 specify the call affected. If an abort is being sent, then error EBADSLT
360 will be returned if there is no call with that user ID.
364 This is delivered to a server application to indicate that the final ACK
365 of a call was received from the client. It will be associated with an
366 RXRPC_USER_CALL_ID to indicate the call that's now complete.
370 This is delivered to an application to indicate that an ICMP error message
371 was encountered in the process of trying to talk to the peer. An
372 errno-class integer value will be included in the control message data
373 indicating the problem, and an RXRPC_USER_CALL_ID will indicate the call
378 This is delivered to a client application to indicate that a call was
379 rejected by the server due to the server being busy. It will be
380 associated with an RXRPC_USER_CALL_ID to indicate the rejected call.
382 (*) RXRPC_LOCAL_ERROR
384 This is delivered to an application to indicate that a local error was
385 encountered and that a call has been aborted because of it. An
386 errno-class integer value will be included in the control message data
387 indicating the problem, and an RXRPC_USER_CALL_ID will indicate the call
392 This is delivered to indicate to a server application that a new call has
393 arrived and is awaiting acceptance. No user ID is associated with this,
394 as a user ID must subsequently be assigned by doing an RXRPC_ACCEPT.
398 This is used by a server application to attempt to accept a call and
399 assign it a user ID. It should be associated with an RXRPC_USER_CALL_ID
400 to indicate the user ID to be assigned. If there is no call to be
401 accepted (it may have timed out, been aborted, etc.), then sendmsg will
402 return error ENODATA. If the user ID is already in use by another call,
403 then error EBADSLT will be returned.
405 (*) RXRPC_EXCLUSIVE_CALL
407 This is used to indicate that a client call should be made on a one-off
408 connection. The connection is discarded once the call has terminated.
410 (*) RXRPC_UPGRADE_SERVICE
412 This is used to make a client call to probe if the specified service ID
413 may be upgraded by the server. The caller must check msg_name returned to
414 recvmsg() for the service ID actually in use. The operation probed must
415 be one that takes the same arguments in both services.
417 Once this has been used to establish the upgrade capability (or lack
418 thereof) of the server, the service ID returned should be used for all
419 future communication to that server and RXRPC_UPGRADE_SERVICE should no
424 This is used to inform the kernel of the total amount of data that is
425 going to be transmitted by a call (whether in a client request or a
426 service response). If given, it allows the kernel to encrypt from the
427 userspace buffer directly to the packet buffers, rather than copying into
428 the buffer and then encrypting in place. This may only be given with the
429 first sendmsg() providing data for a call. EMSGSIZE will be generated if
430 the amount of data actually given is different.
432 This takes a parameter of __s64 type that indicates how much will be
433 transmitted. This may not be less than zero.
435 The symbol RXRPC__SUPPORTED is defined as one more than the highest control
436 message type supported. At run time this can be queried by means of the
437 RXRPC_SUPPORTED_CMSG socket option (see below).
444 AF_RXRPC sockets support a few socket options at the SOL_RXRPC level:
446 (*) RXRPC_SECURITY_KEY
448 This is used to specify the description of the key to be used. The key is
449 extracted from the calling process's keyrings with request_key() and
450 should be of "rxrpc" type.
452 The optval pointer points to the description string, and optlen indicates
453 how long the string is, without the NUL terminator.
455 (*) RXRPC_SECURITY_KEYRING
457 Similar to above but specifies a keyring of server secret keys to use (key
458 type "keyring"). See the "Security" section.
460 (*) RXRPC_EXCLUSIVE_CONNECTION
462 This is used to request that new connections should be used for each call
463 made subsequently on this socket. optval should be NULL and optlen 0.
465 (*) RXRPC_MIN_SECURITY_LEVEL
467 This is used to specify the minimum security level required for calls on
468 this socket. optval must point to an int containing one of the following
471 (a) RXRPC_SECURITY_PLAIN
473 Encrypted checksum only.
475 (b) RXRPC_SECURITY_AUTH
477 Encrypted checksum plus packet padded and first eight bytes of packet
478 encrypted - which includes the actual packet length.
480 (c) RXRPC_SECURITY_ENCRYPTED
482 Encrypted checksum plus entire packet padded and encrypted, including
483 actual packet length.
485 (*) RXRPC_UPGRADEABLE_SERVICE
487 This is used to indicate that a service socket with two bindings may
488 upgrade one bound service to the other if requested by the client. optval
489 must point to an array of two unsigned short ints. The first is the
490 service ID to upgrade from and the second the service ID to upgrade to.
492 (*) RXRPC_SUPPORTED_CMSG
494 This is a read-only option that writes an int into the buffer indicating
495 the highest control message type supported.
502 Currently, only the kerberos 4 equivalent protocol has been implemented
503 (security index 2 - rxkad). This requires the rxkad module to be loaded and,
504 on the client, tickets of the appropriate type to be obtained from the AFS
505 kaserver or the kerberos server and installed as "rxrpc" type keys. This is
506 normally done using the klog program. An example simple klog program can be
509 http://people.redhat.com/~dhowells/rxrpc/klog.c
511 The payload provided to add_key() on the client should be of the following
514 struct rxrpc_key_sec2_v1 {
515 uint16_t security_index; /* 2 */
516 uint16_t ticket_length; /* length of ticket[] */
517 uint32_t expiry; /* time at which expires */
518 uint8_t kvno; /* key version number */
520 uint8_t session_key[8]; /* DES session key */
521 uint8_t ticket[0]; /* the encrypted ticket */
524 Where the ticket blob is just appended to the above structure.
527 For the server, keys of type "rxrpc_s" must be made available to the server.
528 They have a description of "<serviceID>:<securityIndex>" (eg: "52:2" for an
529 rxkad key for the AFS VL service). When such a key is created, it should be
530 given the server's secret key as the instantiation data (see the example
533 add_key("rxrpc_s", "52:2", secret_key, 8, keyring);
535 A keyring is passed to the server socket by naming it in a sockopt. The server
536 socket then looks the server secret keys up in this keyring when secure
537 incoming connections are made. This can be seen in an example program that can
540 http://people.redhat.com/~dhowells/rxrpc/listen.c
547 A client would issue an operation by:
549 (1) An RxRPC socket is set up by:
551 client = socket(AF_RXRPC, SOCK_DGRAM, PF_INET);
553 Where the third parameter indicates the protocol family of the transport
554 socket used - usually IPv4 but it can also be IPv6 [TODO].
556 (2) A local address can optionally be bound:
558 struct sockaddr_rxrpc srx = {
559 .srx_family = AF_RXRPC,
560 .srx_service = 0, /* we're a client */
561 .transport_type = SOCK_DGRAM, /* type of transport socket */
562 .transport.sin_family = AF_INET,
563 .transport.sin_port = htons(7000), /* AFS callback */
564 .transport.sin_address = 0, /* all local interfaces */
566 bind(client, &srx, sizeof(srx));
568 This specifies the local UDP port to be used. If not given, a random
569 non-privileged port will be used. A UDP port may be shared between
570 several unrelated RxRPC sockets. Security is handled on a basis of
571 per-RxRPC virtual connection.
573 (3) The security is set:
575 const char *key = "AFS:cambridge.redhat.com";
576 setsockopt(client, SOL_RXRPC, RXRPC_SECURITY_KEY, key, strlen(key));
578 This issues a request_key() to get the key representing the security
579 context. The minimum security level can be set:
581 unsigned int sec = RXRPC_SECURITY_ENCRYPTED;
582 setsockopt(client, SOL_RXRPC, RXRPC_MIN_SECURITY_LEVEL,
585 (4) The server to be contacted can then be specified (alternatively this can
586 be done through sendmsg):
588 struct sockaddr_rxrpc srx = {
589 .srx_family = AF_RXRPC,
590 .srx_service = VL_SERVICE_ID,
591 .transport_type = SOCK_DGRAM, /* type of transport socket */
592 .transport.sin_family = AF_INET,
593 .transport.sin_port = htons(7005), /* AFS volume manager */
594 .transport.sin_address = ...,
596 connect(client, &srx, sizeof(srx));
598 (5) The request data should then be posted to the server socket using a series
599 of sendmsg() calls, each with the following control message attached:
601 RXRPC_USER_CALL_ID - specifies the user ID for this call
603 MSG_MORE should be set in msghdr::msg_flags on all but the last part of
604 the request. Multiple requests may be made simultaneously.
606 An RXRPC_TX_LENGTH control message can also be specified on the first
609 If a call is intended to go to a destination other than the default
610 specified through connect(), then msghdr::msg_name should be set on the
611 first request message of that call.
613 (6) The reply data will then be posted to the server socket for recvmsg() to
614 pick up. MSG_MORE will be flagged by recvmsg() if there's more reply data
615 for a particular call to be read. MSG_EOR will be set on the terminal
618 All data will be delivered with the following control message attached:
620 RXRPC_USER_CALL_ID - specifies the user ID for this call
622 If an abort or error occurred, this will be returned in the control data
623 buffer instead, and MSG_EOR will be flagged to indicate the end of that
626 A client may ask for a service ID it knows and ask that this be upgraded to a
627 better service if one is available by supplying RXRPC_UPGRADE_SERVICE on the
628 first sendmsg() of a call. The client should then check srx_service in the
629 msg_name filled in by recvmsg() when collecting the result. srx_service will
630 hold the same value as given to sendmsg() if the upgrade request was ignored by
631 the service - otherwise it will be altered to indicate the service ID the
632 server upgraded to. Note that the upgraded service ID is chosen by the server.
633 The caller has to wait until it sees the service ID in the reply before sending
634 any more calls (further calls to the same destination will be blocked until the
642 A server would be set up to accept operations in the following manner:
644 (1) An RxRPC socket is created by:
646 server = socket(AF_RXRPC, SOCK_DGRAM, PF_INET);
648 Where the third parameter indicates the address type of the transport
649 socket used - usually IPv4.
651 (2) Security is set up if desired by giving the socket a keyring with server
654 keyring = add_key("keyring", "AFSkeys", NULL, 0,
655 KEY_SPEC_PROCESS_KEYRING);
657 const char secret_key[8] = {
658 0xa7, 0x83, 0x8a, 0xcb, 0xc7, 0x83, 0xec, 0x94 };
659 add_key("rxrpc_s", "52:2", secret_key, 8, keyring);
661 setsockopt(server, SOL_RXRPC, RXRPC_SECURITY_KEYRING, "AFSkeys", 7);
663 The keyring can be manipulated after it has been given to the socket. This
664 permits the server to add more keys, replace keys, etc. while it is live.
666 (3) A local address must then be bound:
668 struct sockaddr_rxrpc srx = {
669 .srx_family = AF_RXRPC,
670 .srx_service = VL_SERVICE_ID, /* RxRPC service ID */
671 .transport_type = SOCK_DGRAM, /* type of transport socket */
672 .transport.sin_family = AF_INET,
673 .transport.sin_port = htons(7000), /* AFS callback */
674 .transport.sin_address = 0, /* all local interfaces */
676 bind(server, &srx, sizeof(srx));
678 More than one service ID may be bound to a socket, provided the transport
679 parameters are the same. The limit is currently two. To do this, bind()
680 should be called twice.
682 (4) If service upgrading is required, first two service IDs must have been
683 bound and then the following option must be set:
685 unsigned short service_ids[2] = { from_ID, to_ID };
686 setsockopt(server, SOL_RXRPC, RXRPC_UPGRADEABLE_SERVICE,
687 service_ids, sizeof(service_ids));
689 This will automatically upgrade connections on service from_ID to service
690 to_ID if they request it. This will be reflected in msg_name obtained
691 through recvmsg() when the request data is delivered to userspace.
693 (5) The server is then set to listen out for incoming calls:
697 (6) The kernel notifies the server of pending incoming connections by sending
698 it a message for each. This is received with recvmsg() on the server
699 socket. It has no data, and has a single dataless control message
704 The address that can be passed back by recvmsg() at this point should be
705 ignored since the call for which the message was posted may have gone by
706 the time it is accepted - in which case the first call still on the queue
709 (7) The server then accepts the new call by issuing a sendmsg() with two
710 pieces of control data and no actual data:
712 RXRPC_ACCEPT - indicate connection acceptance
713 RXRPC_USER_CALL_ID - specify user ID for this call
715 (8) The first request data packet will then be posted to the server socket for
716 recvmsg() to pick up. At that point, the RxRPC address for the call can
717 be read from the address fields in the msghdr struct.
719 Subsequent request data will be posted to the server socket for recvmsg()
720 to collect as it arrives. All but the last piece of the request data will
721 be delivered with MSG_MORE flagged.
723 All data will be delivered with the following control message attached:
725 RXRPC_USER_CALL_ID - specifies the user ID for this call
727 (9) The reply data should then be posted to the server socket using a series
728 of sendmsg() calls, each with the following control messages attached:
730 RXRPC_USER_CALL_ID - specifies the user ID for this call
732 MSG_MORE should be set in msghdr::msg_flags on all but the last message
733 for a particular call.
735 (10) The final ACK from the client will be posted for retrieval by recvmsg()
736 when it is received. It will take the form of a dataless message with two
737 control messages attached:
739 RXRPC_USER_CALL_ID - specifies the user ID for this call
740 RXRPC_ACK - indicates final ACK (no data)
742 MSG_EOR will be flagged to indicate that this is the final message for
745 (11) Up to the point the final packet of reply data is sent, the call can be
746 aborted by calling sendmsg() with a dataless message with the following
747 control messages attached:
749 RXRPC_USER_CALL_ID - specifies the user ID for this call
750 RXRPC_ABORT - indicates abort code (4 byte data)
752 Any packets waiting in the socket's receive queue will be discarded if
755 Note that all the communications for a particular service take place through
756 the one server socket, using control messages on sendmsg() and recvmsg() to
757 determine the call affected.
760 =========================
761 AF_RXRPC KERNEL INTERFACE
762 =========================
764 The AF_RXRPC module also provides an interface for use by in-kernel utilities
765 such as the AFS filesystem. This permits such a utility to:
767 (1) Use different keys directly on individual client calls on one socket
768 rather than having to open a whole slew of sockets, one for each key it
771 (2) Avoid having RxRPC call request_key() at the point of issue of a call or
772 opening of a socket. Instead the utility is responsible for requesting a
773 key at the appropriate point. AFS, for instance, would do this during VFS
774 operations such as open() or unlink(). The key is then handed through
775 when the call is initiated.
777 (3) Request the use of something other than GFP_KERNEL to allocate memory.
779 (4) Avoid the overhead of using the recvmsg() call. RxRPC messages can be
780 intercepted before they get put into the socket Rx queue and the socket
781 buffers manipulated directly.
783 To use the RxRPC facility, a kernel utility must still open an AF_RXRPC socket,
784 bind an address as appropriate and listen if it's to be a server socket, but
785 then it passes this to the kernel interface functions.
787 The kernel interface functions are as follows:
789 (*) Begin a new client call.
792 rxrpc_kernel_begin_call(struct socket *sock,
793 struct sockaddr_rxrpc *srx,
795 unsigned long user_call_ID,
798 rxrpc_notify_rx_t notify_rx,
801 unsigned int debug_id);
803 This allocates the infrastructure to make a new RxRPC call and assigns
804 call and connection numbers. The call will be made on the UDP port that
805 the socket is bound to. The call will go to the destination address of a
806 connected client socket unless an alternative is supplied (srx is
809 If a key is supplied then this will be used to secure the call instead of
810 the key bound to the socket with the RXRPC_SECURITY_KEY sockopt. Calls
811 secured in this way will still share connections if at all possible.
813 The user_call_ID is equivalent to that supplied to sendmsg() in the
814 control data buffer. It is entirely feasible to use this to point to a
815 kernel data structure.
817 tx_total_len is the amount of data the caller is intending to transmit
818 with this call (or -1 if unknown at this point). Setting the data size
819 allows the kernel to encrypt directly to the packet buffers, thereby
820 saving a copy. The value may not be less than -1.
822 notify_rx is a pointer to a function to be called when events such as
823 incoming data packets or remote aborts happen.
825 upgrade should be set to true if a client operation should request that
826 the server upgrade the service to a better one. The resultant service ID
827 is returned by rxrpc_kernel_recv_data().
829 intr should be set to true if the call should be interruptible. If this
830 is not set, this function may not return until a channel has been
831 allocated; if it is set, the function may return -ERESTARTSYS.
833 debug_id is the call debugging ID to be used for tracing. This can be
834 obtained by atomically incrementing rxrpc_debug_id.
836 If this function is successful, an opaque reference to the RxRPC call is
837 returned. The caller now holds a reference on this and it must be
840 (*) End a client call.
842 void rxrpc_kernel_end_call(struct socket *sock,
843 struct rxrpc_call *call);
845 This is used to end a previously begun call. The user_call_ID is expunged
846 from AF_RXRPC's knowledge and will not be seen again in association with
849 (*) Send data through a call.
851 typedef void (*rxrpc_notify_end_tx_t)(struct sock *sk,
852 unsigned long user_call_ID,
853 struct sk_buff *skb);
855 int rxrpc_kernel_send_data(struct socket *sock,
856 struct rxrpc_call *call,
859 rxrpc_notify_end_tx_t notify_end_rx);
861 This is used to supply either the request part of a client call or the
862 reply part of a server call. msg.msg_iovlen and msg.msg_iov specify the
863 data buffers to be used. msg_iov may not be NULL and must point
864 exclusively to in-kernel virtual addresses. msg.msg_flags may be given
865 MSG_MORE if there will be subsequent data sends for this call.
867 The msg must not specify a destination address, control data or any flags
868 other than MSG_MORE. len is the total amount of data to transmit.
870 notify_end_rx can be NULL or it can be used to specify a function to be
871 called when the call changes state to end the Tx phase. This function is
872 called with the call-state spinlock held to prevent any reply or final ACK
873 from being delivered first.
875 (*) Receive data from a call.
877 int rxrpc_kernel_recv_data(struct socket *sock,
878 struct rxrpc_call *call,
886 This is used to receive data from either the reply part of a client call
887 or the request part of a service call. buf and size specify how much
888 data is desired and where to store it. *_offset is added on to buf and
889 subtracted from size internally; the amount copied into the buffer is
890 added to *_offset before returning.
892 want_more should be true if further data will be required after this is
893 satisfied and false if this is the last item of the receive phase.
895 There are three normal returns: 0 if the buffer was filled and want_more
896 was true; 1 if the buffer was filled, the last DATA packet has been
897 emptied and want_more was false; and -EAGAIN if the function needs to be
900 If the last DATA packet is processed but the buffer contains less than
901 the amount requested, EBADMSG is returned. If want_more wasn't set, but
902 more data was available, EMSGSIZE is returned.
904 If a remote ABORT is detected, the abort code received will be stored in
905 *_abort and ECONNABORTED will be returned.
907 The service ID that the call ended up with is returned into *_service.
908 This can be used to see if a call got a service upgrade.
912 void rxrpc_kernel_abort_call(struct socket *sock,
913 struct rxrpc_call *call,
916 This is used to abort a call if it's still in an abortable state. The
917 abort code specified will be placed in the ABORT message sent.
919 (*) Intercept received RxRPC messages.
921 typedef void (*rxrpc_interceptor_t)(struct sock *sk,
922 unsigned long user_call_ID,
923 struct sk_buff *skb);
926 rxrpc_kernel_intercept_rx_messages(struct socket *sock,
927 rxrpc_interceptor_t interceptor);
929 This installs an interceptor function on the specified AF_RXRPC socket.
930 All messages that would otherwise wind up in the socket's Rx queue are
931 then diverted to this function. Note that care must be taken to process
932 the messages in the right order to maintain DATA message sequentiality.
934 The interceptor function itself is provided with the address of the socket
935 and handling the incoming message, the ID assigned by the kernel utility
936 to the call and the socket buffer containing the message.
938 The skb->mark field indicates the type of message:
941 =============================== =======================================
942 RXRPC_SKB_MARK_DATA Data message
943 RXRPC_SKB_MARK_FINAL_ACK Final ACK received for an incoming call
944 RXRPC_SKB_MARK_BUSY Client call rejected as server busy
945 RXRPC_SKB_MARK_REMOTE_ABORT Call aborted by peer
946 RXRPC_SKB_MARK_NET_ERROR Network error detected
947 RXRPC_SKB_MARK_LOCAL_ERROR Local error encountered
948 RXRPC_SKB_MARK_NEW_CALL New incoming call awaiting acceptance
950 The remote abort message can be probed with rxrpc_kernel_get_abort_code().
951 The two error messages can be probed with rxrpc_kernel_get_error_number().
952 A new call can be accepted with rxrpc_kernel_accept_call().
954 Data messages can have their contents extracted with the usual bunch of
955 socket buffer manipulation functions. A data message can be determined to
956 be the last one in a sequence with rxrpc_kernel_is_data_last(). When a
957 data message has been used up, rxrpc_kernel_data_consumed() should be
960 Messages should be handled to rxrpc_kernel_free_skb() to dispose of. It
961 is possible to get extra refs on all types of message for later freeing,
962 but this may pin the state of a call until the message is finally freed.
964 (*) Accept an incoming call.
967 rxrpc_kernel_accept_call(struct socket *sock,
968 unsigned long user_call_ID);
970 This is used to accept an incoming call and to assign it a call ID. This
971 function is similar to rxrpc_kernel_begin_call() and calls accepted must
972 be ended in the same way.
974 If this function is successful, an opaque reference to the RxRPC call is
975 returned. The caller now holds a reference on this and it must be
978 (*) Reject an incoming call.
980 int rxrpc_kernel_reject_call(struct socket *sock);
982 This is used to reject the first incoming call on the socket's queue with
983 a BUSY message. -ENODATA is returned if there were no incoming calls.
984 Other errors may be returned if the call had been aborted (-ECONNABORTED)
985 or had timed out (-ETIME).
987 (*) Allocate a null key for doing anonymous security.
989 struct key *rxrpc_get_null_key(const char *keyname);
991 This is used to allocate a null RxRPC key that can be used to indicate
992 anonymous security for a particular domain.
994 (*) Get the peer address of a call.
996 void rxrpc_kernel_get_peer(struct socket *sock, struct rxrpc_call *call,
997 struct sockaddr_rxrpc *_srx);
999 This is used to find the remote peer address of a call.
1001 (*) Set the total transmit data size on a call.
1003 void rxrpc_kernel_set_tx_length(struct socket *sock,
1004 struct rxrpc_call *call,
1007 This sets the amount of data that the caller is intending to transmit on a
1008 call. It's intended to be used for setting the reply size as the request
1009 size should be set when the call is begun. tx_total_len may not be less
1014 u64 rxrpc_kernel_get_rtt(struct socket *sock, struct rxrpc_call *call);
1016 Get the RTT time to the peer in use by a call. The value returned is in
1019 (*) Check call still alive.
1021 bool rxrpc_kernel_check_life(struct socket *sock,
1022 struct rxrpc_call *call,
1024 void rxrpc_kernel_probe_life(struct socket *sock,
1025 struct rxrpc_call *call);
1027 The first function passes back in *_life a number that is updated when
1028 ACKs are received from the peer (notably including PING RESPONSE ACKs
1029 which we can elicit by sending PING ACKs to see if the call still exists
1030 on the server). The caller should compare the numbers of two calls to see
1031 if the call is still alive after waiting for a suitable interval. It also
1032 returns true as long as the call hasn't yet reached the completed state.
1034 This allows the caller to work out if the server is still contactable and
1035 if the call is still alive on the server while waiting for the server to
1036 process a client operation.
1038 The second function causes a ping ACK to be transmitted to try to provoke
1039 the peer into responding, which would then cause the value returned by the
1040 first function to change. Note that this must be called in TASK_RUNNING
1043 (*) Get reply timestamp.
1045 bool rxrpc_kernel_get_reply_time(struct socket *sock,
1046 struct rxrpc_call *call,
1049 This allows the timestamp on the first DATA packet of the reply of a
1050 client call to be queried, provided that it is still in the Rx ring. If
1051 successful, the timestamp will be stored into *_ts and true will be
1052 returned; false will be returned otherwise.
1054 (*) Get remote client epoch.
1056 u32 rxrpc_kernel_get_epoch(struct socket *sock,
1057 struct rxrpc_call *call)
1059 This allows the epoch that's contained in packets of an incoming client
1060 call to be queried. This value is returned. The function always
1061 successful if the call is still in progress. It shouldn't be called once
1062 the call has expired. Note that calling this on a local client call only
1063 returns the local epoch.
1065 This value can be used to determine if the remote client has been
1066 restarted as it shouldn't change otherwise.
1068 (*) Set the maxmimum lifespan on a call.
1070 void rxrpc_kernel_set_max_life(struct socket *sock,
1071 struct rxrpc_call *call,
1072 unsigned long hard_timeout)
1074 This sets the maximum lifespan on a call to hard_timeout (which is in
1075 jiffies). In the event of the timeout occurring, the call will be
1076 aborted and -ETIME or -ETIMEDOUT will be returned.
1079 =======================
1080 CONFIGURABLE PARAMETERS
1081 =======================
1083 The RxRPC protocol driver has a number of configurable parameters that can be
1084 adjusted through sysctls in /proc/net/rxrpc/:
1088 The amount of time in milliseconds after receiving a packet with the
1089 request-ack flag set before we honour the flag and actually send the
1092 Usually the other side won't stop sending packets until the advertised
1093 reception window is full (to a maximum of 255 packets), so delaying the
1094 ACK permits several packets to be ACK'd in one go.
1098 The amount of time in milliseconds after receiving a new packet before we
1099 generate a soft-ACK to tell the sender that it doesn't need to resend.
1103 The amount of time in milliseconds after all the packets currently in the
1104 received queue have been consumed before we generate a hard-ACK to tell
1105 the sender it can free its buffers, assuming no other reason occurs that
1106 we would send an ACK.
1110 The amount of time in milliseconds after transmitting a packet before we
1111 transmit it again, assuming no ACK is received from the receiver telling
1114 (*) max_call_lifetime
1116 The maximum amount of time in seconds that a call may be in progress
1117 before we preemptively kill it.
1119 (*) dead_call_expiry
1121 The amount of time in seconds before we remove a dead call from the call
1122 list. Dead calls are kept around for a little while for the purpose of
1123 repeating ACK and ABORT packets.
1125 (*) connection_expiry
1127 The amount of time in seconds after a connection was last used before we
1128 remove it from the connection list. While a connection is in existence,
1129 it serves as a placeholder for negotiated security; when it is deleted,
1130 the security must be renegotiated.
1132 (*) transport_expiry
1134 The amount of time in seconds after a transport was last used before we
1135 remove it from the transport list. While a transport is in existence, it
1136 serves to anchor the peer data and keeps the connection ID counter.
1138 (*) rxrpc_rx_window_size
1140 The size of the receive window in packets. This is the maximum number of
1141 unconsumed received packets we're willing to hold in memory for any
1146 The maximum packet MTU size that we're willing to receive in bytes. This
1147 indicates to the peer whether we're willing to accept jumbo packets.
1149 (*) rxrpc_rx_jumbo_max
1151 The maximum number of packets that we're willing to accept in a jumbo
1152 packet. Non-terminal packets in a jumbo packet must contain a four byte
1153 header plus exactly 1412 bytes of data. The terminal packet must contain
1154 a four byte header plus any amount of data. In any event, a jumbo packet
1155 may not exceed rxrpc_rx_mtu in size.