+@node Streams and Threads
+@section Streams and Threads
+
+@cindex threads
+@cindex multi-threaded application
+Streams can be used in multi-threaded applications in the same way they
+are used in single-threaded applications. But the programmer must be
+aware of the possible complications. It is important to know about
+these also if the program one writes never use threads since the design
+and implementation of many stream functions is heavily influenced by the
+requirements added by multi-threaded programming.
+
+The POSIX standard requires that by default the stream operations are
+atomic. I.e., issuing two stream operations for the same stream in two
+threads at the same time will cause the operations to be executed as if
+they were issued sequentially. The buffer operations performed while
+reading or writing are protected from other uses of the same stream. To
+do this each stream has an internal lock object which has to be
+(implicitly) acquired before any work can be done.
+
+But there are situations where this is not enough and there are also
+situations where this is not wanted. The implicit locking is not enough
+if the program requires more than one stream function call to happen
+atomically. One example would be if an output line a program wants to
+generate is created by several function calls. The functions by
+themselves would ensure only atomicity of their own operation, but not
+atomicity over all the function calls. For this it is necessary to
+perform the stream locking in the application code.
+
+@comment stdio.h
+@comment POSIX
+@deftypefun void flockfile (FILE *@var{stream})
+@safety{@prelim{}@mtsafe{}@assafe{}@acunsafe{@aculock{}}}
+@c There's no way to tell whether the lock was acquired before or after
+@c cancellation so as to unlock only when appropriate.
+The @code{flockfile} function acquires the internal locking object
+associated with the stream @var{stream}. This ensures that no other
+thread can explicitly through @code{flockfile}/@code{ftrylockfile} or
+implicit through a call of a stream function lock the stream. The
+thread will block until the lock is acquired. An explicit call to
+@code{funlockfile} has to be used to release the lock.
+@end deftypefun
+
+@comment stdio.h
+@comment POSIX
+@deftypefun int ftrylockfile (FILE *@var{stream})
+@safety{@prelim{}@mtsafe{}@assafe{}@acunsafe{@aculock{}}}
+The @code{ftrylockfile} function tries to acquire the internal locking
+object associated with the stream @var{stream} just like
+@code{flockfile}. But unlike @code{flockfile} this function does not
+block if the lock is not available. @code{ftrylockfile} returns zero if
+the lock was successfully acquired. Otherwise the stream is locked by
+another thread.
+@end deftypefun
+
+@comment stdio.h
+@comment POSIX
+@deftypefun void funlockfile (FILE *@var{stream})
+@safety{@prelim{}@mtsafe{}@assafe{}@acunsafe{@aculock{}}}
+The @code{funlockfile} function releases the internal locking object of
+the stream @var{stream}. The stream must have been locked before by a
+call to @code{flockfile} or a successful call of @code{ftrylockfile}.
+The implicit locking performed by the stream operations do not count.
+The @code{funlockfile} function does not return an error status and the
+behavior of a call for a stream which is not locked by the current
+thread is undefined.
+@end deftypefun
+
+The following example shows how the functions above can be used to
+generate an output line atomically even in multi-threaded applications
+(yes, the same job could be done with one @code{fprintf} call but it is
+sometimes not possible):
+
+@smallexample
+FILE *fp;
+@{
+ @dots{}
+ flockfile (fp);
+ fputs ("This is test number ", fp);
+ fprintf (fp, "%d\n", test);
+ funlockfile (fp)
+@}
+@end smallexample
+
+Without the explicit locking it would be possible for another thread to
+use the stream @var{fp} after the @code{fputs} call return and before
+@code{fprintf} was called with the result that the number does not
+follow the word @samp{number}.
+
+From this description it might already be clear that the locking objects
+in streams are no simple mutexes. Since locking the same stream twice
+in the same thread is allowed the locking objects must be equivalent to
+recursive mutexes. These mutexes keep track of the owner and the number
+of times the lock is acquired. The same number of @code{funlockfile}
+calls by the same threads is necessary to unlock the stream completely.
+For instance:
+
+@smallexample
+void
+foo (FILE *fp)
+@{
+ ftrylockfile (fp);
+ fputs ("in foo\n", fp);
+ /* @r{This is very wrong!!!} */
+ funlockfile (fp);
+@}
+@end smallexample
+
+It is important here that the @code{funlockfile} function is only called
+if the @code{ftrylockfile} function succeeded in locking the stream. It
+is therefore always wrong to ignore the result of @code{ftrylockfile}.
+And it makes no sense since otherwise one would use @code{flockfile}.
+The result of code like that above is that either @code{funlockfile}
+tries to free a stream that hasn't been locked by the current thread or it
+frees the stream prematurely. The code should look like this:
+
+@smallexample
+void
+foo (FILE *fp)
+@{
+ if (ftrylockfile (fp) == 0)
+ @{
+ fputs ("in foo\n", fp);
+ funlockfile (fp);
+ @}
+@}
+@end smallexample
+
+Now that we covered why it is necessary to have these locking it is
+necessary to talk about situations when locking is unwanted and what can
+be done. The locking operations (explicit or implicit) don't come for
+free. Even if a lock is not taken the cost is not zero. The operations
+which have to be performed require memory operations that are safe in
+multi-processor environments. With the many local caches involved in
+such systems this is quite costly. So it is best to avoid the locking
+completely if it is not needed -- because the code in question is never
+used in a context where two or more threads may use a stream at a time.
+This can be determined most of the time for application code; for
+library code which can be used in many contexts one should default to be
+conservative and use locking.
+
+There are two basic mechanisms to avoid locking. The first is to use
+the @code{_unlocked} variants of the stream operations. The POSIX
+standard defines quite a few of those and @theglibc{} adds a few
+more. These variants of the functions behave just like the functions
+with the name without the suffix except that they do not lock the
+stream. Using these functions is very desirable since they are
+potentially much faster. This is not only because the locking
+operation itself is avoided. More importantly, functions like
+@code{putc} and @code{getc} are very simple and traditionally (before the
+introduction of threads) were implemented as macros which are very fast
+if the buffer is not empty. With the addition of locking requirements
+these functions are no longer implemented as macros since they would
+expand to too much code.
+But these macros are still available with the same functionality under the new
+names @code{putc_unlocked} and @code{getc_unlocked}. This possibly huge
+difference of speed also suggests the use of the @code{_unlocked}
+functions even if locking is required. The difference is that the
+locking then has to be performed in the program:
+
+@smallexample
+void
+foo (FILE *fp, char *buf)
+@{
+ flockfile (fp);
+ while (*buf != '/')
+ putc_unlocked (*buf++, fp);
+ funlockfile (fp);
+@}
+@end smallexample
+
+If in this example the @code{putc} function would be used and the
+explicit locking would be missing the @code{putc} function would have to
+acquire the lock in every call, potentially many times depending on when
+the loop terminates. Writing it the way illustrated above allows the
+@code{putc_unlocked} macro to be used which means no locking and direct
+manipulation of the buffer of the stream.
+
+A second way to avoid locking is by using a non-standard function which
+was introduced in Solaris and is available in @theglibc{} as well.
+
+@comment stdio_ext.h
+@comment GNU
+@deftypefun int __fsetlocking (FILE *@var{stream}, int @var{type})
+@safety{@prelim{}@mtsafe{@mtsrace{:stream}}@asunsafe{@asulock{}}@acsafe{}}
+@c Changing the implicit-locking status of a stream while it's in use by
+@c another thread may cause a lock to be implicitly acquired and not
+@c released, or vice-versa. This function should probably hold the lock
+@c while changing this setting, to make sure we don't change it while
+@c there are any concurrent uses. Meanwhile, callers should acquire the
+@c lock themselves to be safe, and even concurrent uses with external
+@c locking will be fine, as long as functions that require external
+@c locking are not called without holding locks.
+
+The @code{__fsetlocking} function can be used to select whether the
+stream operations will implicitly acquire the locking object of the
+stream @var{stream}. By default this is done but it can be disabled and
+reinstated using this function. There are three values defined for the
+@var{type} parameter.
+
+@vtable @code
+@item FSETLOCKING_INTERNAL
+The stream @code{stream} will from now on use the default internal
+locking. Every stream operation with exception of the @code{_unlocked}
+variants will implicitly lock the stream.
+
+@item FSETLOCKING_BYCALLER
+After the @code{__fsetlocking} function returns the user is responsible
+for locking the stream. None of the stream operations will implicitly
+do this anymore until the state is set back to
+@code{FSETLOCKING_INTERNAL}.
+
+@item FSETLOCKING_QUERY
+@code{__fsetlocking} only queries the current locking state of the
+stream. The return value will be @code{FSETLOCKING_INTERNAL} or
+@code{FSETLOCKING_BYCALLER} depending on the state.
+@end vtable
+
+The return value of @code{__fsetlocking} is either
+@code{FSETLOCKING_INTERNAL} or @code{FSETLOCKING_BYCALLER} depending on
+the state of the stream before the call.
+
+This function and the values for the @var{type} parameter are declared
+in @file{stdio_ext.h}.
+@end deftypefun
+
+This function is especially useful when program code has to be used
+which is written without knowledge about the @code{_unlocked} functions
+(or if the programmer was too lazy to use them).
+
+@node Streams and I18N
+@section Streams in Internationalized Applications
+
+@w{ISO C90} introduced the new type @code{wchar_t} to allow handling
+larger character sets. What was missing was a possibility to output
+strings of @code{wchar_t} directly. One had to convert them into
+multibyte strings using @code{mbstowcs} (there was no @code{mbsrtowcs}
+yet) and then use the normal stream functions. While this is doable it
+is very cumbersome since performing the conversions is not trivial and
+greatly increases program complexity and size.
+
+The Unix standard early on (I think in XPG4.2) introduced two additional
+format specifiers for the @code{printf} and @code{scanf} families of
+functions. Printing and reading of single wide characters was made
+possible using the @code{%C} specifier and wide character strings can be
+handled with @code{%S}. These modifiers behave just like @code{%c} and
+@code{%s} only that they expect the corresponding argument to have the
+wide character type and that the wide character and string are
+transformed into/from multibyte strings before being used.
+
+This was a beginning but it is still not good enough. Not always is it
+desirable to use @code{printf} and @code{scanf}. The other, smaller and
+faster functions cannot handle wide characters. Second, it is not
+possible to have a format string for @code{printf} and @code{scanf}
+consisting of wide characters. The result is that format strings would
+have to be generated if they have to contain non-basic characters.
+
+@cindex C++ streams
+@cindex streams, C++
+In the @w{Amendment 1} to @w{ISO C90} a whole new set of functions was
+added to solve the problem. Most of the stream functions got a
+counterpart which take a wide character or wide character string instead
+of a character or string respectively. The new functions operate on the
+same streams (like @code{stdout}). This is different from the model of
+the C++ runtime library where separate streams for wide and normal I/O
+are used.
+
+@cindex orientation, stream
+@cindex stream orientation
+Being able to use the same stream for wide and normal operations comes
+with a restriction: a stream can be used either for wide operations or
+for normal operations. Once it is decided there is no way back. Only a
+call to @code{freopen} or @code{freopen64} can reset the
+@dfn{orientation}. The orientation can be decided in three ways:
+
+@itemize @bullet
+@item
+If any of the normal character functions is used (this includes the
+@code{fread} and @code{fwrite} functions) the stream is marked as not
+wide oriented.
+
+@item
+If any of the wide character functions is used the stream is marked as
+wide oriented.
+
+@item
+The @code{fwide} function can be used to set the orientation either way.
+@end itemize
+
+It is important to never mix the use of wide and not wide operations on
+a stream. There are no diagnostics issued. The application behavior
+will simply be strange or the application will simply crash. The
+@code{fwide} function can help avoiding this.
+
+@comment wchar.h
+@comment ISO
+@deftypefun int fwide (FILE *@var{stream}, int @var{mode})
+@safety{@prelim{}@mtsafe{}@asunsafe{@asucorrupt{}}@acunsafe{@aculock{}}}
+@c Querying is always safe, but changing the stream when it's in use
+@c upthread may be problematic. Like most lock-acquiring functions,
+@c this one may leak the lock if canceled.
+
+The @code{fwide} function can be used to set and query the state of the
+orientation of the stream @var{stream}. If the @var{mode} parameter has
+a positive value the streams get wide oriented, for negative values
+narrow oriented. It is not possible to overwrite previous orientations
+with @code{fwide}. I.e., if the stream @var{stream} was already
+oriented before the call nothing is done.
+
+If @var{mode} is zero the current orientation state is queried and
+nothing is changed.
+
+The @code{fwide} function returns a negative value, zero, or a positive
+value if the stream is narrow, not at all, or wide oriented
+respectively.
+
+This function was introduced in @w{Amendment 1} to @w{ISO C90} and is
+declared in @file{wchar.h}.
+@end deftypefun
+
+It is generally a good idea to orient a stream as early as possible.
+This can prevent surprise especially for the standard streams
+@code{stdin}, @code{stdout}, and @code{stderr}. If some library
+function in some situations uses one of these streams and this use
+orients the stream in a different way the rest of the application
+expects it one might end up with hard to reproduce errors. Remember
+that no errors are signal if the streams are used incorrectly. Leaving
+a stream unoriented after creation is normally only necessary for
+library functions which create streams which can be used in different
+contexts.
+
+When writing code which uses streams and which can be used in different
+contexts it is important to query the orientation of the stream before
+using it (unless the rules of the library interface demand a specific
+orientation). The following little, silly function illustrates this.
+
+@smallexample
+void
+print_f (FILE *fp)
+@{
+ if (fwide (fp, 0) > 0)
+ /* @r{Positive return value means wide orientation.} */
+ fputwc (L'f', fp);
+ else
+ fputc ('f', fp);
+@}
+@end smallexample
+
+Note that in this case the function @code{print_f} decides about the
+orientation of the stream if it was unoriented before (will not happen
+if the advise above is followed).
+
+The encoding used for the @code{wchar_t} values is unspecified and the
+user must not make any assumptions about it. For I/O of @code{wchar_t}
+values this means that it is impossible to write these values directly
+to the stream. This is not what follows from the @w{ISO C} locale model
+either. What happens instead is that the bytes read from or written to
+the underlying media are first converted into the internal encoding
+chosen by the implementation for @code{wchar_t}. The external encoding
+is determined by the @code{LC_CTYPE} category of the current locale or
+by the @samp{ccs} part of the mode specification given to @code{fopen},
+@code{fopen64}, @code{freopen}, or @code{freopen64}. How and when the
+conversion happens is unspecified and it happens invisible to the user.
+
+Since a stream is created in the unoriented state it has at that point
+no conversion associated with it. The conversion which will be used is
+determined by the @code{LC_CTYPE} category selected at the time the
+stream is oriented. If the locales are changed at the runtime this
+might produce surprising results unless one pays attention. This is
+just another good reason to orient the stream explicitly as soon as
+possible, perhaps with a call to @code{fwide}.
+