(import (rnrs))                     ;R6RS
(import (rnrs bytevectors))         ;R6RS
(import (scheme base))              ;R7RS

(string->utf8 string)
(string->utf8 string start)           ;R7RS
(string->utf8 string start end)       ;R7RS
(utf8->string bytevector)
(utf8->string bytevector start)       ;R7RS
(utf8->string bytevector start end)   ;R7RS
;; The following procedures are in R6RS and are absent from R7RS.
(string->utf16 string)
(string->utf16 string endianness)
(string->utf32 string)
(string->utf32 string endianness)
(utf16->string bytevector endianness)
(utf16->string bytevector endianness endianness-mandatory?)
(utf32->string bytevector endianness)
(utf32->string bytevector endianness endianness-mandatory?)

The string->utf8, string->utf16, and string->utf32 procedures return a bytevector that contains an encoding of string (with no byte-order mark). The utf8->string, utf16->string, and utf32->string procedures return a string whose character sequence is encoded by bytevector.

utf8->string, string->utf8

These procedures use the UTF-8 encoding.

string->utf16

This procedure encodes according to UTF-16BE (default) or UTF-16LE.

string->utf32

This procedure encodes according to UTF-32BE (default) or UTF-32LE.

utf16->string

This procedure decodes according to UTF-16, UTF-16BE, UTF-16LE, or a fourth encoding scheme that differs from all of those, as in the description of endianness-mandatory? below.

utf32->string

This procedure decodes according to UTF-32, UTF-32BE, UTF-32LE, or a fourth encoding scheme that differs from all of those, as in the description of endianness-mandatory? below.

Endianness

If endianness is specified, it must be the symbol big or the symbol little. This differs from other bytevector procedures that can support additional implementation-defined endianness values. See native-endianness(3scm) for a definition of endianness. The default endianness for the string-> procedures is big. The endianness concept is not applicable to UTF-8.

Byte-order marks

A UTF-16 BOM is either the sequence of bytes #xFE, #xFF specifying big and UTF-16BE, or #xFF, #xFE specifying little and UTF-16LE.

A UTF-32 BOM is either the sequence of bytes #x00, #x00, #xFE, #xFF specifying big and UTF-32BE, or #xFF, #xFE, #x00, #x00, specifying little and UTF-32LE.

A UTF-8 BOM is the sequence of bytes #xEF, #xBB, #xBF. Neither R6RS nor R7RS mentions the UTF-8 BOM. It does not specify an endianness, but is sometimes used as a magic string to mark UTF-8 text.

The endianness-mandatory? argument (the fourth encoding scheme)

If endianness-mandatory? is absent or #f, then utf16->string and utf32->string determine the endianness according to a BOM at the beginning of bytevector if a BOM is present; in this case, the BOM is not decoded as a character. Also in this case, if no BOM is present, endianness specifies the endianness of the encoding. If endianness-mandatory? is a true value, endianness specifies the endianness of the encoding, and any BOM in the encoding is decoded as a regular character.

Decoding errors

If an invalid or incomplete character encoding is encountered, then the replacement character U+FFFD is appended to the string being generated, an appropriate number of bytes are ignored, and decoding continues with the following bytes.

R7RS

R7RS provides two extra arguments for restricting the transcoding operation to only a part of the input. Only UTF-8 is provided (and possibly only a small subset, see the errors section).

;; The #vu8() syntax is used in R6RS. R7RS uses #u8() instead.
(utf8->string #vu8(#x41))
          => "A"
(string->utf8 "λ")
          => #vu8(#xCE #xBB)

The number of bytes skipped when decoding an invalid or incomplete character differs between implementations. Relying on the precise number of bytes skipped, or the number of replacement characters used, is not portable.

Some implementations do not return newly allocated strings or bytevectors if they are empty, as they have a single copy of each.

For R7RS, also see the note below in the errors section.