Streams

This page has been automatically translated using the Google Translate API services. We are working on improving texts. Thank you for your understanding and patience.

Water can flow or creep or drip or crash, be water my friend.. Bruce Lee

Functions

Types and Constants

A stream is a data flow that runs from a source to a destination. Think of a phone call. We have an origin (the person who speaks), a destination (the person who listens) and a channel (the line itself). In programming, the stream is the equivalent to the telephone line, it is the pipe that joins the application with a data source or destination (Figure 1) and through which binary information, bit sequences, run. As with any other communication channel, the information is volatile, available for a very limited time. Once it reaches the receiver, it disappears.

In essence, there are three elementary operations to perform when working with streams: Create the channel, read data and write data.

• Use stm_memory to create a read/write memory stream.
• Use stm_read_r32 to read a float from the stream.
• Use stm_write_r32 to write a float to the stream.
• Use stm_close to close the channel and free up resources (destructor).

1. Stream Types

Actually, it is more correct to talk about types of extremes (origin and destination) than of stream types. From the perspective of the programmer, a stream is an abstract type that presents the same functionality regardless of the ends it connects. Therefore, when talking about stream types we are referring to the type of constructor.

1.1. File stream

• Use stm_from_file to open a file and read from it.
• Use stm_to_file to create a file and write to it.
• Use stm_append_file to add content to an existing file.

In File streams (Figure 2), the source is the process memory and the destination is a disk file. The opposite can also happen: that the source is the file and the destination the memory, it will depend on how we create the channel. It will not be possible to perform write operations on an open file for reading or vice versa (Listing 1). Files and directories.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

Stream *stm = stm_to_file("C:\Users\user\john\out.txt", NULL);
if (stm != NULL)
{
    stm_writef(stm, "One ");
    stm_writef(stm, "Two ");
    stm_writef(stm, "Three");
    stm_writef(stm, ".");
    stm_close(&stm);
    // 'out.txt' is closed = "One Two Three."
}

1.2. Socket stream

• Use stm_socket to create a communication channel with a remote process.

A socket is a communication channel between two processes over the Internet (Figure 3). Unlike file streams, sockets allow bidirectional full-duplex) communication, that is, both ends can send and receive information. The sequence of message exchange between partners is determined by the protocol (Listing 2), being HTTP, FTP, SMTP or LDAP some of the most used for Internet transmissions. See Sockets.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

uint32_t ip = bsocket_url_ip("www.myserver.com", NULL);
Socket *socket = bsocket_connect(ip, 80, 0, NULL);
if (socket != NULL)
{
    Stream *stm = stm_socket(socket);
    stm_writef(stm, "GET /mypage.html HTTP/1.1\r\n");
    stm_writef(stm, "Host: www.myserver.com\r\n");
    stm_writef(stm, "\r\n");
    stm_lines(line, stm)
        bstd_printf(line);
        bstd_printf("\n");
    stm_next(line, stm)

    // Socket will be closed too
    stm_close(&stm);
}

1.3. Block stream

• Use stm_from_block to read data from a memory block.

Block streams are used to read formatted data from a generic memory block (Figure 4) (Listing 3). This memory area is considered read-only and will not be modified, so write operations will not be allowed in this type of stream. When the end of the block is reached, the ekSTEND state will be activated.

1
2
3
4
5
6
7
8
9

const byte_t *data = ...
uint32_t size = ...
ArrPt(String) *lines = arrpt_create(String);
Stream *stm = stm_from_block(data, size);
while(stm_state(stm) == ekSTOK)
{
    String *line = str_c(stm_read_line(stm));
    arrpt_append(lines, line);
}

1.4. Memory stream

• Use stm_memory to create a stream in memory.
• Use stm_buffer to access the internal buffer.
• Use stm_buffer_size to get the size of the internal buffer.

Memory streams are read/write channels that allow implementing the producer/consumer model (Figure 5). First, the information reaches the stream through write operations and is stored in an internal memory buffer. Subsequently, said information can be read by another function, thread or process. After each reading the information read will disappear from the channel. The concept is similar to that of IPC-pipes, except that there is no size limit for the buffer, but it will grow on demand. Read and write operations can be done simultaneously depending on the established protocol.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

// Main thread
Stream *stm = stm_memory(2048);
while(true)
{
    UserEvent event;
    if (event_incomming(&event))
    {
        bmutex_lock(mutex);
        stm_write_u32(stm, event.code);
        stm_write_u32(stm, event.userid);
        str_write(stm, str_event.name);
        bmutex_unlock(mutex);

        if (event.last == TRUE)
            break;
    }
}

bmutex_lock(mutex);
stm_close(&stm);
bmutex_unlock(mutex);

...
// Other thread
bool_t next = TRUE;
while(next)
{

    bmutex_lock(mutex);
    if (stm != NULL)
    {
        if (stm_buffer_size(stm) > 0)
        {
            UserEvent event;
            event.code = stm_write_u32(stm);
            event.userid = stm_write_u32(stm);
            str_event.name = str_read(stm);
            ...
        }
    }
    else
    {
        next = FALSE;
    }
    bmutex_unlock(mutex);
    bthread_sleep(50);
}

Although this type of stream supports read and write operations it is not considered full-duplex. The reading is done on previously written data, but cannot "answer" the interlocutor. It is not a "conversation".

1.5. Standard stream

• kSTDIN: To read from the standard imput.
• kSTDOUT: To write in standard output.
• kSTDERR: To write in the error output.

The Standard I/O can be managed by streams using three predefined objects (Figure 6) (Listing 5). These objects are created when the program starts and will be automatically released when finished.

1
2
3
4
5

real64_t value;
const char_t *line;
value = stm_read_r64(kSTDIN);
line = stm_read_line(kSTDIN);
stm_printf(kSTDOUT, "Value = %.4f", value);

1.6. Null stream

• Use kDEVNULL to write to a sink that will ignore all received data.

Sometimes it can be useful to have a "sink" that ignores all write operations (Figure 7) (Listing 6). Think of debugging tasks where we want to activate or deactivate the output of information but deleting or commenting on the code is cumbersome. The idea is similar to the Unix /dev/null.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

#if defined __ASSERTS__
Stream *stm = kSTDOUT;
#else
Stream *stm = kDEVNULL;
#endif

...
i_large_dump_func(obj1, stm);
...
// More debug functions
stm_printf(stm, "More debug data...\n");
...
i_other_dump_func(obj2, stm);

Cannot read from kDEVNULL.

1.7. Binary stream

• Use stm_read_u32 to read a 32-bit unsigned integer.
• Use stm_write_r64 to write a real 64bits (double).
• Use stm_write_bool to write a boolean.

Generic binary data always travels through a stream as bytes. How these data are interpreted depends on the interlocutors and their communication protocol. But by emphasizing "binary data" we mean that numeric values are written to the channel as they appear in the CPU registers using binary, two's complement, or IEEE754 (Figure 8) code. In multi-byte types we must take into account the Byte order. In stream.h several functions are defined to read and write binary types.

1.8. Text stream

• Use stm_printf to write text in a stream.
• Use stm_read_char to read a character from a stream.
• Use stm_read_line to read a text line from a stream.
• Use stm_col to get the column number of the last character read.
• Use stm_row to get the row number of the last character read.

Text streams are a particular case where the binary information is assumed to represent Unicode character codes (codepoints) (Figure 9) (Listing 7). This means that the content of the stream will be readable directly by a human, but it will require a post-processing (parsing) in destination to interpret these texts and translate them into binary. You do not have to do anything special when creating a stream to indicate that it is of type text, you just have to use the appropriate functions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

Stream *stm = stm_from_file("/home/fran/Desktop/text.txt", NULL);
const char_t *line = stm_read_line(stm);
while(line != NULL)
{
    // Do something with 'line'
    textview_writef(text, line);
    textview_writef(text, "\n");

    // Read next line
    line = stm_read_line(stm);
}

stm_close(&stm);

stm_read_line and other reading functions will always return the text in UTF8. But if the data inside the stream were in another format, we must use stm_set_read_utf, in order to carry out the conversion correctly. See UTF encodings.

On the other hand, stm_printf also receives the text in UTF8, but the receiver may need it in another format. We will use stm_set_write_utf to set the output encoding. We will write in UTF8, but the channel will be sent in UTF16 or UTF32.

Streams do not have to be "pure" text or binary. They can combine both types of representations.

2. Tokens

• Use stm_read_token to read a token.
• Use stm_token_lexeme to obtain the string associated with the last token read.
• Use stm_read_r64_tok to read a real64_t from text.
• Use stm_token_col to get the column of the last token.
• Use stm_token_row to get the row of the last token.

When reading from text streams, an interpretation (parsing) of the content is necessary in order to transfer the data to memory variables (in binary). The first step is to break the text into symbols (or words) called tokens. Internally, the streams incorporate a simple lexical analyzer that recognizes the tokens of the C language, very common in countless grammars and file formats (Figure 10). It is implemented as a finite state machine and will greatly facilitate the processing of these text flows. In (Listing 8) we see the code necessary to read one by one all the tokens from a .c file. We have the result of processing the file (Listing 9) in (Listing 10).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

Stream *stm = stm_from_file("source.c", NULL);
token_t token;

while ((token = stm_read_token(lex)) != ekTEOF)
{
    switch (token) {
    case ekTIDENT:
        // It's an IDENTIFIER
        ...

    case ekTREAL:
        // It's a REAL NUMBER
        ...
    }
}

1
2
3
4
5
6
7

void func(int a)
{
    int i;
    char *str = "Hello";

    i = 5 + 2.5;
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

Token           Lexeme
-----           ------
ekTIDENT        "void"
ekTIDENT        "func"
ekTOPENPAR      "("
ekTIDENT        "int"
ekTIDENT        "a"
ekTCLOSPAR      ")"
ekTOPENCURL     "{"
ekTIDENT        "int"
ekTIDENT        "i"
ekTSCOLON       ";"
ekTIDENT        "char"
ekTASTERK       "*"
ekTIDENT        "str"
ekTEQUALS       "="
ekTSTRING       ""Hello""
ekTSCOLON       ";"
ekTIDENT        "i"
ekTEQUALS       "="
ekTINTEGER      "5"
ekTPLUS         "+"
ekTREAL         "2.5"
ekTSCOLON       ";"

2.1. Identifiers

An identifier is an alphanumeric "word" that must begin with a letter or '_' and contains numbers, letters, or '_'. It is used to name variables, functions, reserved words, etc. They do not allow spaces or symbols. (Listing 11) (Figure 11).

1
2

OK: while cos _reSult a56B _06_t aG h9 _12AcVb
NO: 045 ?er "_5G _tg(

Certain identifiers can be reserved to act as language keywords. For example for, while or if are C keywords and cannot be used for the naming of variables or functions. Being general purpose, our scanner does not recognize any type of reserved word, but must be expressly tagged after reading the token (Listing 12).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

while ((token = stm_read_token(stm)) != ekTEOF)
{
    if (token == ekTIDENT)
    {
        const char_t *lex = stm_token_lexeme(stm, NULL);

        if (str_equ_c(lex, "while") == TRUE)
            token = ekTRESERVED;
    }
}

2.2. Strings

A text string is a series of Unicode characters enclosed in quotation marks ("") (Figure 12). The parser recognizes C escape sequences to represent non-printable codes or unavailable characters on the keyboard (Listing 13).

• Use stm_token_escapes to make escape sequences effective when reading strings.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

\a 07 Alert (Beep, Bell) (added in C89) \b 08 Backspace \f 0C Formfeed Page Break \n 0A Newline (Line Feed) \r 0D Carriage Return \t 09 Horizontal Tab \v 0B Vertical Tab \\ 5C Backslash \' 27 Single quotation mark \" 22 Double quotation mark \? 3F Question mark (used to avoid trigraphs) \nnn Octal number \xhh Hexadecimal number \Uhhhhhhhh Unicode code point \uhhhh Unicode code point

2.3. Numbers

In the case of numerical tokens the thing is complicated a bit due to the different numerical bases and the exponential representation of real numbers (Figure 13). We briefly summarize it, although it is common to many programming languages (C included).

• If the number starts with 0 it will be considered octal (base 8), therefore, the following digits are limited to 0-7, eg: 043, 001, 0777.
• If the number starts with 0x will be considered hexadecimal (base 16) with digits 0-9 a-f A-F, eg: 0x4F, 0XAA5, 0x01EAC.
• At the moment a decimal point appears '.' will be considered real number. A point at starting is valid, eg: .56.
• An integer or real number allows exponential notation with the character 'e' ('E'), eg: 12.4e2, .56e3, 1e4.

2.4. Symbols

The symbols are single-character tokens that represent almost all US-ASCII punctuation marks and are often used as operators, separators or limiters within grammars. (Listing 14) (Figure 14).

1

< > , . ; : ( ) [ ] { } + - * = $ % # & ' " ^ ! ? | / \ @

2.5. Comments

By default, C /* Comment */ or C++ // Comment are ignored by stm_read_token.

• Use stm_token_comments so that it returns ekTSLCOM or ekTMLCOM if it finds any.
• Use stm_token_spaces to return ekTSPACE when it finds blank spaces.

3. Stream advantages

Although it is possible to read or write directly to the I/O channels (Memory, Files and directories, Sockets, Standard I/O), do it through Stream objects has certain advantages. Therefore, we recommend using them instead of low-level APIs for the following reasons:

3.1. Unify serialization

Streams offer a uniform interface, regardless of the origin and destination of the data (Figure 15). For the object serialization, we just have to write a reader and a writer, without worrying if the object will be saved to disk, transmitted over the Internet or stored temporarily in memory (Listing 15).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

typedef struct _product_t
{
    type_t type;
    String *code;
    String *description;
    Image *image64;
    real32_t price;
} Product;

void product_write(Stream *stm, Product *product)
{
    stm_write_enum(stm, product->type, type_t);
    str_write(stm, product->code);
    str_write(stm, product->description);
    image_write(stm, product->image64);
    stm_write_r32(stm, product->price);
}

void product_read(Stream *stm, Product *product)
{
    product->type = stm_read_enum(stm, type_t);
    product->code = str_read(stm);
    product->description = str_read(stm);
    product->image64 = image_read(stm);
    product->price = stm_read_r32(stm);
}

3.2. More elegance

The I/O channels only work with byte blocks. Streams implement high-level functions for texts and binary types. This will make our code much more readable. (Listing 16).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

void product_write(File *file, Product *product)
{
    uint32_t size = str_len(product->description);
    const char_t *data = tc(product->description);
    bfile_write(file, (byte_t*)&product->id, sizeof(uint32_t), NULL, NULL);
    bfile_write(file, (byte_t*)&product->price, sizeof(real64_t), NULL, NULL);
    bfile_write(file, (byte_t*)&size, sizeof(uint32_t), NULL, NULL);
    bfile_write(file, (byte_t*)data, size, NULL, NULL);
}

void product_write(Stream *stream, Product *product)
{
    stm_write_u32(stream, product->id);
    stm_write_r64(stream, product->price);
    str_write(stream, product->description);
}

3.3. Higher productivity

Related to the previous one, streams can "parse" text strings directly. You can get characters, words or lines without having to scan the entry character by character (Listing 17).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

String *getline(File *file)
{
    /* Potentially unsafe. */
    /* Risk of buffer overflow. */
    char_t buffer[MAXBUFF];
    uint32_t i = 0;
    char_t c;

    bfile_read(file, (byte_t*)&c, 1, NULL, NULL);
    while (c != '\n')
    {
        buffer[i] = c;
        i += 1;
        bfile_read(file, (byte_t*)&c, 1, NULL, NULL);
    }

    buffer[i] = '\0';
    return str_c(buffer);
}

String *getline(Stream *stream)
{
    /* Totally safe. */
    /* 'line' is managed by dynamic cache. */
    const char_t *line = stm_read_line(stream);
    return str_c(line);
}

3.4. Higher performance

File streams and socket streams implement an internal cache. This allows less access to the channel with a higher volume of data, which means faster processing speed. (Figure 16).

• Use stm_flush to clear the cache and dump the data in the channel.

3.5. Byte order

• Use stm_set_write_endian to establish the endianness of the output channel. The data will pass from endian CPU to Stream endian before being written.
• Use stm_set_read_endian to establish the endianness of the input channel. The data will pass from Stream endian to CPU endian at the time of being read.

When reading or writing binary data from an I/O channel, special attention must be paid to the order of the bytes in 16, 32 or 64 bit data types, which is known as endianness. On litte endian machines, as is the case with the Intel x86/x64 family processors, the lowest order byte will be located at the lowest memory address. In the case of the big endian (Motorola 68000, PowerPC) it happens on the contrary, it will go in the highest. For example, if we write a 32-bit integer in a file or socket from a little endian machine and read it from a big endian, the data will be corrupted by altering the internal order of bits (Figure 17). The Stream objects automatically adjust the endianness in each read/write operation. Default is set ekLITEND, except in sockets that will be ekBIGEND for being the accepted agreement for network communications. However, it can be changed if necessary.

Endianness does not influence UTF-8 text strings, but it does in the UTF-16 and UTF-32.

4. Stream state

• Use stm_state to know the current status of the channel.
• Use stm_file_err to get extended error information on disk streams.
• Use stm_sock_err to get extended error information in sockets.
• Use stm_corrupt to mark a stream as ekSTCORRUPT. Sometimes it is the application itself that detects that the data is not correct (eg out of range).
• Use stm_bytes_written to get the total number of bytes written to the stream.
• Use stm_bytes_readed to get the total number of bytes read from the stream.

A stream can be affected by two types of problems. On the one hand the data corruption that is evident when we read binary data from the stream (Listing 18). A clear example would be to read a Boolean by stm_read_bool and get a value of 129 when obviously this value should be 0 (TRUE) or 1 (FALSE). Typically, a stream becomes corrupted due to lack of coordination between writer and reader and is usually due to a programming error. This situation should be resolved by debugging and correcting the serialization of objects or reviewing the data protocol. On the other hand, there may be "physical" errors in the channel (file deleted, loss of Internet connection, permissions, etc.). In both cases, the stream will be blocked and subsequent read or write operations that we carry out on it will be ignored. We can also ask the total number of bytes read and/or written in the channel, in case we need to know if there is information available for reading.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

uint32_t nw = stm_bytes_written(stm);
uint32_t nr = stm_bytes_readed(stm);
if (nw - nr > 0)
{
    if (stm_state(stm) == ekSTOK)
    {
        uint32_t v1 = stm_read_u32(stm);
        real32_t v2 = stm_read_r32(stm);
        ...
    }
    else
    {
        // Error in stream
    }
}
else
{
    // No data in stream
}

kSTDIN

Stream* kSTDIN;

Stream connected to the standard input stdin.

kSTDOUT

Stream* kSTDOUT;

Stream connected to standard output stdout.

kSTDERR

Stream* kSTDERR;

Stream connected to error output stderr.

kDEVNULL

Stream* kDEVNULL;

Null write stream. All content sent through this channel will be ignored.

stm_from_block ()

Create a read stream from an existing memory block.

Stream*
stm_from_block(const byte_t *data,
               const uint32_t size);

Return

The stream.

Remarks

The original block will not be modified (read only). When the end of the block is reached stm_state will return ekSTEND. Block stream.

stm_memory ()

Create a read/write memory stream.

Stream*
stm_memory(const uint32_t size);

Return

The stream.

Remarks

It can be used as an internal pipeline for the information exchange between functions or threads. It behaves like a FIFO (First In Fist Out) buffer. For multi-threaded access you must be protected with a Mutex. Memory stream.

stm_from_file ()

Create a stream to read from a file on disk.

Stream*
stm_from_file(const char_t *pathname,
              ferror_t *error);

Return

The stream or NULL if the file opening fails.

Remarks

File stream.

stm_to_file ()

Create a stream to write data to a file on disk.

Stream*
stm_to_file(const char_t *pathname,
            ferror_t *error);

Return

The stream or NULL if file creation fails.

Remarks

If the file already exists it will be overwritten. File stream.

stm_append_file ()

Create a stream to write data to the end of an existing file.

Stream*
stm_append_file(const char_t *pathname,
                ferror_t *error);

Return

The stream or NULL if the file opening fails.

Remarks

It will fail if the file does not exist (do not create it). File stream.

stm_socket ()

Create a stream from a socket.

Stream*
stm_socket(Socket *socket);

Return

The stream.

Remarks

Allows to use the streams functionality to read or write to a remote process. The socket must have been previously created with bsocket_connect (client) or bsocket_accept (server). The stream will close the socket at stm_close. bsocket_close should not be called. See Socket stream.

stm_close ()

Close the stream. All resources such as file descriptors or sockets will be released. Before to closing, the data will be written to the channel stm_flush.

void
stm_close(Stream **stm);

stm_get_write_endian ()

Get the current byte order when writing to the stream.

endian_t
stm_get_write_endian(const Stream *stm);

Return

The Byte order.

stm_get_read_endian ()

Get the current byte order when reading from the stream.

endian_t
stm_get_read_endian(const Stream *stm);

Return

The Byte order.

stm_set_write_endian ()

Set the order of bytes when writing to the stream, from now on.

void
stm_set_write_endian(Stream *stm,
                     const endian_t endian);

Remarks

Default is ekLITEND, except in sockets that will be ekBIGEND.

stm_set_read_endian ()

Set the order of bytes when reading from the stream, from now on.

void
stm_set_read_endian(Stream *stm,
                    const endian_t endian);

Remarks

Default is ekLITEND, except in sockets that will be ekBIGEND.

stm_get_write_utf ()

Gets the UTF encoding with which the texts are being written in the stream.

unicode_t
stm_get_write_utf(const Stream *stm);

Return

UTF encodings.

Remarks

See Text stream.

stm_get_read_utf ()

Get the UTF encoding with which the texts are being read in the stream.

unicode_t
stm_get_read_utf(const Stream *stm);

Return

UTF encodings.

Remarks

See Text stream.

stm_set_write_utf ()

Set the UTF encoding when writing texts in the stream, from now on.

void
stm_set_write_utf(Stream *stm,
                  const unicode_t format);

Remarks

See Text stream.

stm_set_read_utf ()

Set the UTF encoding when reading texts in the stream, from now on.

void
stm_set_read_utf(Stream *stm,
                 const unicode_t format);

Remarks

See Text stream.

stm_is_memory ()

Gets if it is a memory-resident stream.

bool_t
stm_is_memory(const Stream *stm);

Return

TRUE if it was created by stm_from_block or stm_memory.

stm_bytes_written ()

Gets the total bytes written in the stream since its creation.

uint64_t
stm_bytes_written(const Stream *stm);

Return

The total number of bytes written.

stm_bytes_readed ()

Get the total bytes read from the stream since its creation.

uint64_t
stm_bytes_readed(const Stream *stm);

Return

The total number of bytes readed.

stm_col ()

Get the column in text streams.

uint32_t
stm_col(const Stream *stm);

Return

Column number.

Remarks

When we read characters in text streams with stm_read_char or derivatives, the columns and rows are counted in a similar way as text editors do. This information can be useful when displaying warnings or error messages. In mixed streams (binary + text), the count stops when reading binary data and continues when reading the text is resumed. View Text stream.

stm_row ()

Get row in text streams.

uint32_t
stm_row(const Stream *stm);

Return

Row number.

Remarks

See stm_col.

stm_token_col ()

Gets the column of the last token read.

uint32_t
stm_token_col(const Stream *stm);

Return

Column number.

Remarks

It only takes effect after calling stm_read_token or derivatives. See stm_col and Tokens.

stm_token_row ()

Gets the row of the last token read.

uint32_t
stm_token_row(const Stream *stm);

Return

Row number.

Remarks

It only takes effect after calling stm_read_token or derivatives. See stm_col and Tokens.

stm_token_lexeme ()

Gets the lexeme of the last token read.

const char_t*
stm_token_lexeme(const Stream *stm);

Return

The lexeme. It is stored in a temporary buffer and will be lost when reading the next token. If you need it, make a copy with str_c.

Remarks

It only takes effect after calling stm_read_token or derivatives. See stm_col and Tokens.

stm_token_escapes ()

Escape sequences option when reading tokens.

void
stm_token_escapes(const Stream *stm,
                  const bool_t active_escapes);

Remarks

It will take effect on the next call to stm_read_token. See Tokens.

stm_token_spaces ()

Blanks option when reading tokens.

void
stm_token_spaces(const Stream *stm,
                 const bool_t active_spaces);

Remarks

It will take effect on the next call to stm_read_token. See Tokens.

stm_token_comments ()

Comments option when reading tokens.

void
stm_token_comments(const Stream *stm,
                   const bool_t active_comments);

Remarks

It will take effect on the next call to stm_read_token. See Tokens.

stm_state ()

Get the current state of the stream.

sstate_t
stm_state(const Stream *stm);

Return

The Stream state.

stm_file_err ()

Get additional information about the error, in disk streams.

ferror_t
stm_file_err(const Stream *stm);

Return

File error.

Remarks

It is only relevant in File stream with the state ekSTBROKEN.

stm_sock_err ()

Get additional information about the error, in network streams.

serror_t
stm_sock_err(const Stream *stm);

Return

Socket error.

Remarks

It is only relevant in Socket stream with the state ekSTBROKEN.

stm_corrupt ()

Set the stream status to ekSTCORRUPT.

void
stm_corrupt(Stream *stm);

Remarks

Sometimes, it is the application that detects that the data is corrupted since the data semantics wasn't expected.

stm_str ()

Create a string with the current content of the internal buffer. It is only valid for stream in memory. stm_memory.

String*
stm_str(const Stream *stm);

Return

The string with the buffer content.

stm_buffer ()

Gets a pointer to the current content of the internal buffer. Only valid for stream in memory. stm_memory.

const byte_t*
stm_buffer(const Stream *stm);

Return

Internal buffer pointer.

Remarks

This pointer is read only. Writing here will have unexpected consequences. Contains the information written to the stream, but not yet consumed. Any read operation on the stream will reduce the buffer size.

stm_buffer_size ()

Get the current size of the internal buffer. Only valid for stream in memory. stm_memory.

uint32_t
stm_buffer_size(const Stream *stm);

Return

The size of the internal buffer (in bytes).

stm_write ()

Write bytes in the stream.

void
stm_write(Stream *stm,
          const byte_t *data,
          const uint32_t size);

Remarks

The block is written as is, regardless of the Byte order neither the UTF encodings.

stm_write_char ()

Write a Unicode character in the stream.

void
stm_write_char(Stream *stm,
               const uint32_t codepoint);

Remarks

The encoding can be changed with stm_set_write_utf.

stm_printf ()

Write text in the stream, using the printf format .

uint32_t
stm_printf(Stream *stm,
           const char_t *format,
           ...);

1

stm_printf(stream, Code: %-10s Price %5.2f\n", code, price);

Return

The number of bytes written.

Remarks

The final null character ('\0') will not be written. The encoding can be changed with stm_set_write_utf.

stm_writef ()

Writes a UTF8 C string in the stream.

uint32_t
stm_writef(Stream *stm,
           const char_t *str);

Return

The number of bytes written.

Remarks

The final null character ('\0') will not be written. This function is faster than stm_printf when the string is constant and does not need formatting. For String objects use str_writef. The encoding can be changed with stm_set_write_utf.

stm_write_bool ()

Write a bool_t variable in the stream.

void
stm_write_bool(Stream *stm,
               const bool_t value);

Remarks

It is a binary write. Do not use in "pure" text streams.

stm_write_i8 ()

Write a int8_t variable in the stream.

void
stm_write_i8(Stream *stm,
             const int8_t value);

Remarks

It is a binary write. Do not use in "pure" text streams.

stm_write_i16 ()

Write a int16_t variable in the stream.

void
stm_write_i16(Stream *stm,
              const int16_t value);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_write_i32 ()

Write a int32_t variable in the stream.

void
stm_write_i32(Stream *stm,
              const int32_t value);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_write_i64 ()

Write a int64_t variable in the stream.

void
stm_write_i64(Stream *stm,
              const int64_t value);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_write_u8 ()

Write a uint8_t variable in the stream.

void
stm_write_u8(Stream *stm,
             const uint8_t value);

Remarks

It is a binary write. Do not use in "pure" text streams.

stm_write_u16 ()

Write a uint16_t variable in the stream.

void
stm_write_u16(Stream *stm,
              const uint16_t value);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_write_u32 ()

Write a uint32_t variable in the stream.

void
stm_write_u32(Stream *stm,
              const uint32_t value);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_write_u64 ()

Write a uint64_t variable in the stream.

void
stm_write_u64(Stream *stm,
              const uint64_t value);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_write_r32 ()

Write a real32_t variable in the stream.

void
stm_write_r32(Stream *stm,
              const real32_t value);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_write_r64 ()

Write a real64_t variable in the stream.

void
stm_write_r64(Stream *stm,
              const real64_t value);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_write_enum ()

Write a enum variable in the stream.

void
stm_write_enum(Stream *stm,
               const type value,
               type);

Remarks

It is a binary write. Do not use in "pure" text streams. Byte order.

stm_read ()

Read bytes from the stream.

uint32_t
stm_read(Stream *stm,
         byte_t *data,
         const uint32_t size);

Return

The number of bytes actually read.

stm_read_char ()

Read a text character from the stream.

uint32_t
stm_read_char(Stream *stm);

Return

The Unicode character code.

Remarks

The encoding of the input text can be changed with stm_set_read_utf.

stm_read_chars ()

Read several characters from the stream.

const char_t*
stm_read_chars(Stream *stm,
               const uint32_t n);

Return

Pointer to the UTF8 C string read. It will end with the null character '\0'.

Remarks

The returned pointer is temporary and will be overwritten in the next reading. If necessary, make a copy with str_c. The encoding of the input text can be changed with stm_set_read_utf.

stm_read_line ()

Read stream characters until an end of line is reached '\n'.

const char_t*
stm_read_line(Stream *stm);

Return

Pointer to the UTF8 C string, terminated with the null character '\0'. The characters '\n' or '\r\n' will not be included in the result. NULL will be returned when the end of the stream is reached.

Remarks

The returned pointer is temporary and will be overwritten in the next reading. If necessary, make a copy with str_c. The encoding of the input text can be changed with stm_set_read_utf.

stm_read_trim ()

Read the following sequence of characters removing the blank spaces.

const char_t*
stm_read_trim(Stream *stm);

Return

Pointer to the C UTF8 string read. It will end with the null character '\0'.

Remarks

Useful for reading strings from text streams. It will ignore all leading blanks and read characters until the first blank is found (' ', '\\t', '\\n', '\\v', '\\f', '\\r'). If you need more control over tokens use stm_read_token. The pointer returned is temporary and will be overwritten on the next read. If necessary, make a copy with str_c. The input text encoding can be adjusted with stm_set_read_utf. It will update the row and column counter. See stm_col.

stm_read_token ()

Get the following token in Text stream.

token_t
stm_read_token(Stream *stm);

Return

The type of token obtained.

Remarks

To get the text string associated with the token, use stm_token_lexeme. See Tokens.

stm_read_i8_tok ()

Read the following token with stm_read_token and, if it is an integer, convert it to int8_t.

int8_t
stm_read_i8_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

In case a token of type ekTINTEGER cannot be read (with or without ekTMINUS) or the numeric value is out of range, 0 will be returned and the stream will be marked as corrupt with stm_corrupt.

stm_read_i16_tok ()

Read the next token and convert it to int16_t.

int16_t
stm_read_i16_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

See stm_read_i8_tok.

stm_read_i32_tok ()

Read the next token and convert it to int32_t.

int32_t
stm_read_i32_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

See stm_read_i8_tok.

stm_read_i64_tok ()

Read the next token and convert it to int64_t.

int64_t
stm_read_i64_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

See stm_read_i8_tok.

stm_read_u8_tok ()

Read the following token with stm_read_token and, if it is an integer, convert it to uint8_t.

uint8_t
stm_read_u8_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

In case a token of type ekTINTEGER cannot be read or the numeric value is out of range, 0 will be returned and the stream will be marked as corrupt with stm_corrupt.

stm_read_u16_tok ()

Read the next token and convert it to uint16_t.

uint16_t
stm_read_u16_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

See stm_read_u8_tok.

stm_read_u32_tok ()

Read the next token and convert it to uint32_t.

uint32_t
stm_read_u32_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

See stm_read_u8_tok.

stm_read_u64_tok ()

Read the next token and convert it to uint64_t.

uint64_t
stm_read_u64_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

See stm_read_u8_tok.

stm_read_r32_tok ()

Read the following token with stm_read_token and, if it is a real number, convert it to real32_t.

real32_t
stm_read_r32_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

In case a token of type ekTINTEGER or ekTREAL cannot be read (with or without ekTMINUS), 0 will be returned and the stream will be marked as corrupt with stm_corrupt.

stm_read_r64_tok ()

Read the next token and convert it to real64_t.

real64_t
stm_read_r64_tok(Stream *stm);

Return

The numeric value of the token.

Remarks

See stm_read_r32_tok.

stm_read_bool ()

Read a bool_t value from the stream.

bool_t
stm_read_bool(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams.

stm_read_i8 ()

Read a int8_t value from the stream.

int8_t
stm_read_i8(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams.

stm_read_i16 ()

Read a int16_t value from the stream.

int16_t
stm_read_i16(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_read_i32 ()

Read a int32_t value from the stream.

int32_t
stm_read_i32(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_read_i64 ()

Read a int64_t value from the stream.

int64_t
stm_read_i64(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_read_u8 ()

Read a uint8_t value from the stream.

uint8_t
stm_read_u8(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams.

stm_read_u16 ()

Read a uint16_t value from the stream.

uint16_t
stm_read_u16(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_read_u32 ()

Read a uint32_t value from the stream.

uint32_t
stm_read_u32(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_read_u64 ()

Read a uint64_t value from the stream.

uint64_t
stm_read_u64(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_read_r32 ()

Read a real32_t value from the stream.

real32_t
stm_read_r32(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_read_r64 ()

Read a real64_t value from the stream.

real64_t
stm_read_r64(Stream *stm);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_read_enum ()

Read a enum value from the stream.

type
stm_read_enum(Stream *stm,
              type);

Return

Value read.

Remarks

It is a binary reading. Do not use in "pure" text streams. Byte order.

stm_skip ()

Skip and ignore the next bytes of the stream.

void
stm_skip(Stream *stm,
         const uint32_t size);

stm_skip_bom ()

Skip the possible Byte Order Mark sequence "" found at the beginning of some text streams.

void
stm_skip_bom(Stream *stm);

Remarks

This function will have no effect if there is no such sequence at the beginning of the stream. The BOM is common in streams coming from some web servers.

stm_skip_token ()

Skip the next token in the stream. If the token does not correspond to the one indicated, the stream will be marked as corrupt.

void
stm_skip_token(Stream *stm,
               const token_t token);

1
2
3
4
5
6

void stm_skip_token(Stream *stm, const token_t token)
{
    token_t tok = stm_read_token(stm);
    if (tok != token)
        stm_corrupt(stm);
}

stm_flush ()

Write in the channel the existing information in the cache.

void
stm_flush(Stream *stm);

Remarks

To improve performance, write operations on disk streams or standard I/O are stored in an internal cache. This function forces writing on the channel and cleans the buffer. It will be useful with full-duplex protocols where the receiver awaits reply to continue.

stm_pipe ()

Connect two streams, reading data from one and writing it to another.

void
stm_pipe(Stream *from,
         Stream *to,
         const uint32_t n);

Remarks

The transfer will be made on raw data, regardless of Byte order or UTF encodings. If you are clear that this does not affect, it is much faster than using atomic read/write operations.

stm_lines ()

Iterate over all lines in a Text stream. You should use stm_next to close the loop.

void
stm_lines(const char_t *line,
          Stream *stm);

1
2
3
4
5
6

uint32_t i = 1;
Stream *stm = stm_from_file("/home/john/friends.txt", NULL);
stm_lines(line, stm)
    bstd_printf("Friend %d, name %s\n", i++, line);
stm_next(line, stm)
stm_close(&stm);

stm_next ()

Close a loop open by stm_lines.

void
stm_next(const char_t *line,
         Stream *stm);