# System-Level I/O IO is the process of coping data between the main memory and external devices. In a Linux, **file** is a sequence of $m$ bytes. All I/O devices are represented as files. Even the kernel is represented as a file. ## Unix IO * `open` and `close` * `read` and `write` * `lseek` changing **current file position** ### File Types * Regular files * Directory * Socket * ... #### Regular Files A regular file contains arbitary data. For example **text file** is a sequence of text lines. EOL is different in different OS: (`\n` in Unix, `\r\n` in Windows & Internet). #### Directories Directory contains an array of links. Least two links are `.`(itself) and `..`(parent dir). * `ls` * `mkdir` * `rmdir` All files are orgnaized as a hierarchy anchored by root dir named `/`. Kernel maintains curr working dir (cwd) for each process that modified using the `cd` command. Path names * Absolute `/home/yenru0/workspace` * Relative `../workspace` ### Open & Close & Read & Write ```c int fd; if ((fd = open("file.txt", O_RDONLY)) < 0) { perror("open"); exit(1); } ``` * `open` returns a non-negative integer called **file descriptor** (fd). * `fd == -1` indicates an error. * `0`: stdin, `1`: stdout, `2`: stderr ```c int fd; int ret; if ((ret = close(fd)) < 0) { perror("close"); exit(1); } ``` Closing an already closed can lead to a disastrous situation in threaded programs. So always check the return code. ```c char buf[512]; nbytes = read(fd, buf, sizeof(buf)); ``` ```c ssize_t read(int fd, void *usrbuf, size_t n); ``` read returns the number of bytes read from the `fd` into `buf`. `ssize_t` is signed version of `size_t`. If `read` returns negative value, an error occurred. ```c ssize_t write(int fd, const void *usrbuf, size_t n); ``` If `write` returns negative value, an error occurred. ### Short Counts It means that `read` or `write` transfers fewer bytes than requested. It can occur in these situations: * `EOF` on reads * Reading text lines from an terminal * Reading from a network socket Never occurs: * Reading from disk files (except for `EOF`) * Writing to disk files ## RIO pakcage RIO is a set of wrappers efficient and robust I/O functions subject to **short couunts**. * unbuffered RIO functions `rio_readn`, `rio_writen` * buffered RIO functions `rio_readnb`, `rio_readlineb` * buffered RIO functions are thread-safe and can be interleaved arbitrarily on the same descriptor. ### Buffered RIO To read efficiently from a file, RIO uses partially cached in an interal memory buffer. (`rio_t` structure) For reading from file, Buffer has buffered portion of already read and unread data. It is refilled automatically by `rio_readnb` and `rio_readlineb` as needed. This is **partially cached**. ```c typedef struct { int rio_fd; // Descriptor for this internal buf int rio_cnt; // Unread bytes in internal buf char *rio_bufptr; // Next unread byte in internal buf char rio_buf[RIO_BUFSIZE]; // Internal buffer } rio_t; ``` example: ```c int main(int argc, char **argv) { int n; rio_t rio; char buf[MAXLINE]; rio_readinitb(&rio, STDIN_FILENO); while ((n = rio_readlineb(&rio, buf, MAXLINE)) != 0) { rio_writen(STDOUT_FILENO, buf, n); } exit(0); } ``` ## Metadata Metadata is data about data. (file access, file size, file type) * Per-process metadata * when a process opens a file, the kernel creates an entry in a per-process table called the **file descriptor table** * Per-file metadata * can be accessed using `stat` system call ```c struct stat { dev_t st_dev; // ID of device containing file ino_t st_ino; // inode number mode_t st_mode; // protection nlink_t st_nlink; // number of hard links uid_t st_uid; // user ID of owner gid_t st_gid; // group ID of owner dev_t st_rdev; // device ID (if special file) off_t st_size; // total size, in bytes blksize_t st_blksize; // blocksize for filesystem I/O blkcnt_t st_blocks; // number of 512B blocks allocated time_t st_atime; // time of last access time_t st_mtime; // time of last modification time_t st_ctime; // time of last status change }; ``` ### How to Kernel represents Open Files * Descriptor table(per-process) * Open file table(shared by all processes) * v-node table(shared by all processes) When a process opens a file, the kernel creates an entry in the per-process file descriptor table. Each entry contains a pointer to an entry in the open file table. Each entry in the open file table contains a pointer to an entry in the v-node table. When a `fork` calls: the child process inherits copies of the parent's file descriptors. And the entry points to open file table's entry increasing `refcnt`. ### IO redirection for example: `ls > foo.txt` Answer: `dup2(oldfd, newfd)` it means copies descriptor table entry `oldfd` to `newfd` so `dup2(4, 1)` makes `stdout` point to the same open file as descriptor 4. ## stdio The C standard library (`libc.so`) provides a collection of higher-level standard I/O functions. * `fopen`, `fclose`, `fread`, `fwrite`, `fgets`, `fputs`, `fscanf`, `fprintf` `stdio` models open files as **streams**, which are abstraction for a file descriptor and a buffer in memory. ```c extern FILE * stdin; extern FILE * stdout; extern FILE * stderr; ``` ### Buffered I/O Application often read and write one char at a time. However, UNIX System calls `read` and `write` calls expensive. So we need buffered read & write; use unix `read` & `write` to **get a block of data into a buffer**. And then user application reads/writes **one char at a time from/to the buffer**; it is efficient because it is simple memory access. `stdio` uses buffer. `printf` is not write immediately to the `stdout` file; it is stored in a buffer. And then when `fflush(stdout)`, `exit`, or return from `main`, the buffer is flushed to the file using `write` syscall. ## Remark * UNIX IO * RIO package * stdio When to use * stdio: disk or terminal files * unix io: signal handlers, or when you need absolute high performance * RIO: networking ### Binary DO NOT USE: * text oriented I/O: `fgets`, `scanf`, `rio_readlineb` * string functions: `strlen`, `strcpy`, `strcat`, `strcmp`