complement 9.md 10.md 11.md
This commit is contained in:
218
notes/10.md
Normal file
218
notes/10.md
Normal file
@@ -0,0 +1,218 @@
|
||||
# Signals and Nonlocal Jumps
|
||||
|
||||
## Shell
|
||||
|
||||
Linux Process Hierachy: all is rooted from `init` or `systemd` (pid = 1).
|
||||
|
||||
A shell is an application that runs programs on behalf of the user.
|
||||
|
||||
* `sh` Original Unix shell
|
||||
* `csh` BSD Unix C Shell
|
||||
* `bash` GNU Bourne-Again Shell (default Linux shell)
|
||||
|
||||
|
||||
## Signals
|
||||
|
||||
All signal is a small message that notifies a process that an event has occured in the system.
|
||||
|
||||
Signal is sent by kernel.
|
||||
|
||||
Signal is identified by small integer ID (1-30).
|
||||
|
||||
Only information in a signal is **its ID** and **the fact that it arrived**.
|
||||
|
||||
| ID | name | Default Action | Corresponding Event |
|
||||
| --- | ------- | -------------- | ----------------------------------- |
|
||||
| 2 | SIGINT | terminate | user types `Ctrl+C` |
|
||||
| 9 | SIGKILL | terminate | kill program |
|
||||
| 11 | SIGSEGV | terminate | segmentation violation |
|
||||
| 14 | SIGALRM | terminate | timer expired |
|
||||
| 17 | SIGCHLD | ignore | child process stopped or terminated |
|
||||
|
||||
Sending a signal is essentially **updating some state** in the context of the destination process.
|
||||
Kernel sends a signal for one of the following reasons:
|
||||
|
||||
A destination process receives a signal when it is forced by the kernel and reacts some way to the signal.
|
||||
* Ignore
|
||||
* Terminate (with optional core-dump)
|
||||
* Catch
|
||||
* executing a user-level handler function called **signal handler**.
|
||||
|
||||
### Pending & Blocking
|
||||
|
||||
A signal is **pending** if sent but not yet received. There can be at most one pending signal of any particular type because signals are **not queued**. Signal is managed by bitmask. So subsequent signals of same type is discarded.
|
||||
* sets `k` bit when delivered
|
||||
* clears `k` bit when received
|
||||
|
||||
Blocking signals: a process can block the receipt of certain signals. It is not received until unblocked. It is also managed by bitmask.
|
||||
* can be set and cleared by using `sigprocmask`
|
||||
|
||||
### Receiving
|
||||
|
||||
Suppose kernel is returning from an exception handler and is ready to pass control to process `p`
|
||||
Kernel computes `pnb = pending(p) & ~blocked(p)`.
|
||||
* if `pnb == 0`, no unblocked pending signals for `p`, so just return to `p` normally.
|
||||
* otherwise, choose least nonzero bit `k` in `pnb` and force process `p` to receive signal `k`
|
||||
* The receipt of the signal triggers action by `p`
|
||||
* Repeat for all nonzero `k`
|
||||
|
||||
### Default Action & Signal Handler
|
||||
|
||||
Each signal type has a predefined default action:
|
||||
* terminates
|
||||
* stops until restarted by a SIGCONT signal
|
||||
* ignores
|
||||
|
||||
```c
|
||||
handler_t * signal(int signum, handler_t *handler);
|
||||
```
|
||||
|
||||
* `SIG_IGN` ignore
|
||||
* `SIG_DFL` revert to default action
|
||||
* handler function pointer
|
||||
|
||||
### Blocking and Unblocking
|
||||
|
||||
* Implicit blocking
|
||||
Kernel blocks any pending signals of type currently being handled.
|
||||
* Explicit blocking `sigprocmask()`
|
||||
|
||||
Supporting functions:
|
||||
* `sigemptyset()` create empty set
|
||||
* `sigfillset()` add every signal number to set
|
||||
* `sigaddset()` add signal number to set
|
||||
* `sigdelset()` delete signal number from set
|
||||
|
||||
```c
|
||||
sigset_t mask, prev_mask;
|
||||
sigemptyset(&mask);
|
||||
sigaddset(&mask, SIGINT);
|
||||
|
||||
sigprocmaksk(SIG_BLOCK, &mask, &prev_mask);
|
||||
// critical section
|
||||
sigprocmask(SIG_SETMASK, &prev_mask, NULL);
|
||||
```
|
||||
|
||||
### Safe Signal Handling
|
||||
|
||||
Handlers are concurrent with main program and share global data structures. This can lead to troubles.
|
||||
|
||||
Some guidlines help to avoid troubles:
|
||||
|
||||
0. Keep you handlers as simple as possible
|
||||
1. Call only **async-signal-safe** functions in your handlers
|
||||
2. Save and restore `errno` on entry and exit
|
||||
3. Protect accesses to shared data structure by temporarily blocking all signals
|
||||
4. Declare global variables as `volatile`
|
||||
5. Declare global flags as `volatile sig_atomic_t`
|
||||
|
||||
#### Async-Signal-Safety
|
||||
|
||||
A function satisfying either two conditions(below) is **Async-Signal-Safety function**:
|
||||
* **reentrant**: not modifying global data (or static data), only use local data(stack frame).
|
||||
* **non-interruptible**: blocking singnals during modifying global data.
|
||||
|
||||
POSIX guarantees that the 117 functions to be async-signal-safe, including:
|
||||
* `_exit`, `write`, `wait`, `waitpid`, `sleep`, `kill`
|
||||
|
||||
**IMPORTANT**: `printf`, `malloc`, `sprintf`, `eixt` are **NOT** async-signal-safe.
|
||||
|
||||
### Correct Signal Handling Example for reaping multiple childs
|
||||
|
||||
```c
|
||||
void child_handler(int sig) {
|
||||
int olderrno = errno; pid_t pid;
|
||||
while ((pid = wait(NULL)) > 0) {
|
||||
ccount --;
|
||||
sio_puts("Handler reaped child ");
|
||||
sio_putl((long) pid);
|
||||
sio_puts(" \n");
|
||||
}
|
||||
if (errno != ECHILD) {
|
||||
sio_error("wait error");
|
||||
}
|
||||
errno = olderrno;
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
### Portable Signal Handling
|
||||
|
||||
Different UNIX systems have different signal handling sematnics.
|
||||
|
||||
So `sigaction` is introduced to provide a portable interface for signal handling.
|
||||
|
||||
```c
|
||||
handler_t * Signal(int signum, handler_t *handler) {
|
||||
struct sigaction action, old_action;
|
||||
action.sa_handler = handler;
|
||||
sigemptyset(&action.sa_mask); // block sigs of type being handled
|
||||
action.sa_flags = SA_RESTART; // restart syscalls if possible
|
||||
if (sigaction(signum, &action, &old_action) < 0)
|
||||
unix_error("Signal error");
|
||||
return (old_action.sa_handler);
|
||||
}
|
||||
```
|
||||
|
||||
### Concurrent Flows to lead Races
|
||||
|
||||
```c
|
||||
void handler(int sig) {
|
||||
int olderrno = errno;
|
||||
pid_t pid;
|
||||
while ((pid = waitpid(-1, NULL, 0)) > 0) {
|
||||
deletejob(pid);
|
||||
}
|
||||
if (errno != ECHILD) sio_error("waitpid error");
|
||||
errno = olderrno;
|
||||
}
|
||||
|
||||
int main() {
|
||||
while (1) {
|
||||
if ((pid = Fork()) == 0) {
|
||||
execve("/bin/date", argv, NULL);
|
||||
}
|
||||
}
|
||||
addjob(pid);
|
||||
}
|
||||
```
|
||||
|
||||
In above example, the ecf flow can be executed before the deletejob; thus it cannot be deleted.
|
||||
Therefore, we need to synchronize the two concurrent flows.
|
||||
|
||||
```c
|
||||
void handler(int sig) {
|
||||
int olderrno = errno;
|
||||
|
||||
sigset_t mask_all, prev_all;
|
||||
pid_t pid;
|
||||
|
||||
sigfillset(&mask_all);
|
||||
while ((pid = waitpid(-1, NULL, 0)) > 0) {
|
||||
sigprocmask(SIG_BLOCK, &mask_all, &prev_all);
|
||||
deletejob(pid);
|
||||
sigprocmask(SIG_SETMASK, &prev_all, NULL);
|
||||
}
|
||||
if (errno != ECHILD) sio_error("waitpid error");
|
||||
errno = olderrno;
|
||||
}
|
||||
|
||||
int main() {
|
||||
pid_t pid;
|
||||
sigset_t mask_all, prev_one;
|
||||
|
||||
sigfillset(&mask_all);
|
||||
signal(SIGCHLD, handler);
|
||||
|
||||
while (1) {
|
||||
sigprocmask(SIG_BLOCK, &mask_all, &prev_one); // Block SIGCHLD
|
||||
if ((pid = Fork()) == 0) {
|
||||
sigprocmask(SIG_SETMASK, &prev_one, NULL); // Unblock SIGCHLD
|
||||
execve("/bin/date", argv, NULL);
|
||||
}
|
||||
sigprocmask(SIG_BLOCK, &mask_one, NULL);
|
||||
addjob(pid);
|
||||
sigprocmask(SIG_SETMASK, &prev_one, NULL);
|
||||
}
|
||||
}
|
||||
```
|
||||
223
notes/11.md
Normal file
223
notes/11.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# System-Level I/O
|
||||
|
||||
IO is the process of coping data between the main memory and external devices.
|
||||
|
||||
In a Linux, **file** is a sequence of $m$ bytes.
|
||||
|
||||
All I/O devices are represented as files. Even the kernel is represented as a file.
|
||||
|
||||
## Unix IO
|
||||
|
||||
* `open` and `close`
|
||||
* `read` and `write`
|
||||
* `lseek` changing **current file position**
|
||||
|
||||
### File Types
|
||||
|
||||
* Regular files
|
||||
* Directory
|
||||
* Socket
|
||||
* ...
|
||||
|
||||
#### Regular Files
|
||||
|
||||
A regular file contains arbitary data.
|
||||
|
||||
For example **text file** is a sequence of text lines. EOL is different in different OS: (`\n` in Unix, `\r\n` in Windows & Internet).
|
||||
|
||||
#### Directories
|
||||
|
||||
Directory contains an array of links. Least two links are `.`(itself) and `..`(parent dir).
|
||||
|
||||
* `ls`
|
||||
* `mkdir`
|
||||
* `rmdir`
|
||||
|
||||
All files are orgnaized as a hierarchy anchored by root dir named `/`.
|
||||
|
||||
Kernel maintains curr working dir (cwd) for each process that modified using the `cd` command.
|
||||
|
||||
Path names
|
||||
* Absolute `/home/yenru0/workspace`
|
||||
* Relative `../workspace`
|
||||
|
||||
### Open & Close & Read & Write
|
||||
|
||||
```c
|
||||
int fd;
|
||||
|
||||
if ((fd = open("file.txt", O_RDONLY)) < 0) {
|
||||
perror("open");
|
||||
exit(1);
|
||||
}
|
||||
```
|
||||
|
||||
* `open` returns a non-negative integer called **file descriptor** (fd).
|
||||
* `fd == -1` indicates an error.
|
||||
* `0`: stdin, `1`: stdout, `2`: stderr
|
||||
|
||||
```c
|
||||
int fd; int ret;
|
||||
if ((ret = close(fd)) < 0) {
|
||||
perror("close");
|
||||
exit(1);
|
||||
}
|
||||
```
|
||||
|
||||
Closing an already closed can lead to a disastrous situation in threaded programs. So always check the return code.
|
||||
|
||||
```c
|
||||
char buf[512];
|
||||
|
||||
nbytes = read(fd, buf, sizeof(buf));
|
||||
|
||||
```
|
||||
|
||||
```c
|
||||
ssize_t read(int fd, void *usrbuf, size_t n);
|
||||
```
|
||||
|
||||
read returns the number of bytes read from the `fd` into `buf`.
|
||||
`ssize_t` is signed version of `size_t`.
|
||||
|
||||
If `read` returns negative value, an error occurred.
|
||||
|
||||
```c
|
||||
ssize_t write(int fd, const void *usrbuf, size_t n);
|
||||
```
|
||||
|
||||
If `write` returns negative value, an error occurred.
|
||||
|
||||
### Short Counts
|
||||
|
||||
It means that `read` or `write` transfers fewer bytes than requested. It can occur in these situations:
|
||||
|
||||
* `EOF` on reads
|
||||
* Reading text lines from an terminal
|
||||
* Reading from a network socket
|
||||
|
||||
Never occurs:
|
||||
* Reading from disk files (except for `EOF`)
|
||||
* Writing to disk files
|
||||
|
||||
## RIO pakcage
|
||||
|
||||
RIO is a set of wrappers efficient and robust I/O functions subject to **short couunts**.
|
||||
|
||||
* unbuffered RIO functions `rio_readn`, `rio_writen`
|
||||
* buffered RIO functions `rio_readnb`, `rio_readlineb`
|
||||
* buffered RIO functions are thread-safe and can be interleaved arbitrarily on the same descriptor.
|
||||
|
||||
|
||||
### Buffered RIO
|
||||
|
||||
To read efficiently from a file, RIO uses partially cached in an interal memory buffer. (`rio_t` structure)
|
||||
|
||||
For reading from file, Buffer has buffered portion of already read and unread data. It is refilled automatically by `rio_readnb` and `rio_readlineb` as needed. This is **partially cached**.
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
int rio_fd; // Descriptor for this internal buf
|
||||
int rio_cnt; // Unread bytes in internal buf
|
||||
char *rio_bufptr; // Next unread byte in internal buf
|
||||
char rio_buf[RIO_BUFSIZE]; // Internal buffer
|
||||
} rio_t;
|
||||
```
|
||||
|
||||
example:
|
||||
|
||||
```c
|
||||
int main(int argc, char **argv) {
|
||||
int n; rio_t rio; char buf[MAXLINE];
|
||||
rio_readinitb(&rio, STDIN_FILENO);
|
||||
while ((n = rio_readlineb(&rio, buf, MAXLINE)) != 0) {
|
||||
rio_writen(STDOUT_FILENO, buf, n);
|
||||
}
|
||||
exit(0);
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
## Metadata
|
||||
|
||||
Metadata is data about data. (file access, file size, file type)
|
||||
|
||||
* Per-process metadata
|
||||
* when a process opens a file, the kernel creates an entry in a per-process table called the **file descriptor table**
|
||||
* Per-file metadata
|
||||
* can be accessed using `stat` system call
|
||||
|
||||
|
||||
```c
|
||||
struct stat {
|
||||
dev_t st_dev; // ID of device containing file
|
||||
ino_t st_ino; // inode number
|
||||
mode_t st_mode; // protection
|
||||
nlink_t st_nlink; // number of hard links
|
||||
uid_t st_uid; // user ID of owner
|
||||
gid_t st_gid; // group ID of owner
|
||||
dev_t st_rdev; // device ID (if special file)
|
||||
off_t st_size; // total size, in bytes
|
||||
blksize_t st_blksize; // blocksize for filesystem I/O
|
||||
blkcnt_t st_blocks; // number of 512B blocks allocated
|
||||
time_t st_atime; // time of last access
|
||||
time_t st_mtime; // time of last modification
|
||||
time_t st_ctime; // time of last status change
|
||||
};
|
||||
```
|
||||
|
||||
### How to Kernel represents Open Files
|
||||
|
||||
* Descriptor table(per-process)
|
||||
* Open file table(shared by all processes)
|
||||
* v-node table(shared by all processes)
|
||||
|
||||
When a process opens a file, the kernel creates an entry in the per-process file descriptor table. Each entry contains a pointer to an entry in the open file table. Each entry in the open file table contains a pointer to an entry in the v-node table.
|
||||
|
||||
When a `fork` calls: the child process inherits copies of the parent's file descriptors. And the entry points to open file table's entry increasing `refcnt`.
|
||||
|
||||
### IO redirection
|
||||
|
||||
for example: `ls > foo.txt`
|
||||
|
||||
Answer: `dup2(oldfd, newfd)` it means copies descriptor table entry `oldfd` to `newfd`
|
||||
so `dup2(4, 1)` makes `stdout` point to the same open file as descriptor 4.
|
||||
|
||||
## stdio
|
||||
|
||||
The C standard library (`libc.so`) provides a collection of higher-level standard I/O functions.
|
||||
|
||||
* `fopen`, `fclose`, `fread`, `fwrite`, `fgets`, `fputs`, `fscanf`, `fprintf`
|
||||
|
||||
`stdio` models open files as **streams**, which are abstraction for a file descriptor and a buffer in memory.
|
||||
|
||||
```c
|
||||
extern FILE * stdin;
|
||||
extern FILE * stdout;
|
||||
extern FILE * stderr;
|
||||
```
|
||||
|
||||
### Buffered I/O
|
||||
|
||||
Application often read and write one char at a time. However, UNIX System calls `read` and `write` calls expensive. So we need buffered read & write; use unix `read` & `write` to **get a block of data into a buffer**. And then user application reads/writes **one char at a time from/to the buffer**; it is efficient because it is simple memory access.
|
||||
|
||||
`stdio` uses buffer. `printf` is not write immediately to the `stdout` file; it is stored in a buffer. And then when `fflush(stdout)`, `exit`, or return from `main`, the buffer is flushed to the file using `write` syscall.
|
||||
|
||||
## Remark
|
||||
|
||||
* UNIX IO
|
||||
* RIO package
|
||||
* stdio
|
||||
|
||||
|
||||
|
||||
When to use
|
||||
* stdio: disk or terminal files
|
||||
* unix io: signal handlers, or when you need absolute high performance
|
||||
* RIO: networking
|
||||
|
||||
### Binary
|
||||
|
||||
DO NOT USE:
|
||||
* text oriented I/O: `fgets`, `scanf`, `rio_readlineb`
|
||||
* string functions: `strlen`, `strcpy`, `strcat`, `strcmp`
|
||||
59
notes/9.md
59
notes/9.md
@@ -84,7 +84,7 @@ Single processor executes multiple processes concurrently. process execution int
|
||||
|
||||
Multicore processor share main memory each can execute a separate process. scheduling of processors onto cores done by kernel.
|
||||
|
||||
### Concurrent Processes
|
||||
#### Concurrent Processes
|
||||
|
||||
Concurrency is **not at the exact same time**.
|
||||
|
||||
@@ -94,7 +94,7 @@ Control flows for concurrent processes are pysically disjoint in time. But user
|
||||
|
||||
* Execution time of instruction may vary because of the Nondeterminism of the System: OS scheduling, Interrupts, Cache miss or Page fault, I/O device delays.
|
||||
|
||||
### Context Switching
|
||||
#### Context Switching
|
||||
|
||||
Prcess are managed by a shared chunk of memory-resident OS code called the **kernel**.
|
||||
|
||||
@@ -180,7 +180,7 @@ int main() {
|
||||
}
|
||||
```
|
||||
|
||||
```sh {cmd}
|
||||
```sh {cmd hide}
|
||||
while ! [ -r 9_1.out ]; do sleep .1; done; ./9_1.out
|
||||
```
|
||||
|
||||
@@ -197,7 +197,58 @@ Any topological sort of the graph corresponds to a feasible total ordering.
|
||||
|
||||
### Reaping Child Processes
|
||||
|
||||
When a child process terminates, it becomes a **zombie** until its parent calls `wait` to read its exit status.
|
||||
When process terminates, it still consumes system resources. It is called a "zombie".
|
||||
|
||||
**"Reaping"** is performed by the parent by using `wait` or `waitpid`. And then parent is given exit status information. Finally kernel then deletes zombie child process.
|
||||
|
||||
If any parent terminates without reaping a child, then the orphaned child will be reaped by the `init` process (pid 1).
|
||||
However, long-running processes should reap their children to avoid accumulating zombies.
|
||||
|
||||
```c {cmd=gcc, args=[-O2 -x c $input_file -o 9_2.out]}
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <unistd.h>
|
||||
|
||||
int main() {
|
||||
if(fork() == 0) {
|
||||
printf("Terminating child process\n");
|
||||
exit(0);
|
||||
} else {
|
||||
printf("Running Parent\n");
|
||||
sleep(1);
|
||||
printf("Parent exiting\n");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```sh {cmd hide}
|
||||
while ! [ -r 9_2.out ]; do sleep .1; done; ./9_2.out & ps -ef | grep "9_2.out" | grep -v grep;
|
||||
```
|
||||
|
||||
|
||||
```c
|
||||
pid_t wait(int * child_status);
|
||||
```
|
||||
|
||||
`wait` suspends current process until **one of its children** terminates.
|
||||
|
||||
* **return value**: pid of terminated child
|
||||
* **child_status**: if not `NULL`, the integer points will be set to the termination status(reason and exit status) of the child.
|
||||
* It can be checked by using macros defined in `wait.h`
|
||||
|
||||
## execve
|
||||
|
||||
```c
|
||||
int execve(const char *filename, char *const argv[], char *envp[]);
|
||||
```
|
||||
|
||||
Load and runs in the current process.
|
||||
|
||||
* `filename`: path of the executable file
|
||||
* `argv`: argument list
|
||||
* `envp`: environment variables "name=value"
|
||||
* `getenv`, `putenv`, `printenv`
|
||||
|
||||
It does **overwrite code, data, stack**. retain only **pid, file descriptors, signal context**.
|
||||
|
||||
Called **once and never returns on success**.
|
||||
|
||||
Reference in New Issue
Block a user