# operating system **Repository Path**: chujin_w/nix_os ## Basic Information - **Project Name**: operating system - **Description**: learn unix v6 etc - **Primary Language**: C - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2024-11-27 - **Last Updated**: 2025-01-11 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # operating system ## 介绍 学习unix v6系列操作系统 ## MIT6.828 ### Introduce 6.828 teaches the fundamentals of engineering operating systems. You will study, in detail, virtual memory, kernel and user mode, system calls, threads, context switches, interrupts, interprocess communication, coordination of concurrent activities, and the interface between software and hardware. Most importantly, you will study the interactions between these concepts, and how to manage the complexity introduced by the interactions. To master the concepts, 6.828 is organized in three parts: lectures, readings, and a major lab. The lectures and readings familiarize you with the main concepts. The lab forces you to understand the concepts at a deep level, since you will build an operating system from the ground up. After the lab you will appreciate the meaning of design goals such as "reducing complexity" and "conceptual integrity". The lectures are organized in two main blocks. The first block introduces one operating system, xv6 (x86 version 6), which is a re-implementation of Unix Version 6, which was developed in the 1970s. In each lecture we will take one part of xv6 and study its source code; homework assignments will help you prepare for these lectures. At the end of the first block (about half-way through the term), you will understand the source code for one well-designed operating system for an Intel-based PC, which will help you in building your own operating system. The second block of lectures covers important operating systems concepts invented after Unix v6. We will study the more modern concepts by reading research papers and discussing them in lecture. You will also implement some of these newer concepts in your operating system. You may wonder why we are studying an operating system that resembles Unix v6 instead of the latest and greatest version of Linux, Windows, or BSD Unix. xv6 is big enough to illustrate the basic design and implementation ideas in operating systems. On the other hand, xv6 is far smaller than any modern production O/S, and correspondingly easier to understand. xv6 has a structure similar to many modern operating systems; once you've explored xv6 you will find that much is familiar inside kernels such as Linux. The lab is the place where the rubber meets the road. In the lab you will internalize the details of the concepts and their interactions. For example, although you have seen virtual memory in 6.004, 6.033, and again in 6.828 lectures, you will soon discover, during the labs, that you didn't really understand virtual memory and how it interacts with other concepts. The lab is split into 6 major parts that build on each other, culminating in a primitive operating system on which you can run simple commands through your own shell. We reserve the last lecture for you to demo your operating system to the rest of the class. The operating system you will build, called JOS, will have Unix-like functions (e.g., fork, exec), but is implemented in an exokernel style (i.e., the Unix functions are implemented mostly as user-level library instead of built-in to the kernel). The major parts of the JOS operating system are: 1. Booting 2. Memory management 3. User environments 4. Preemptive multitasking 5. File system, spawn, and shell 6. Network driver 7. Open-ended project We will provide skeleton code for pieces of JOS, but you will have to do all the hard work. You'll have design freedom for the details of the first few assignments, and freedom over the whole design in the last assignments. You'll find that xv6 helps you understand many of the goals you're trying to achieve in JOS, but that JOS occupies a very different point in the design and implementation space from xv6. The first 6 assignments are done individually. The last assignment is a team assignment. You will develop your JOS operating system for a standard x86-based personal computer, the same one used for xv6. To simplify development we will use a complete machine simulator (QEMU) in the class for development and debugging. This simulator is real enough, however, that you will be able to boot your own operating system on physical hardware if you wish. At the end of the lab you will be able to find your way around the source code of most operating systems, and more generally, be comfortable with system software. You will understand many operating systems concepts in detail and will be able to use them in other environments. You will also understand the x86 processor and the C programming language well. Acknowledgements 6.828 would not exist today had it not been for a wonderful set of past TAs (Josh Cates, Austin Clements, Russ Cox, Bryan Ford, Max Krohn, and Emil Sit). They made this class a reality. Collectively we dedicate 6.828 to the memory of Josh Cates; We hope that many students will be inspired by Josh's enthusiasm for operating systems, and have named the operating system JOS. We are also grateful to the students and teaching staff at Harvard, MIT (including SIPB), UCLA, and NYU for their many contributions. ### Lecture #### Lecture 1: O/S overview ##### Overview * 6.828 goals * Understand operating system design and implementation * Hands-on experience by building small O/S * What is the purpose of an O/S? * Support applications * Abstract the hardware for convenience and portability * Multiplex the hardware among multiple applications * Isolate applications in order to contain bugs * Allow sharing among applications * Provide high performance * What is the O/S design approach? * the small view: a h/w management library * the big view: physical machine -> abstract one w/ better properties * Organization: layered picture h/w: CPU, mem, disk, &c kernel services user applications: vi, gcc, &c * we care a lot about the interfaces and internal kernel structure * What services does an O/S kernel typically provide? * processes * memory allocation * file contents * directories and file names * security * many others: users, IPC, network, time, terminals * What does an O/S abstraction look like? * Applications see them only via system calls * Examples, from UNIX (e.g. Linux, OSX, FreeBSD): ```c fd = open("out", 1); write(fd, "hello\n", 6); pid = fork(); ``` * Why is O/S design/implementation hard/interesting? * the environment is unforgiving: quirky h/w, weak debugger * it must be efficient (thus low-level?) ...but abstract/portable (thus high-level?) * powerful (thus many features?) ...but simple (thus a few composable building blocks?) * features interact: `fd = open(); ...; fork()` * behaviors interact: CPU priority vs memory allocator * open problems: security; performance * You'll be glad you learned about operating systems if you... * want to work on the above problems * care about what's going on under the hood * have to build high-performance systems * need to diagnose bugs or security problems ##### Class structure * See web site: https://pdos.csail.mit.edu/6.828 * Lectures * O/S ideas * detailed inspection of xv6, a traditional O/S * xv6 programming homework to motivate lectures * papers on some recent topics * Labs: JOS, a small O/S for x86 in an exokernel style * you build it, 5 labs + final lab of your choice * kernel interface: expose hardware, but protect -- few abstractions! * unprivileged user-level library: fork, exec, pipe, ... * applications: file system, shell, .. * development environment: gcc, qemu * lab 1 is out * Two exams: midterm during class meeting, final in finals week ##### Introduction to system calls * 6.828 is largely about design and implementation of system call interface. let's look at how programs use that interface. we'll focus on UNIX (Linux, Mac, POSIX, &c). * a simple example: what system calls does "ls" call? * Trace system calls: * On OSX: sudo dtruss /bin/ls * On Linux: strace /bin/ls * so many system calls! * example: copy input to output ```sh cat copy.c cc -o copy copy.c ./copy ``` read a line, then write a line note: written in C, the traditional O/S language * first read/write argument is a "file descriptor" (fd) passed to kernel to tell it what "open file" to read/write must previously have been opened, connects to file/device/socket/&c UNIX convention: fd 0 is "standard input", 1 is "standard output" * sudo dtruss ./copy read(0x0, "123\n\0", 0x80) = 4 0 write(0x1, "123\n@\213\002\0", 0x4) = 4 0 * example: creating a file ```sh cat open.c cc -o open open.c ./open cat output.txt ``` note: creat() turned into open() note: can see actual FD with dtruss note: this code ignores errors -- don't be this sloppy! * example: redirecting standard output ```sh cat redirect.c cc -o redirect redirect.c ./redirect cat output.txt man dup2 sudo dtruss ./redirect ``` note: writes output.txt via fd 1 note: stderr (standard error) is fd 2 -- that's why creat() yields FD 3 * a more interesting program: the Unix shell. * it's the Unix command-line user interface * it's a good illustration of the UNIX system call API * some example commands: ```sh ls ls > junk ls | wc -l ls | wc -l > junk ``` * the shell is also a programming/scripting language ```sh cat > script echo one echo two sh < script ``` * the shell uses system calls to set up redirection, pipes, waiting programs like wc are ignorant of input/output setup * Let's look at source for a simple shell, sh.c * main() basic organization: parse into tree, then run main process: getcmd, fork, wait child process: parsecmd, runcmd why the fork()? we need a new process for the command what does fork() do? copies user memory copies kernel state e.g. file descriptors so "child" is almost identical to "parent" child has different "process ID" both processes now run, in parallel fork returns twice, once in parent, once in child fork returns child PID to parent fork returns 0 to child so sh calls runcmd() in the child process why the wait()? what if child exits before parent calls wait()? * runcmd() executes parse tree generated by parsecmd() distinct cmd types for simple command, redirection, pipe * runcmd() for simple command with arguments execvp(cmd, args) man execvp ls command &c exist as executable files, e.g. /bin/ls execvp loads executable file over memory of current process jumps to start of executable -- main() note: execvp doesn't return if all goes well note: execvp() only returns if it can't find the executable file note: it's the shell child that's replaced with execvp() note: the main shell process is still wait()ing for the child * how does runcmd() handle I/O redirection? e.g. echo hello > junk parsecmd() produces tree with two nodes cmd->type='>', cmd->file="junk", cmd->cmd=... cmd->type=' ', cmd->argv=["echo", "hello"] the open(); dup2() causes FD 1 to be replaced with FD to output file it's the shell child process that changes its FD 1 execvp preserves the FD setup so echo runs with FD 1 connected to file junk again, very nice that echo is oblivious, just writes FD 1 * why are fork and exec separate? perhaps wasteful that fork copies shell memory, only to have it thrown away by exec the point: the child gets a chance to change FD setup before calling exec and the parent's FD set is not disturbed you'll implement tricks to avoid fork() copy cost in the labs * how does the shell implement pipelines? $ ls | wc -l * the kernel provides a pipe abstraction int fds[2] pipe(fds) a pair of file descriptors: a write FD, and a read FD data written to the write FD appears on the read FD * example: pipe1.c read() blocks until data is available write() blocks if pipe buffer is full * pipe file descriptors are inherited across fork so pipes can be used to communicate between processes example: pipe2.c for many programs, just like file I/O, so pipes work for stdin/stdout * for ls | wc -l, shell must: - create a pipe - fork - set up fd 1 to be the pipe write FD - exec ls - set up wc's fd 0 to be pipe read FD - exec wc - wait for wc [diagram: sh parent, ls child, wc child, stdin/out for each] case '|' in sh.c note: sh close()es unused FDs so exit of writer produces EOF at reader * you'll implement pieces of a shell in an upcoming homework ## 参考资料 1. [MIT 6.828: Operating System Engineering](https://pdos.csail.mit.edu/6.828/2017/index.html#top) 2. [OpenCourseWare](https://ocw.mit.edu/courses/6-828-operating-system-engineering-fall-2012/) 3. [MIT 6.828 操作系统课实验中文翻译](https://github.com/jacksonwuu/MIT-6.828-zh) 4. [xv6 中文文档](https://th0ar.gitbooks.io/xv6-chinese/content/index.html) 5. [MIT6.s081课程教材](https://mit-public-courses-cn-translatio.gitbook.io/mit6-s081/lec01-introduction-and-examples)