The Operating System: The Purpose

An operating system provides an environment for the execution of programs

by providing services needed by those programs.

The services programs request fall into five categories

1. Process Control
2. File System Management
3. I/O Operation
4. Interprocess Communication
5. Information Maintenance

The operating system must try to satisfy these requests in a multi-user, multi-process
environment while managing
• Resource allocation
• Error Detection
• Protection
 The Operating System: Process Control

A process is the unit of work.

Processes need to be able to control their own execution, as well as spawn new processes
to perform tasks concurrently. Processes request services to
• fork : create a new process
• exit: normal termination, abort: abnormal termination
• execve: load and execute another program
• wait : wait for another process to finish
• Signal Management: handling asynchronous events
– signal, sigaction : setting signal handlers
– kill : sending signal
• Threads: creating and managing multiple execution threads in a single process
 The File System: The User’s View

The User sees . . .

ddedeh YWWYY
d ddd d
e d
eedededhedhedhedhedhoehohoo \ NNNWNWYNWYWYWYWYWYWYYYYY
d
dd ee h o NNN WWWWWYYYYYYY
dddddddeeeee hhhh oo WWWW YYYYYYY
ddddededededeeeeeehehhhhhhh oooo NNN WWWW
ddddddd dddddd
e eee hhh oo NNN WWWW YYYYYYYYYYY
dq dddddd reeeeee shhhh wo  & WW+ YYYY,
...
bin etc dev home proc cdrom usr var

       
bash passwd pts kaharris 1 assign1 include log

    
2 51081OO status stdio.h wtmp
OOO
OOO
OOO
 O'
assign1 notes

but underneath the hood things are not as they first appear . . .
The File System: Smoke and Mirrors

The Unix File System is much more complicated than it appears

• The root directory is located on hard disk whose file system is organized using a
Linux filesystem organization, ext2.
• There is a CD-ROM in the cd drive whose file system is organized using High-Sierra
File Systems, HSFS.
• My own directory, kaharris, is located on another server
– brahman.cs.uchicago.edu:/export/u2/kaharris
and sent to admiral using the Network File System, NFS.
• The directory /proc does not even exist on a device. The files are not storage loca-
tions of data yet they appear to have content: cat /proc/1/status or cat /proc/mounts
• The ”files” in /dev are actually physical devices. For example, /dev/pts/2 is my
terminal device, /dev/mouse is my mouse and /dev/lp0 is a printer.
The File System: Layers of Abstraction

Block Physical Devices

Device Drivers

Individual File Systems

Virtual File System

Program

User
 The Operating System: File System Management

The Unix File System is a high-level organization of the resources available

to processes and users.

The Unix File System is organized as an acyclic directed graph with nodes representing
files and arcs representing a containment relation. Directories are nodes with children,
so contain files. Processes move around and modify the structure:
• chdir : moving to a new directory
• getcwd : get the current working directory
• opendir, closedir : open and close a directory
• readdir : read from a directory
• stat, fstat : Retrieving file status information
• link, unlink : create and release a hard link (an alias for a file.)
• symlink : create a soft link (a pointer to another file)
 The Operating System: I/O Operation

Unix uses a uniform device interface that allows the same I/O calls to be
used for terminals, disks, audio and network communication.

Unix provides a universal interface for I/O:

• open : open a file or device for I/O operations
• close : close a file or device from I/O operations
• read : read from a file or device
• write : write to a file or device
• Refined I/O control:
– fcntl : getting and setting attributes of an open file
– ioctl : getting device status information and setting device control options
– poll,select : handling I/O from multiple sources
 The Operating System: Interprocess Communication

Concurrently running processes need to communicate to work together effec-

tively.

Unix provides a variety of means for processes to communicate with each other:
• pipe : a one-way data stream between related processes
• mkfifo : a one-way data stream between unrelated precesses (called a named pipe
or FIFO)
• System V IPC: refined communication channels for unrelated processes
– Message Queues : Linked lists of messages stored in the operating system
– Shared Memory : Allows two processes to share a given region of memory
– Semaphores : Provides controlled access to a shared object
• Sockets : Two-way data stream, used to establish Network connections.
 The Operating System: System Information

Unix provides means for accessing information about the system for process use or
accounting:
• Special directories which provide Start-up and Run-time information. Some of the
files are specially configured, so provide special functions to retrieve information.
Examples
– /etc/ : System configuration files. Examples
∗ /etc/passwd
∗ /etc/group
– /var/ : Runtime changable files. Examples
∗ /var/run/utmp : Currently logged in users
∗ /var/log/wtmp : All logins and logouts
– /proc/ : Process information
• Time and Date functions in <time.h>
• uname : system identification
• sysconf, pathconf : Information on system limits
 Unix System Call: The System Call Model

System Calls are requests by programs for services to be provided by the

Operating System.

The system call represents the interface of a program to the kernel of the Operating
System
1. The system call API (Application Program Interface) is given in C
2. The system call is actually a wrapper routine typically consisting of a short piece of
assembly language code.
3. The system call generates a hardware trap, and passes the user’s arguments and an
index into the kernel’s system call table for the service requested.
4. The kernel processes the request by looking-up the index passed to see the service
to perform, and carries out the request.
5. Upon completion, the kernel returns a value which represents either successful com-
pletion of the request or an error. If an error occurs, the kernel sets a global variable,
errno, indicating the reason for the error.
6. The process is delivered the return value and continues its execution.
Unix System Call: Motivation behind the Model

The system call model ensures safety, fairness and modularity.

The benefits of the system call model include

• The Operating System ensures the integretity of its data and the data of other users,
by carefully controling access to these resources.
• The Operating System ensures fair use of system resources.
• The system call interface is standardized by POSIX (Portable Operating System
Interface), so is identical for all Unix flavors. (BSD, Linux, Solaris, Mach, etc.)
regardless how any operating system actually implements the services the process
requests.
• The system call interface simplifies program construction by removing the details of
implementing the request to the operating system.
 The System Call Model: Layers of Interaction

There are layers of interaction in the system to ensure fair access to resources,
protect privacy, and provide convenience.

Users
O

User Programs
O

Kernel
O

CPU/Devices
 System Calls versus Standard Library Functions

The system call model is very different from the implementation of Standard
C Library functions

• Library functions provide an interface which may be run on any operating system
which has a C compiler; System calls are specific to Unix Operating Systems.
• Library functions are executed by your program; System calls are executed by the
kernel.
• Library functions handle error checking; you are responsible for error checking with
system calls.
• Library functions provide a convenient interface: such as I/O buffering and formatted
input and output; system calls are raw–the programmer is responsible for making
them convenient.
• The standard C library interface is found in the manpages in section 3; the system
call interface is found in the manpages in section 2.
Error handling functions for System Calls

SYNOPSIS

#include <errno.h>

int errno;

SYNOPSIS

#include <stdlib.h>

void perror(const char *s);

Return: No return value; No error

SYNOPSIS

#include <string.h>

char *strerror(int errnum);

Return: Appropriate description string or an unknown error message; No error

 Error Handling on System Calls

You must examine the return value of every library function that can return
an error.

• Traditionally, Unix system calls return -1 (or sometimes NULL) and set a global
variable errno.
• POSIX standard committee has specified a list of errors that each system call must
report. In addition to these Unix implementations may have more errors. The
manpages list all possible errors for each system call on the system.
• There are two standard C library functions for reading errors:
– perror(s) prints a message to standard error: s :msg where msg is a short
description of the error. The text of msg is based upon the current value of
errno.
– strerror(errnum) returns a pointer to a string which contains a short descrip-
tion of the error number errnum. Beware: you may not modify this string, and
subsequent calls to strerr may overwrite this string!!
• Beware: errno is a global variable. Many standard C library routines use system
calls which may overwrite the value in errno. If you need this value, then save it to
another variable.
System Calls for I/O Operation: The Universal Interface

Everything in Unix is a file.

• All I/O devices are represented by files that are located in directory /dev and can
be accessed for I/O exactly as any regular file on disk.
• The Kernel communicates to devices using device drivers.
• Processes access and control devices for I/O through four system calls to the Kernel.
• I/O operation is a simple three step procedure for any file, whether it is an I/O
device or regular file:
1. Open the file for I/O.
2. Read and Write to the file.
3. Close the file when finished with I/O.
• Access to an open file is through a file descriptor.
File I/O: The Universal Interface

Everything in Unix is a file.

All devices are represented by files that are located in directory /dev
• /dev/tty is the terminal
• /dev/null is a byte black hole: reading from it always produces EOF and writing
to it eats bytes.
• /dev/zero is an infinite source of 0’s
• /dev/stdin /dev/stdout , /dev/stderr . Example of use:
$: cat file1 /dev/stdin
Outputs file1 then waits for a line of input and appends this to the end of its
output. Equivalent to cat file1 - .
• See Molay, Understanding Unix/Linux Programming section 5.2 for further discus-
sion of some of the devices found in the directory /dev .
File Descriptors: The Process handle on I/O

• A file descriptor is a non-negative integer, which is an offset in the process file

descriptor table.
• Each process has its own file descriptor table which is stored in the process memory
space. This will be a very important point for later.
• Each process is allowed OPEN MAX file descriptors in use at one time. (This value is
defined in <limits.h> and is at least 64.)
• Each process starts with three open file descriptors: STDIN_FILENO , STDOUT_FILENO ,
STDERR_FILENO . These constants are defined in unistd.h and by convention have
magic numbers 0, 1, 2.
• /dev/fd/n is equivalent to the process file descriptor n. For example, the following
lines perform the same task
fd = open("/dev/fd/0", mode);
fd = open("/dev/stdin", mode);
They duplicates file descriptor 0, so now fd refers to standard input.
• In fact, the file /dev/stdin is a pointer (a soft link) to /dev/fd/0. Similarly, for
/dev/stdout and /dev/stderr
File I/O: System Calls

Most Unix file I/O can be performed using only the following five functions:

• open: open a file descriptor

• read: read from a file descriptor
• write: write to a file descriptor
• close: close a file descriptor
• lseek: reposition file offset (this system call only makes sense for regular files and
block devices which allow random access to their data.)
read: read from a file descriptor

SYNOPSIS

#include <unistd.h>

ssize t read(int fd, void *buf, size t nbyte);

Return: number of bytes read on success or 0 on EOF -1 on error

read: Some Errors

errno

EINTR Call was interrupted by signal before any data was read

EBADF fd not a valid file descriptor or not open for reading

EAGAIN O NONBLOCK is set and the read would block

EISDIR fd refers to a directory.

read: Usage

• The read operation starts at the file’s current offset.

• Make sure you have allocated nbytes for buf
• read may read less than the amount requested
– When reading from a regular file, the end of file is reached before the number of
requested bytes read.
– When reading from a terminal device, normally up to one line is read.
– When reading from buffered devices such as a network or a pipe, buffering may
cause less than the requested amount read.
• Return value is of type ssize_t which may be int, but may be larger. The argument
nbyte is size_t which is an unsigned integer.
• Always check for errors, but you may not always want to bail-out:
while( (rbyte = read(fd,buf,nbyte)) == -1 && errno == EINTR ) ;
See the restart library.
write: write to a file descriptor

SYNOPSIS

#include <unistd.h>

ssize t write(int fd, const void *buf, size t nbyte);

Return: number of bytes written on success -1 on error

write: Some Errors

errno

EINTR Call was interrupted by signal before any data written

EBADF fd not a valid file descriptor or not open for writing

EAGAIN O NONBLOCK is set and the write would block

fd is connected to a pipe or socket whose reading end

EPIPE
is closed

Attempting to write a file that exceeds implementation-

EFBIG defined maximum; or, starting at a position exceeds
offset maximum

ENOSPC No free space remaining on the device containing the file

write: Usage

• The write operation starts at the file’s current offset.

• After writing the file’s offset is incremented by the number of bytes written.
• write may write less than the amount requested. In this case an error is generated.
– When attempting to write a file that exceeds implementation-defined maximum;
or, starting at a position exceeds offset maximum.
– There is noo free space remaining on the device containing the file.
• Return value is of type ssize_t which may be int, but may be larger. The argument
nbyte is size_t which is an unsigned integer.
• Some file systems (such as the minix files system) place a limit on file sizes. Writing
beyond this size sets an error, EFBIG. This is not the case for the standard files
systems: ufs on Unix systems and ext2 for Linux systems.
• Always check for errors, but you may not always want to bail-out. See the restart
library.
open: open a file or device

SYNOPSIS

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

int open(const char *path, int flags);

int open(const char *path, int flags, mode t mode);
int creat(const char *path, mode t mode);

Return: file descriptor on success -1 on error

open: Some Errors
errno

EINTR open call was interrupted by signal

Requested access somewhere along path is not allowed

EACCES

EISDIR Requesting to open a directory

EEXIST path exists and O CREAT and O EXCL used

EMFILE OPEN MAX file descriptors currently open in process

Maximum allowable number of open files currently

ENFILE
open in system

A directory component in path does not exist or is a

ENOENT
dangling symbolic link.

ENAMETOOLONG path was too long

 open: Options

The argument flag is obtained by OR’ing together one of the following constants with
an optional flag (defined in <fcntl.h>)

O_RDONLY Open for reading only

O_WRONLY Open for writing only
O_RDWR Open for reading and writing

Example:
fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, mode);
creates the file path for writing only. This operation is equivalent to
fd = creat(path, mode);
Example:
fd = open(path, O_RDWR | O_CREAT | O_EXCL, mode);
Guarantees the file path is a new file or returns EEXIST error.
open: Flags
flag

O APPEND Append to end of file on each write.

Create the file if it does not exist. Requires third

O CREAT argument, mode which specifies the access permissions
for the new file.

Generate error EEXIST when combined with O CREAT,

O EXCL if the file already exists. This makes the existence check
and file creation an atomic operation.

If the file exists and opened for either write-only or

O TRUNC
read-write only, truncate length to 0

Set nonblocking mode for subsequent I/O. Used with

O NONBLOCK
pipes, FIFO or device special files.

Prevents device from becoming controling terminal.

O NOCTTY
Used with terminal devices only.
open: Access Permissions

Always use the defined constants

mode
S_IRUSR read by owner
S_IWUSR write by owner
S_IXUSR execute by owner
S_IRWXU read, write, execute by owner
S_IRGRP read by group
S_IWGRP write by group
S_IXGRP execute by group
S_IRWXG read, write, execute by group
S_IROTH read by others
S_IWOTH write by others
S_IXOTH execute by others
S_IRWXO read, write, execute by others
S_ISUID set user ID on execution
S_ISGID set group ID on execution
close: close a file descriptor

SYNOPSIS

#include <unistd.h>

int close(int fd);

Return: 0 on success -1 on error

close: All Errors

errno

EINTR close call was interrupted by signal

EFBAD fd is not a valid file descriptor

 open and close: Usage

Usage of close
• When a process terminates all open files are automatically closed by the kernel.
• A process has a limited number of file descriptors available to it, OPEN_MAX (defined
in <limits.h>.)
• Closing a file releases any record locks that the process may have on the file.

Usage of open
• The initial file offset is set to 0 when opening file, unless O_APPEND is specified, in
which case the offset is set to the end of the file.
• The file descriptor returned by open is guaranteed to be the lowest numbered unused
descriptor. This is useful
close(STDOUT_FILENO);
fd = open("my_file,) O_WRONLY);
Any output to stdout will now go to the file my_file
Always check for errors, but you may not always want to bail-out. See the restart
library.
lseek: reposition read/write file offset

SYNOPSIS

#include <sys/types.h>
#include <unistd.h>

off t lseek(int fd, off t offset, int whence);

Return: new file offset on success -1 on error

whence

SEEK SET New file offset is offset bytes from beginning of the file.

SEEK CUR New file offset is current value plus offset bytes

SEEK END New file offset is the size of the file plus offset bytes
lseek: All Errors

errno

EBADF fd is not a valid file descriptor

fd is associated with a device which is incapable of

EPIPE seeking. Pipes, sockets, and FIFOs are all incapable of
seeking.

EINVAL whence is not a proper value.

lseek: Usage

• Every open file has associate a current file offset: the number of bytes from the
beginning of the file.
• Certain devices allow negative offsets; but for regular files the offset must be non-
negative. Thus, for regular files offset must be non-negative if whence is SEEK_SET.
• lseek only changes the current file offset within the kernel–it does not cause I/O to
take place. (See next point.)
• The file offset can be repositioned beyond the file size. In this case the next write
begins beyond the file size. This is referred to as creating a hole in the file. Any
bytes in the file that have not been written to are read back as 0. Why is this? (A
sparse file, such as a database with few entries.)
• lseek can be used to determine the current file offset
currpos = lseek(fd, 0, SEEK_CUR);
• The previous code can also be used to determine if the current file is a pipe, socket
of FIFO by testing (currpos == -1 && errno == EPIPE) . Why do we need the
second clause?