TDC368 Week1

TDC368 UNIX and Network Programming
Week 1: Course Introduction Overview of UNIX Operating Systems Architecture UNIX process concepts
Camelia Zlatea, PhD Email: czlatea@cs.depaul.edu
Course Outline
y Overview of UNIX operating system architecture. y UNIX process concepts. Process control primitives (fork, exec, exit, wait, waitpid). y Filters and I/O Redirection. Interprocess Comunications (IPC): pipes, FIFO's. y Asynchronous Events. Signals. Signal Functions (kill, system, sleep, raise, alarm, pause, abort). y UNIX system calls: C library functions, C++ classes, POSIX API's. y UNIX File System: files, catalogs (inodes; types,size, permissions). y UNIX File I/O: unbuffered I/O functions (open ,close,seek,umask,read,write). y Standard I/O buffering (fopen, fclose, fflush). File Sharing (dup,dup2,fcntl).
UNIX Network Programming TDC368-901 Spring 2003 Page 2
Course Outline (cont.)

y Interprocess Communication: Shared Memories, Semaphores, POSIX Semaphores, lock files. y Socket API (TCP, UDP). y Design Methods for Client/Server Applications y UNIX shells. C-shell commands, shell variables. C-Shell Programming. y Managing UNIX projects (make). Compilation and Debugging. y UNIX shells: I/O redirection, filters(grep, sed), awk.
UNIX Network Programming TDC368-901
Spring 2003
Page 3
Objectives Upon completion of this course, students

will understand the fundamental concepts of UNIX operating system and use them to create various multitasking applications. Will be familiar with UNIX programming environment, system call interface, ANSI C and ANSI/ISO C++ functions and classes Will be familiar with Client/Server methods and Socket APIs Will be proficient in UNIX shell programming
Spring 2003
Page 4
Prerequisites
CSC 309 Basic understanding of operating systems and programming techniques C/C++ will be used to illustrate the concepts presented in the lecture material and for the implementation of the programming assignments and the project.
Spring 2003
Page 5
Texts
Dave Curry, UNIX Systems Programming for SVR4, O'Reilly & Assoc., Sebastopol, CA, 1996. W. Richard Stevens, UNIX Network Programming Interprocess Communication, Volume 2, Second Edition, Prentice Hall PTR, Upper Saddle River, NJ, 1999. W. Richard Stevens, Advanced Programming in the UNIX Environment, Addison-Wesley, Reading, MA, 1992. John Shapley Gray, Interprocess Communications in Unix, The Nooks & Crannies, 2nd ed. Prentice Hall PTR, NJ, 1998. HP-UX man pages
http://docs.hp.com/hpux/onlinedocs/B2355-90682/B2355-90682.html (system calls)

Spring 2003
Page 6
Other References
Daniel Gilly, Unix in a Nutshell, O'Reilly and Associates, Inc., Sebastopol, CA, 1992. K.A.Robbins and S. Robbins, Practical UNIX Programming - A Guide to Concurrency, Communication, and Multithreading , Prentice Hall PTR, NJ, 1996 Ellie Quingley, UNIX Shells by Example, Prentice Hall PTR, NJ, 1997 Prabhat K. Andleigh, UNIX System Architecture, PrenticeHall, Englewood Cliffs, NJ, 1990. Andrew Oran and Steve Talbot, Managing Projects with make, O'Reilly and Associates, Inc., Sebastopol, CA, 1991. Brian W. Kernighan and Dennis M. Ritchie, The C Programming Language, Second Edition, Prentice-Hall, Englewood Cliffs, NJ, 1989.
Spring 2003 Page 7

Class Work and Grading
Programming Assignment 1 Programming Assignment 2 Midterm Programming Assignment 3 Final Project
due by April 22th 20% due by May 6th 20% 20% May 13th due by May 27th 20% 20% due by June 10th
Spring 2003
Page 8
Programming Environment
All students will get an account on hawk.depaul.edu server (running HP-UX). Optional, you can use Linux system Project submissions via COL (dlweb.cti.depaul.edu )
use Submit Assignments link
Grading will be done on HP-UX or Linux

An experimental report is required for each programming project
The the report should include the program design description and the test cases and test scenarios for the application.
Spring 2003
Page 9
TDC368 Network Programming

Instructor: Camelia Zlatea, PhD
Office Hours: T 4:45-5:45PM; 9:00-9:30PM Where: Rm. 429, ext. x26149 (during office hours) or leave a message to setup a conference: +630-979-3621 Email: czlatea@cs.depaul.edu Web: http://condor.depaul.edu/~czlatea/TDC368/ (*)
Note: * In addition, course materials will be available from dlweb.cti.depaul.edu
Spring 2003
Page 10
Class Communication
COL (dlweb.cti.depaul.edu) will be used primarily for:
Links to course materials/assignments Grades Announcements Email/Communication Newsgroup
Asking questions
Send me an email at czlatea@cs.depaul.edu OR Use dlweb.cti.depaul.edu and post questions to class forum Anybody may reply (including your classmates) Everyone benefits from common issues
Spring 2003
Page 11
UNIX - Brief History
1969, Ken Thompson, Bell-Labs. UNIX OS written in Assembly on PDP-7 (DEC) 1973, K.Thompson&D.Richie UNIX re-written in C 1975, Bell-Labs. offered UNIX to educational institutions, at minimal cost
Spring 2003
Page 12
UNIX Features

Portability Multi-process architecture (multitasking) Multi-user capability Ability to initiate asynchronous processes A hierarchical file system Device independent I/O operations User interface: Shell; selectable per user basis
Spring 2003
Page 13
UNIX Major Versions

AT&T UNIX version V 1983 /SV, 1986 /SVR3.0 AT&T and Sun Microsystems 1988 /SVR4.0 - tries to unify Berkeley UNIX & other Berkeley UNIX, CSRG at U. of California BSD (Berkeley Software Distribution)
Spring 2003
Page 14
UNIX Standards
System V Interface Definition (SVID), AT&T Portable Operating System Interface for Computer Environments (POSIX), based on SVID, IEEE ANSI C, American National Standard Institute ANSI/ISO C++ Standard (draft)
Spring 2003
Page 15
UNIX Implementations
Solaris - Sun Microsystems SunOS (later called Solaris), Solaris 2.x based on SVR4 HP-UX, Hewlett-Packard, SVR2 Linux, Linus Torvalds, free distribution, PC-based AIX, IBM, similar to SVR4 IRIX, Silicon Graphics, SVR4
Spring 2003
Page 16
UNIX Implementations
UNIX-like same facilities as SV, does not use entirely the same code Mach (CMU), kernel is based on OO model server Next (implementation inspired by Mach with MacOS UI) UNIX-alike portability is no longer supported attributes not compliant with SV or BSD OS/9(MicroWare, Motorola 680xx, RT kernel) QNX(Quantum Software, Intel, RT processing)
Spring 2003
Page 17
UNIX Architectural Overview
Applications
Network Applications DBMS
Utilities
Shells
Executable Programs
pipe, filters
Commands
System Calls Interface Kernel UNIX
Hardware Architecture
Spring 2003
Page 18
UNIX Features

Portability Multi-process architecture (multitasking) Multi-user capability Ability to initiate asynchronous processes A hierarchical file system Device independent I/O operations User interface: Shell; selectable per user basis
Spring 2003
Page 19
Multi-process/Multi-user architecture
Virtual Machine
timesharing OS process, process quantum, process states
Kernel, base OS
manages all HW dependent functions users have no direct access to it
System Calls Interface

service routine performing user requests
Spring 2003
Page 20
UNIX Processes
A process may create sub-processes

fork();
A process may terminate

exit();
A process may put itself to sleep temporarily

sleep(20); pause(); wait();
Processes
synchronization mechanisms communication mechanisms
Spring 2003
Page 21
UNIX Threads
Multiple Processes - concurrency at OS level Multiple Threads - concurrency at process level

thread = flow of control in a process multiple threads (stream of instructions) are executed within the same process threads share code & data (address space) threads have their own execution stack, PC, register set and states context switches are avoided efficient mapping on multi-processor machines
Spring 2003
Page 22
Some Unix Terms
TRAP INSTRUCTION
switches a system call from user to kernel mode handles error conditions synchronous execution Ex: division by zero/overflow
INTERRUPT
used by peripherals to request services from the devices handles asynchronous execution
SIGNALS software notification of an event example:

when typing ctrl-c an interrupt is generated for the keyboard driver, which notifies the process by sending a signal
Spring 2003
Page 23
UNIX Kernel - model
User User Processes System Call Interface (Library Routines)

Process Memory File System I/O Services Mgmt. Mgmt.
Space
Kernel Space
Scheduler
Device Drivers Hardware
I/O Buffers
Spring 2003
Page 24
Apps./Utilities
System Calls UNIX KERNEL Hardware Interrupts
Hardware
Spring 2003
Page 25
UNIX Kernel - low level

Process representation, scheduling, dispatching Memory allocation and de-allocation Interrupt handling Low level device control Disk Mgmt., data buffering Process synchronization and IPC
Spring 2003
Page 26
Kernel - services level
Maps user-level requests with device driver actions A user system call is translated to a call of the kernel routine, providing that requested service Type of Services:
process creation and termination I/O services UNIX file system services terminal handling services
Spring 2003
Page 27
Kernel - services level
Executable Programs
User program: system(ps -e); System calls actions: fork(); //process creation service execl(/bin/ps, ps -e); //code execution
Library Functions
System Calls Kernel Hardware
Spring 2003
Page 28
System Call Interface level
A user mode process is translated into a protected kernel mode process Now, program can call kernel routines
Spring 2003
Page 29
User Processes level
User processes running:

shells Unix commands utilities application programs
Spring 2003
Page 30
UNIX Standards

UNIX and POSIX API Library Functions ANSI C Standard ANSI/ISO C++ Standard
Spring 2003
Page 31
UNIX and POSIX API

UNIX API - system calls UNIX API are called by
C library functions and C++ standard classes Example: iostream class
Program confirms POSIX.1?

#define _POSIX_SOURCE or % CC -D_POSIX_SOURCE *.C
Spring 2003
Page 32
UNIX and POSIX API

API set to perform:
determine system configuration and user information file management process creation and management inter-process communication network communication
Spring 2003
Page 33
UNIX and POSIX API
User Process (User Mode of Execution) an API is invoked API execution completed UNIX APIs level Kernel mo e of execution API executed in protected mode
UNIX and POSIX API

Context Switch from user to kernel mode
more overhead than library functions, for the same task I/O lib.functions are buffered
Spring 2003
Page 35
UNIX and POSIX API

UNIX API - system calls UNIX API are called by
C library functions and C++ standard classes Example: iostream class
Program confirms POSIX.1?

#define _POSIX_SOURCE or % CC -D_POSIX_SOURCE *.C
Spring 2003
Page 36
UNIX and POSIX API
Development elements: Headers

<unistd.h> <sys> (/usr/include/sys directory) <stdio.h> function perror - system diagnostic message for an API execution failure
Object Code
libc.a and libc.so (on most systems) other special libraries, libsocket.so libsocket.a and
Spring 2003
Page 37
UNIX and POSIX API
#include <stdio.h> void perror(const char *s); #include <errno.h> int errno; #include <string.h> char *strerror(int errnum);
Spring 2003
Page 38
Library Functions
Library of functions object code of the functions stored in archives (library files) run-time library routines UNIX archive ar utility manage library files. Displaying the table of contents of the standard library: ar -t /usr/lib/libc.a
Spring 2003
Page 39
ANSI C Standard
Standard C library functions: memory management get system information get/set time information data conversion, processing Advantages: portability maintainability cost effective for application design Disadvantages: duration of execution weak support for time-critical apps.
Spring 2003
Page 40
Standard C library functions

Header files directory: Header files:
- <stdio.h> - <string.h> - <memory.h> - <time.h> - <assert.h> - <getopt.h> - <stdlib.h> - <malloc.h> - <stdarg.h> - <setjmp.h>
/usr/include
Object code directory:

libc.a , libc.so
/usr/lib
Spring 2003
Page 41
<stdio.h> - declares FILE data type functions/macros -manage stream files open/close stream files: fopen/fclose read/write block data: fread/fwrite read/write lines: fgets/fputs write formatted data: fprintf positions/returns stream file location:fseek/ftell macros: feof, ferror unnamed pipes: popen/pclose
Spring 2003
Page 42
Example: ls -l|grep tdc368

#include <stdio.h> int main() { /* execute the command */ FILE *cmd = popen(ls -l|grep tdc368,r); if (!cmd) { perror(popen); return 1; } char answer[256]; /* read grep process execution output */ while(fgets(answer, sizeof(answer),cmd)) fputs(answer, stdout); /* echo */ pclose(cmd); return 0; }
Spring 2003
Page 43
<stdlib.h> - declares set of functions for data conversions, get/set env. variables, shell command execution, process termination
atoi, atof, atol, strtod, strtol rand, srand getenv, putenv system exit
Spring 2003
Page 44
Example: ls -l|grep tdc368

#include <stdlib.h> int main() { /* execute the command */ if (system(ls -l|grep tdc368)== -1) { perror(system); return 1; } return 0; }
Spring 2003
Page 45
Example: getenv/putenv
#include <iostream.h> #include <stdlib.h> int main() { char *env = getenv(REMOTEHOST); cout << \REMOTEHOST\ is:<<env<<\n; if (putenv(CC=c++)) cer<<putenv of CC failed\n); return 0; }
Spring 2003
Page 46
Example: <malloc.h>
//C style char* adr1 = (char *)malloc(4096); free(adr1);
//C++ style , no use of malloc ! char* adr2 = new char[4096]; delete adr2;
Spring 2003
Page 47
<time.h> - declares set of functions for system clock query time - #sec since 01/01/70 (UNIX birth) ctime - current local time localtime gmtime asctime mktime clock - (ANSI C) , #microsec since the process first called clock
Spring 2003
Page 48
Example: <time.h>
// evaluate process execution time #include <iostream.h> #include <time.h> main() { time_t tick = CLOCKS_PER_SECOND; clock_t start_time = clock(); // start timer /* process code here */ clock_t duration = clock() - start_time; cout <<Duration:<<(duration/tick)<<endl; }
Spring 2003
Page 49

<assert.h> - declares a macro to assert conditions or invariants of the program
easier program debugging compiler option -DNDEBUG take them out or #ifndef NDEBUG #define assert(p) {if (!(p)) {\ fprintf(stderr,Assertion failed: \ file:\%s\,line %d\n,\ _FILE_, _LINE_); exit(1); } #endif
Spring 2003
Page 50
Example: <assert.h>
#include <fstream.h> #include <string.h> #include <assert.h> int main(int argc, char * argv[]) { assert(argc >1); // should have 1 arg ifstream ifs(argv[1]) assert(ifs.good()); // should be opened OK char *adr = new char[strlen(argv[1])+1]; assert(adr); // should not be NULL }
Spring 2003
Page 51
ANSI/ISO C++ Standard

1980, Bjarne Stroustrup, C++ 1994, ANSI/ISO C++ standard (draft) C++ compiler (AT&T C++ V3.0) support:
C++ classes, derived classes virtual functions, operator overloading templates classes, functions exception handling iostream classes

Spring 2003
Page 52
Compilation cc -o file file.c file Man Pages man cc man sys_call man shell_cmd
Spring 2003
Page 53
Processes
Process - a program in execution
process - active entity program - passive entity (binary file)
Address Space - list of memory locations from where a process reads/writes (code/data/stack) Set of registers (PC, SP, ...) Process Table - linked list of structures associates w/ processes System Calls - interface between OS and User process
Spring 2003
Page 54
Process Control Block (process attributes)
Process State
new, ready, running, blocked, terminated
Process Image Map Pointer Process ID

assigned at process creation
Program Counter (PC)

address of next instruction to be executed

CPU Registers (saved process context) List of Open File Descriptors I/O Devices Attached CPU Scheduling Info (priority)
Spring 2003
Page 55
Process Image Map
Process Table Proc. 1
Process Image Text/Code Segment
Proc. n Data Segment Stack Segment Process Control Block

P r o c e ss M e m o r y p ro g ra m
M a p L o A r e ss
te x t s e g m e n t s ta tic d a ta s ta tic d a ta U s e r a d d re s s a b le a re a
in itia liz e d
u n in itia liz e d h e a p s ta c k a rg c
a rg v ; e n v iro n m e n t P ro c e s s In fo . S ta c k
S y s te m S y s te m
S y s te m a d d re s s a b le a re a ig h A r e ss
S y ste m
M e m o r y
M a p
p ro c e ss 0 p ro c e ss 1
.....
p ro c e ss i
....
U N IX u a re a
D a ta S tru c tu re s
B u ffe rs K e rn e l c o d e L o w le v e l d e v ic e d riv e rs
Spring 2003 Page 57
Example:
/* Display Segment Address Information */ #include <stdio.h> extern int etext,edata,end; void main(void) { printf(etext: %6X\t edata: %6X \t end: %6X \n, &etext, &edata, &end); }
Spring 2003
Page 58
Process States
New - process created ( Ex: fork(); ) Ready process is waiting to be assigned to processor (inserted in ready queue) Running - instructions are being executed Blocked - wait for events to occur (inserted in queue) Ex: wait(); pause(); Terminated - normal/abnormal termination (exit();)
Spring 2003
Page 59
Process Model
Ne wakeup Blocke / Suspen e
created
Rea y dispatch
Quantum Expired
Running sleep User Mo e exit System Call Return Running Interrupt Kernel Mo e Terminate
Interrupt Interrupt return Spring 2003
Page 60
Context of a Process
process state (defined by its code) value of u-area values of registers the process uses contents of user and kernel stacks is associated with process image map
Context Switching
system executes a process in the context of the process when the kernel decides to execute another process, it does context switching kernel saves enough information such that it can later switch back to the first process and resumes its execution
Mode Switching
moving from user to kernel mode kernel save information to return to user mode
Spring 2003
Page 61
User mode
processes in use mode can access their own instructions and data; NOT kernel or other processs code or data
Kernel mode
process can access system(kernel) code and data and user addresses Kernel is part of each process Kernel executes on behalf of the process
P1
P2
P3
P4
OS HW Kernel Mode User Mode

K U U
Spring 2003
Page 62
Context Switching
P2
P1
OS Save state in PCB1 Reload state from PCB2 Save state in PCB2 Reload state from PCB1
Spring 2003
Page 63
Context Switching
Switching the CPU to another process by saving the state of the old process (PCB) and load the state of the new process (PCB) Pure Overhead Performance Bottleneck Avoid Overhead of Context Switching by introducing new structures: THREADS
Spring 2003
Page 64
Context Switching
Multitasking Sequential Execution
Spring 2003
Page 65

TDC368 Week1

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

TDC368 Week1

Uploaded by

Copyright:

Available Formats

TDC368 UNIX and Network Programming

Camelia Zlatea, PhD Email: czlatea@cs.depaul.edu

Course Outline (cont.)

UNIX Network Programming TDC368-901

Objectives  Upon completion of this course, students

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

Class Work and Grading

Programming Assignment 1 Programming Assignment 2 Midterm Programming Assignment 3 Final Project

UNIX Network Programming TDC368-901

 Grading will be done on HP-UX or Linux

UNIX Network Programming TDC368-901

TDC368 Network Programming

Note: * In addition, course materials will be available from dlweb.cti.depaul.edu

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

UNIX - Brief History

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

UNIX Major Versions

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

UNIX Architectural Overview

Network Applications DBMS

System Calls Interface Kernel UNIX

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

 System Calls Interface

UNIX Network Programming TDC368-901

 A process may create sub-processes

 A process may terminate

 A process may put itself to sleep temporarily

UNIX Network Programming TDC368-901

 Multiple Processes - concurrency at OS level  Multiple Threads - concurrency at process level

UNIX Network Programming TDC368-901

Some Unix Terms

 SIGNALS software notification of an event  example:

UNIX Network Programming TDC368-901

UNIX Kernel - model

User User Processes System Call Interface (Library Routines)

Device Drivers Hardware

UNIX Network Programming TDC368-901

System Calls UNIX KERNEL Hardware Interrupts

UNIX Network Programming TDC368-901

UNIX Kernel - low level

UNIX Network Programming TDC368-901

Kernel - services level

UNIX Network Programming TDC368-901

Kernel - services level

System Calls Kernel Hardware

UNIX Network Programming TDC368-901

System Call Interface level

UNIX Network Programming TDC368-901

User Processes level

 User processes running:

UNIX Network Programming TDC368-901

UNIX Network Programming TDC368-901

UNIX and POSIX API

Objectives Upon completion of this course, students

Grading will be done on HP-UX or Linux

System Calls Interface

A process may create sub-processes

A process may terminate

A process may put itself to sleep temporarily

Multiple Processes - concurrency at OS level Multiple Threads - concurrency at process level

SIGNALS software notification of an event example:

User processes running:

Program confirms POSIX.1?

Program confirms POSIX.1?

Development elements: Headers

Object code directory:

Process Image Map Pointer Process ID

Program Counter (PC)