Ubuntu (operating system) UBUNTU Operating system concepts and its implementation

Submitted to: Sir Amir Raza

Submitted by: Noman Iqbal

MIT-10-02

3rd SEMESTER

Department Of Information Technology

Bahauddin Zakariya University

Multan.

Introduction General Information. Ubuntu is a computer operating system based on the Debian Linux distribution and distributed as free and open source software. It is named after the Southern African philosophy of Ubuntu ("humanity towards others").Ubuntu is designed primarily for use on personal computers, although a server edition also exists. Ubuntu is sponsored by the UK-based company Canonical Ltd., owned by South African entrepreneur Mark Shuttleworth. Canonical generates revenue by selling technical support and services related to Ubuntu, while the operating system itself is entirely free of charge. The Ubuntu project is committed to the principles of free software development; people are encouraged to use free software, improve it, and pass it on.

Ubuntu 11.04 - General Overview

General Information. UBUNTU Linux 11.04 (with code name natty narwhal) is a GNU/LINUX stable distribution provided by Canonical Ltd. and released on April 2011. It supports Intel x86 (IBM-compatible PC), AMD64 (Hammer) and PowerPC (Apple iBook and Powerbook, G4 and G5) architectures. It is available in both 32-bit and 64-bit versions. It is based on 2.6.38.2. kernel release and comes with UNITY graphical user interface. Features against competition - Target group. UBUNTU 11.04 Desktop edition, is a free Operating System committed to the principles of open source software development. It is and will always be free of charge, including updates. The release will be supported for 18 months. But this is not all. Through UBUNTU software Center, it provides a database for easy and fast access to free, compatible, open source applications that fit almost all needs. Useful everyday commands and symbols Comman Description

ls cd cp mv rm nano and vi man locate rmdir mkdir wget apt-get > |

List files and folders. Example: ls -l. Shows all files in folder, including hidden ones. Change folder. Example: 'cd ~/Documents' will move to Documents folder in the /home/current user folder Copy file or folder. Example: cp file.doc Desktop/, will copy file.doc from current path to /home/current user/desktop Move files or folders. If no destination path is provided, renames current file/folder Deletes files Non graphical editors, Gedit equievelant View the manual for a specified command Finds a specified file. It used a database which can be updated manually using sudo updatedb Deletes folder Creates folder Powerful download program (Advanced Packaging Tool) Redirects the output of a command to a specified location/file Pipe. Uses the output of one command as input for another

Ubuntu 11.04 - Process management

Introduction UBUNUT Linux11.04 is based on 2.6.38 Linux kernel. As far as the process management is concerned, UBUNTU Linux11.04 has preemptive kernel and supports Symmetric Multiprocessing (SMP). There are two spaces that we identify in process circle. The kernel space and the Processes and threads. Processes and threads are not that different for linux. Both fork() and exec() are used to create processes. Threads are created by the clone() system call. Clone() creates a child process allowing it to use shared information with the parent process by sets of flags that are passed to it. This how a thead is described in other Operating Systems. Exec() is the system call used to run new program whereas fork() is the

system call to create a new process. In general it can be said that fork() is similar to clone() when no flags are passed to the new process. (Operating Systems Concepts, p. 815) Process properties Process poperties fall into three groups:

1. Process identity

Process ID (PID)

It is the unique identifier of the process. It is used to specify the process to the OS. It does not change until the process termination.

Credentials

Also known as User ID (UID) and Group ID (GID). It determines the rights and privilegdes of the process to access system resources.

Personality

It can modify the semantics of certain system calls. It is used primarly by emulation libraries in order to request the combatible system call with certain varieties of UNIX.

2. Process environment It is inherited from its aprent process and is composed of two null-terminated vectors; the argument vector and the environment vector . The environment vector is a list of

NAME=VALUE paris that associate named enviroment variables with arbitary textual values. eg. The LANG variable determines the language that is to be used to display system messages.

2. Process Context

It represents the state of the process and it constantly changes. It inculdes the parts listed in the table 1 below. Description 1 Scheduling Context 2 Accoutning 3 File table 4 File-sysstem context 5 Signal-handler table 6 Virtual memory context Table 1. Parts of the Process context (Operating System Concepts, p.814)

Scheduling UBUNTU Linux 11.04 have two seperate process-scheduling algorithms. One is time-sharing for fair, preemptive scheduling among multiple processes; the other is designed fo real-time tasks. Real-time algorithms are also supported, for special timecritical applications that need precise control over the way in which runnable processes are selected for execution. The time sharing algorithms are three.

SCHED_OTHER SCHED_BATCH SCHED_IDLE

For the time sharing algorithms, the standard Round-Robin method is used. It is a First-Comes-First-Served (FCFS) method, but also assingning a time quanta for each process, creating a loop ending with the init() system call, which is the parent of all processes. Two seperate priority ranges are used. A real time range from 0-99 and a nice value ranging from 100 to 140. In this scheme the numerically lower processes have higher priority. When a task is runnable, it enters a runqueue, mainted by the kernel. The runqueue includes two arrays. The active list, which contains the processes to be executed, and the expired list, which contains the tasks that have expired. If the time quanta does not allow the task to be completed, then it is considerd as expired and it goes back to the expired array list of the runqueue. When the acitve array of the active list = 0 then the expired list becomes the active one and vice versa. The runqueue is processor indepented. The nice priority value is determined by how much time the task has been sleeping. It is dynamically changed and the scheduler infavors interactive tasks, which are tasks that have longer sleep times. The nice priority value is recalculated when the task enters the expired list. The real time algorithms are the FCFS and the Round-Robin, which was described above, with the addition of a priority to the one of its scheduling class. When two processes have the same class, the one waiting in queue the lognest has higher priority. For processesof different class the one with the hight class has the higher priority. Administration utilities used from the command-line

There are several commands that be used to monitor and manage the running processes of the system In order to interact with a task in terminal, we need to know either the PID or the name of the task. The table below summarises the most useful commands for process management through the terminal. The description of each command as well as the available arguments can be viewed by typing (command) --help, or man (the name of the command) Name ps Description With the appropriate arguments it can show all the running processes, the process runnning from a user pstree Shows the runnning processes in a tree scheme kill Kill a process by PID killall Kill a process by name nice Sets the priority of new processe renice Changes the priority of a currently running process skill Reports process status However, I believe that the most usefull command for the process management

1 2 3 4 5 6 7

throught the terminal is the command top. It is a real time non graphical System Monitor. Althought the number of the taks running is bigger and because it changes dynamically, the best proposed use of it is to use the results of the top command exporting them in home folder in a text file, like this top >> ~/top.txt, which will overwrite a text file in the home folder with the name top. It will provide with all the necessary information to manipulate the running tasks in the system. The h key shows the help menu na the q is used to exit the top command.

Ubuntu 11.04 - Memory management

Overview Allocation UBUNTU as a GNU/Linux distribution is designed for different architectures. Due to this characteristic, UBUNTU separates physical memory into three different zones:

zone_DMA zone_NORMAL zone_HIGHMEM

Zone_DMA is the zone that includes the first 16MB of physical memory. It is consisted of the DMA-able pages (Direct Access Memory). This happens because certain hardware, Industry Standard Architecture (ISA) devices, can only access the lower 16MB of physical memory. Zone_NORMAL is the zone that includes the normally addressable pages up to the first 896 MB of physical memory. The remaining physical memory (>896 MB) is allocated from zone_HIGHMEM. It is referred as high memory and includes the dynamically mapped pages. The primary physical-memory manager in the Linux kernel is the page allocator. There is one allocator for each zone, responsible for allocating, freeing all physical pages for each zone, and capable of allocating ranges of physically contiguous pages on request, but even though some allocations may require a page from a particular zone, a normal allocation can come form zone_DMA or zone_NORMAL. However, it is preferable to use a page from the particular zone, mostly to save the DMA space, but if needed and memory is low, Kernel can use whichever zone is available and suitable.

(Robert Love, Linux Kernel Development Second Edition). It is important to note that many kernel operations can only take place using zone that utilises available physical memory, reducing the percentage of internal fragmentation. However, as of 2.6.38 kernel, the one UBUNTU 11.04 uses the transparent huge page (THP) feature is added, which takes advantage of the page sizes that contemporary processors can handle simultaneously UBUNTU do very high use of virtual memory. mmap() service is responsible for mapping files or devices into memory. Linux reserves for its own internal use a constant, architecture-dependent region of the virtual memory address space of every process. The page-table entries that map to these kernel pages are marked as protected Two regions are used, one where the core of kernel, along with all pages allocated by the normal page allocator reside, and another which is reserved for no specific purpose. The vmalloc() service is used to allocate an arbitrary number of physical pages of memory that may not be contuguous into a single region of virtually contiguous kernel memory. The mmap() service has flag arguments that transform the mapping to shared with other processes, or private creating a private copy-on0write mapping. The virtual mapping procedure is directly connected to the way the process was created. If it is created through the exec() servicem then a new mapping address is provided. If it is created through fork() then the parent's pages are directly copied to the child's, resulting that the parent and child share physical pages in memory. UBUNTU 11.04 use the least frequently used (LFU) as page out policy .It is through the age variable of a page that it is chosen which page to swap.

Memory Management OS Utilities The only graphical tool about memory management that comes with UBUNTU installation is the System Monitor. One can reach it from System Settings System Monitor. As it can be seen, the only available information one can get is the amount and percentage of memory and swap currently being used. A graphical representation over time is created helping us to visualize the constant chagnes in the values. However, these information are poor for an advanced user and even less for the system root user. More detailed utilities are available through the command-line, most of which are only available for the root group.

PID : Process ID USER: User the process belongs to PR: Priority NI: Nice value VIRT: Virtual Image RES: Resident size SHR:Shared memory size %CPU: CPU usage %MEM: Memory usage TIME+:CPU Time COMMAND: command name/line

Additional commands and files containing memory information are shown below:

free -m /proc/meminfo /proc/interrupts uptime vmstat -s sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches" pagesize uname -r sudo swapoff -a sudo swapon -a

Limited memory information Full memory information System interrups served per CPU System uptime Virtual memory information Drops caches Page size currently used Kernel version (PAE included) Turning swap off Turning swap on

Ubuntu 11.04 - File Systems

File systems and UBUNTU 11.04 Ubuntu 11.04 support the Linux Logical Volume Manager, which allows us to increase the size of a partition online, while the system is running. In addition, Ubuntu 11.04 allows us to use different devices as mount points for several heavy/crucial directories. These are:

/boot, the directory containing the basic instructions to load the OS / (root) directory /home, the user(s) home directory, equivalent to My documents folder in MS Windows A full list of the supported file systems in UBUNTU 11.04 can be found using the cat

command in the virtual file system /proc as shown below. cat /proc/filesystems Table 1. Show some supported file systems in UBUNTU 11.04 according to Ubuntu

minix

is the file system used in the Minix operating system, the first to run under Linux. It has a number of shortcomings: a 64MB partition size limit, short filenames, a single timestamp, etc. It remains useful for floppies and RAM disks. an elaborate extension of the minix file system. It has been completely superseded by the second version of the extended file system (ext2) and has been removed from the kernel (in 2.1.21). the high performance disk file system used by Linux for fixed disks as well as removable media. The second extended file system was designed as an extension of the extended file system (ext). ext2 offers the best performance (in terms of speed and CPU usage) of the file systems supported under Linux. the file system used by DOS, Windows, and some OS/2 computers. msdos filenames can be no longer than 8 characters, followed by an optional period and 3 character extension. an extended DOS file system used by Linux. It adds capability for long filenames, UID/GID, POSIX permissions, and special files (devices, named pipes, etc.) under the DOS file system, without sacrificing compatibility with DOS. an extended DOS file system used by Microsoft Windows95 and Windows NT. VFAT adds the capability to use long filenames under the MSDOS file system. replaces Microsoft Window's FAT file systems (VFAT, FAT32). It has reliability, performance, and space-utilization enhancements plus features like ACLs, journaling, encryption, and so on. the network file system used to access disks located on remote computers. However, support for additional file systems can be added by adding additional modules

ext

ext2

msdos

umsdos

vfat

ntfs

nfs

to kernel. As it can be seen above, UBUNTU 11.04 supports several file systems. The result when using the cat /proc/filesystems command, will include several virtual file systems connected to no devices, that are used from Ubuntu kernel for administrative actions, such as memory caching. e.g.

ramfs, a very simple filesystem that exports Linux's disk caching mechanisms (the page cache and dentry cache) as a dynamically resizable RAM-based filesystem. rootfs, a special instance of ramfs (or tmpfs, if that's enabled), which is always present in 2.6 systems. You can't unmount rootfs for approximately the same reason you can't kill the init process; rather than having special code to check for and handle an empty list, it's smaller and simpler for the kernel to just make sure certain lists can't become empty. Adding to those above, the swap file system must be noted as well, even though it is a raw file system, used for memory caching. Swap is a very important parameter in Linux. The ones available to use in a physical drive during a clean installation, using the default partitioning tool are:

ext2 ext3 ext4 reiserfs xfs jfs btrfs

A very brief comparison of the File systems most commonly found in UBUNTU.

File System

Max File Size

Max Size 2 GB 8 TB 256 TB

Partition Journaling

Notes

Fat16 Fat32 NTFS

2 GB 4 GB 2 TB

No No Yes

Legacy Legacy For Windows Compatibility) NTFS-3g is installed by default in Ubuntu, allowing Read/Write support) Legacy Standard linux filesystem for many years. Best choice for super-standard installation. Modern iteration of ext3. Best choice for new installations where superstandard isn't necessary. No longer well-maintained.

ext2 ext3

2 TB 2 TB

32 TB 31 TB

No Yes

ext4

16 TB

1EB

Yes

reiserfs JFS XFS

8 TB 4 PB 8 EB

16 TB 32 PB 8 EB

Yes

Yes (metadata) Created by IBM - Not well maintained. Yes (metadata) Created by SGI. Best choice for a mix of stability and advanced journaling.

(https://help.ubuntu.com/community/LinuxFilesystemsExplained)

Tools for the ext2,ext3,ext4 and XFS file systems in Ubuntu. The available tools for file system information and modifications, are separated ion this paper in two parts. The one where the only the graphical tools are explained, and the second one where the commands in terminal are briefly explained. GUI tools

A fast overview of the available file systems can be found in the system monitor, in the file systems tab.

Further actions and information for the available storage devices can be found in the Disk Utility in the Control Center. Disk Utility provide us with an overview of the storage devices currently connected in the system. It also provides us with an easy graphical interface to:

get hardware information mount devices unmount devices check the filesystem in the device edit partitions delete partitions format volumes

Graphical tools for fie systems, as for any other administrative actions, are poorer comparing to the equivalents in terminal. They addressed to normal users and available actions are limited, e.g define block size, cannot be processed thought them. Terminal Commands They are addressed to advanced users and the can modify any available option for a file system. This is why they must be used as superuser, or by borrowing su privileges. Additional information, about parameters and syntax, are provided in the links below.

blkid, used to find the Universally Unique Identifier (UUID) for a device, which is prefered from Ubuntu over the deveice path, eg /dev/sda, since it can change during boot.

mkfs, used to create a file system fsck, used to check and repair file system fdisk df, used to report file system usage badblocks, used to check a device for bad blocks umount and mount, used to unmount and mount devices

Ubuntu 11.04 - Device management

Introduction UBUNTU has support for all types of hardware built into the kernel. (Ubuntu pocket guide and reference, p.19). To the extent possible, all device drivers appear as normal files. Devices are separated in three classes:

block devices

Here belong all the devices that allow random access to completely independent, ficed-size blocks of data, such as HDDs, floppy disks, CD-ROMs, flash memory. They are accessed randomly.

character devices

In this category belong most other devices, most of which are pointing devices such as keyboards, mice etc. They are accessed serially.

network devices

Here we find the network devices that users cannot communicate directly. Their transfer data by opening connection with the kernel's network subsystem. A fundamental difference in Ubuntu 11.04, based on their kernel version, 2.6.38 is the use of deadline I/O shceduler in order address the starvation issue. The deadline for read request is 0,5 second and for write requests is 5 seconds. It maintains a sorted queue, a read and a write queue. Four are the key folders for devices in UBUNTU, show in the table below:

Folder /dev /media /proc /etc

Description Virtual files representing hardware devices Contains folder where storage devices can be mounted Virtual folder containing file representing stats and settings System (global) configuration files UBUNTU does not have a Computer view like windows. Storage devices are mounted

in folders inside the /media folder. Moreover, for HDDs Master Boot record is important, because bootloader (GRUB 2.0) cannot be after the 1024th cyclinder, since it may cause system to halt at boot. Through kernel updates the drivers are updated and more are included. However, it is possible to compile drivers in kernel. However, it is addressed to advanced users only. The administration of the devices using graphical tools is limited, compared to the tools available in terminal. The utilities can be found in the System settings Hardware and System section. Additionally the gnome-device-manager package can be installed through the synaptic manager or through terminal, $sudo apt-get install gnome-device-manager

in order to graphically list all the available connected devices. Connected devices does not mean operational too. System administrator must count in terminal to list and compare values from the /proc folder and /dev for the problematic device, so as to verify the amount of interrupts, mapping, memory allocation addresses, pages etc to figure out where the problem is located.