DAEMON

Daemon stands for Disk and Execution Monitor. A daemon is a long-running

background process that answers requests for services.
Daemons are usually instantiated as processes. A process is an executing (i.e.,
running) instance of a program. Processes are managed by the kernel (i.e., the core
of the operating system), which assigns each a unique process identification number
(PID).
A daemon is a process that detaches itself from the terminal and runs, disconnected,
in the background, waiting for requests and responding to them. It can also be
defined as the background process that does not belong to a terminal session.
Examples: init, crond, sendmail, inetd, httpd, nfsd, sshd, named, and
lpd.
Difference between daemon process and background process:
A daemon process is a process that is specifically made to run in the
background. It contains code to put itself in the background as soon as it
starts.
A background process is any process that the user has chosen to run in the
background.
o For example if you have a very large sort job to run, you might choose
to run it in the background simply for convenience, to allow you to
continue reading mail or editing files or whatever else you want to
work on while the sort job is running.
ZOMBIE PROCESS:

When a process finishes execution, it will have an exit status to report to its parent
process. Because of this last little bit of information, the process will remain in the
operating systems process table as a zombie process, indicating that it is not to be
scheduled for further execution, but that it cannot be completely removed (and its
process ID cannot be reused) until it has been determined that the exit status is no
longer needed.
When a child exits, the parent process will receive a SIGCHLD signal to indicate that
one of its children has finished executing; the parent process will typically call the
wait() system call at this point. That call will provide the parent with the childs exit
status, and will cause the child to be reaped, or removed from the process table.

Orphan:

If a parent process dies, but its children have not, then those children are
"orphans process.

.PROFILE:

A profile file is a start-up file of an UNIX user, like the autoexec.bat file of
DOS. When a UNIX user tries to login to his account, the operating system
executes a lot of system files to set up the user account before returning the
prompt to the user.
To achieve this in UNIX, at the end of the login process, the operating system
executes a file at the user level, if present. This file is called profile file.

The specific settings which an unix user usually does is:

Setting of any environment variable
Setting of any alias.(Though it is always recommended to keep the aliases in
a separate file).
Setting of PATH variable or any other path variables
.profile - it say to spawn a new child shell and execute the script.
. ~/.profile - it means you are executing the script in present shell.
The command to run the profile file, say .profile is:
. $HOME/.profile

In case of sourcing the profile file in tcsh or csh, the command is:
source $HOME/.login
PATH:

PATH variable tells the places from where the Operating System(OS) will search for
whenever a command is given.
When the ls command is executed, the OS starts searching for an executable
named ls.
Where does it search? It searches in all the directories mentioned in the PATH
variable.
The moment it finds the executable, it executes the command the the output
is displayed.

SHELL:

A Unix shell is a command-line interpreter or shell that provides a user interface for
the Unix operating system and for Unix-like systems.
The UNIX shell is a program that serves as the interface between the user and the
UNIX operating system. It is not part of the kernel, but communicates directly with
the kernel. The shell translates the commands you type in to a format which the
computer can understand. It is essentially a command line interpreter.

Shebang(#!):
#! line states the interpreter to be used by the shell to interpret the script.
shebang line is used to tell shell which interpreter to use for your rest of the
script.
Running it as ./script.sh will make the kernel read the first line (the
shebang), and then invoke kornshell to interpret the script.
The dirname(1) and basename(1) commands
$ line="/dir1/dir2/gr3/file.ksh"
$ echo $line
/dir1/dir2/gr3/file.ksh
$ basename $line
file.ksh
$ dirname $line
/dir1/dir2/gr3
Maximum number of arguments that can be passed to shell:
The maxmium number of arguments can pass to a script is 9. If want to pass more
parameters, need to use the shift function.

#!/bin/sh
set -- "1" "2" "3" "4" "5"
echo $*
shift
echo $*
shift
echo $*
[blstst1a]:/home/bliss/kannan
$ ./"shift_ex.sh"
1 2 3 4 5
2 3 4 5
3 4 5

#!/bin/ksh
echo "Printing second args"
until [[ $# -eq 0 ]];do
echo $1
shift
done
./printer first second "third
forth"
Output:
Printing second args
first
second
third forth

Bourne shell:
var=value
export var
Korn shell:

export var=value

C-shell:

setenv var value

INODE:

All UNIX files have its description stored in a structure called 'inode'. The inode
contains info about the file-size, its location, time of last access, time of last
modification, permission and so on.

Directories are also represented as files and have an associated inode. In

addition to descriptions about the file, the inode contains pointers to the data
blocks of the file.
If the file is large, inode has indirect pointer to a block of pointers to
additional data blocks (this further aggregates for larger files). A block is
typically 8k.

Inode consists of the following fields:

File owner identifier
File type
File access permissions
File access times
Number of links
File size
Location of the file data

[blstst1a]:/home/bliss/kannan/debug_prg $ ls -li a.out

8334 -rwxrwxr-x
1 bliss
bliss 20480 Jul 11 09:19 a.out
PROCESS STATES IN

UNIX:

As a process executes it changes state according to its circumstances. Unix

processes have the following states:

Running : The process is either running or it is ready to run.

Waiting : The process is waiting for an event or for a resource.
Stopped : The process has been stopped, usually by receiving a signal.
Zombie : The process is dead but have not been removed from the process
table

SPECIAL VARIABLES:

Variable

Description

$0

The filename of the current script.

$n

These variables correspond to the arguments with which a script was

invoked. Here n is a positive decimal number corresponding to the
position of an argument (the first argument is $1, the second argument
is $2, and so on).

$#

The number of arguments supplied to a script.

$*

All the arguments are double quoted. If a script receives two

arguments, $* is equivalent to $1 $2.

$@

All the arguments are individually double quoted. If a script receives

two arguments, $@ is equivalent to $1 $2.

$?

The exit status of the last command executed.

$$

The process number of the current shell. For shell scripts, this is the
process ID under which they are executing.

$!

The process number of the last background command.

Difference between $* and $@:

$* and $@ both will act the same unless they are enclosed in double quotes, "".
Example:
#!/usr/bin/ksh
echo "Printing \$* "
for i in $*
do
echo i is: $i
done
echo "Printing \$@ "
for i in "$@"
do
echo i is: $i
done

Now, on running the above script:

$ ./cmd a b "c d" e
Printing $*
i is: a
i is: b
i is: c
i is: d
i is: e
Printing $@
i is: a
i is: b
i is: c d
i is: e

When we pass the command line argument in double quotes("c d"),

The $* does not consider them as a single entity, and splits them.
The $@ considers them as a single entity and hence the 3rd echo statement
shows "c d" together. This is the difference between $* and $@.

2>&1 and file >/dev/null 2>&1:

0 stdin

1 stdout

2 stderr

2>1 redirect srderr to stdout

However, it will actually be interpreted as "redirect stderr to a file named 1". &
indicates that what follows is a file descriptor and not a filename. So the construct
becomes: 2>&1.
Example:
cat food >file 2>&1
The shell sees >file first and redirects stdout to file. Next 2>&1 sends fd2 ( stderr ) to

the same place fd1 is going - that's to the file

Example 2:
cat food 2>&1 >file

//output: cat: can't open food

the shell sees 2>&1 first. That means "make the standard error (file descriptor 2) go to the
same place as the standard output (fd1) is going." There's no effect because both fd2 and
fd1 are already going to the terminal. Then >file redirects fd1 ( stdout ) to file . But
fd2 ( stderr ) is still going to the terminal.
file >/dev/null 2>&1
diff f1 f2 >/dev/null 2>&1
>/dev/null redirects standard out to /dev/null, i.e. throw the output away. 2>&1
redirects standard error to standard out; in this case, it means to throw all the
error output away as well

LINKS IN

UNIX

A links in UNIX are pointers pointing to a file or a directory. Creating links is a

kind of shortcuts to access a file.
The two different types of links in UNIX are:
1. Soft Links or Symbolic Links
2. Hard links
Soft link:
ln -s soft.txt soft1.txt

Soft Links can be created across file systems.

Soft link has a different inode number than the original file.
On deleting the original file, soft link cannot be accessed.
Soft link needs extra memory to store the original file name as its data.
Source file need not exist for soft link creation.
Can be created on a file or on a directory.
Access to the file is slower due to the overhead to access file.

cpath@ /home/cpath:ln -s /home/cpath/kk457h/Ph1_inbound/timer.sh

timer.sh
cpath@ /home/cpath:ls -l timer.sh
lrwxrwxrwx
1 cpath
cpath
39 Aug 26 01:00 timer.sh ->
/home/cpath/kk457h/Ph1_inbound/timer.sh
cpath@ /home/cpath:ls -li timer.sh
49194 lrwxrwxrwx
1 cpath
cpath
39 Aug 26 01:00
timer.sh -> /home/cpath/kk457h/Ph1_inbound/timer.sh
cpath@ /home/cpath:ls -li /home/cpath/kk457h/Ph1_inbound/timer.sh
22532 -rwxrwxrwx
1 cpath
cpath
309 Mar 16 15:49
/home/cpath/kk457h/Ph1_inbound/timer.sh

The file size of a soft linked file is the length of the filename of the original
file. In this case, the original file "file1" is of length 8.
In a soft linked file, the location where the file content is to be stored, the file
name of the original file gets stored, and hence the file size is so
Example:

_______
FIle1--------|

________

|______| welcome|

|_______|

|________|

9962464
_______
FIle3--------|

________

|______| File1

|_______|

|________|

9962471

Soft Link Representation (Files pointing to inodes, in turn pointing to data location)

File1 and File3 both have different inode numbers.

These different inode numbers point to different data locations.
File1 location points to the contents of the file. However, if you see the
content location of File3, it contains "File1" which is nothing but the original
file from which the soft link is formed.
Now, when File3 is accessed, through its inode metadata, it gets the name
"File1" and searches for "File1" and reaches to the content of "File1".
Hence, once the "File1" is deleted, "File3" becomes dangling meaning it points
to something which does not exist. And so, the soft link becomes inaccessible.

Hard Link:
ln hard.txt hard1.txt
[blstst1a]:/home/bliss/kannan/unix_work $ ls
total 4
lrwxrwxr-x
1 bliss
bliss
8
soft.txt
-rw-rw-r-2 bliss
bliss
143
-rw-rw-r-2 bliss
bliss
143
ln hard.txt hard2.txt
[blstst1a]:/home/bliss/kannan/unix_work $ ls
total 6
lrwxrwxr-x
1 bliss
bliss
8
soft.txt
-rw-rw-r-3 bliss
bliss
176
-rw-rw-r-3 bliss
bliss
176
-rw-rw-r-3 bliss
bliss
176

-lrt
Dec

4 22:47 soft2.txt ->

Dec
Dec

4 22:50 hard.txt
4 22:50 hard1.txt

-lrt
Dec

4 22:47 soft2.txt ->

Dec
Dec
Dec

4 22:51 hard.txt
4 22:51 hard1.txt
4 22:51 hard2.txt

[blstst1a]:/home/bliss/kannan/unix_work $ rm hard.txt
[blstst1a]:/home/bliss/kannan/unix_work $ ls -lrt
total 4
lrwxrwxr-x
1 bliss
bliss
8 Dec 4 22:47 soft2.txt ->
soft.txt

-rw-rw-r--rw-rw-r-

2 bliss
2 bliss

bliss
bliss

176 Dec
176 Dec

4 22:51 hard1.txt
4 22:51 hard2.txt

Hard links can be created only within the file system.

Hard links have the same inode number as the original file.
On deleting the original file, hard linked file can still be accessed.
Hard links do not need any extra data memory to save since it uses links
Source file should exist.
Can be created only on files, not on directories.
Access to the file is faster compared to soft link.

Example:

File1 and File2 are hardlinks.

They both have the same inode number(9962464) and hence they both point
to the same inode structure.
One of the parameters in the inode tells the location of the file contents, and
hence the contents are accessed by both the files.
Now we try to delete the file "File2".
When the file is deleted, only the link connecting of File2 to the inode
structure gets disconnected.
However, the inode and the file contents and the link of File1 are still in tact,
and hence the file is still accessible.
_______
FIle1--------\_____|inode

|______| welcome|

_____|_______|
File2--------/

________

|________|

9962464

RELATIVE PATH AND ABSOLUTE PATH

Absolute path : Exact path from root directory.

the absolute path is a path that contains the root directory and all other sub
directories required to get into the directory
Relative path : Relative to the current path.
A relative path describes the location of a file in relation to another file on the same
current Directory

FIND COMMAND
find path_list selection_criteria action

find . -name "rc.conf" -exec chmod o+r '{}' \

The argument '{}' inserts each found file into the chmod command line. The \;
argument indicates the exec command line has ended.

+ sign is used to search for greater than, - sign is used to search for less than and
without sign is used for exact.
1. find . -mtime 1 (find all the files modified exact 1 day)
2. find . -mtime -1 (find all the files modified less than 1 day)
3. find . -mtime +1 (find all the files modified more than 1 day)
4. find . -mtime +10 (find all the file modified more than 10 days old)
find . -ctime -2 (Display the files which are changed within 2 days)
find . -size +10000c -print -exec ls -l {} \;
Always use a c after the number, and specify the size in bytes, otherwise you
will get confuse because find -size list files based on size of disk block.
To find files using a range of file sizes, a minus or plus sign can be specified
before the number.
The minus sign means "less than," and the plus sign means "greater than."
find . -size +10000c -size -50000c print
lists all files that are greater than 10,000 bytes, but less than 50,000 bytes

To find the smallest file in the current directory and sub directories
find . -type f -exec ls -s {} \; | sort -n
find
find
find
find
find

.
.
.
.
.

-type
-type
-type
-type
-type

inum n
-perm nnn
-links n

s
d
d
f
f

| head -1

--- finding socket

--- finding directory
-name ".*" -- finding hidden directory
--- finding regular files
-name ".*"
--- finding hidden files
--- selects file having inode number n
--- selects files if octal permission match nnn
--- selects file if having n links

Difference between xargs and exec

find . -name H* -exec ls -l {} \; executes the command ls -l on each individual file.
find . -name H* | xargs ls -l constructs an argument list from the output of the find
commend and passes it to ls.
consider if the ouput of the find command produced:
H1
H2
H3
the first command would execute
ls -l H1
ls -l H2
ls -l H3

but the second would execute

ls -l H1 H2 H3

UMASK:

Whenever create files and directories, the default permissions that are assigned to
them depend on the systems default setting.
The default permission
rw_rw_rw_ (octal 666) for regular files
rwxrwxrwx (octal 777) for directories
These are not the permission, when create a file or direcotry. The default is
transformed by subtracting the user mask from it to remove the one or more
permission.
umask equal to 022
666 022 = 644 (rw_r__r__ for regular files)
777 022 = 755 (rwx_r_xr_x for directory)

TOUCH:

Sometimes need to set the modification and access time to predefined values.
Syntax:
touch option expression filename
Whenever touch is used without option and expression, both time are set to the
current time.
The expression consist of an eight digit number format MMDDhhmm
Option: -m mean modification time
-a mean access time
touch m 12071030 file.txt (Dec 07 10:30)
touch a 12061030 file1.txt (Dec 06 10:30)

TEE:

TYPE:

Tee command is used to store and view (both at the same time) the output of
any other command.
Tee command writes to the STDOUT, and to a file at a time. By default tee
command overwrites the file.
o $ ls | tee file
Can instruct tee command to append to the file using the option a as shown
below.
o $ ls | tee a file

type is a Unix command that describes how its arguments would be interpreted if
used as command names.
type will display the command name's path. Possible command types are:
o shell built-in
o function

o alias
o hashed command
o keyword
The command returns a non-zero exit status if command names cannot be found.
Example:
$ type test
test is a shell builtin
$ type cp
cp is /bin/cp
$ type unknown
-bash: type: unknown: not found
$ type type
type is a shell builtin
$ type -a gzip
gzip is /opt/local/bin/gzip
gzip is /usr/bin/gzip

USER ENVIRONMENT
ENV:

env - run a program in a modified environment

env is used to either print a list of environment variables or run another utility in an
altered environment without having to modify the currently existing environment.
Using env, variables may be added or removed, and existing variables may be
changed by assigning new values to them.

To clear the environment (creating a new environment without any

existing environment variables) for a new shell:
o env -i /bin/sh

remove variable from the environment

o -u, --unset=NAME

FINGER:

finger is a program you can use to find information about computer users. It usually
lists the login name, the full name, home directory, shell information about a
particular user as shown below.

finger username@node.domain

ID:

prints the user or group identifier of the account by which the program is executed
The root account has a UID of 0
id -un # Where `-u` refers to `--user` and `-n` refers to `--name`

SU:

The su command, also referred to as substitute user, super user, or switch user,
allows a computer operator to change the current user account associated with the
running virtual console.

When run from the command line, su asks for the target user's password, and if
authenticated, grants the operator access to that account and the files and
directories that account is permitted to access.
It should generally be used with a hyphen by administrators (su -, which is identical
to su - root), which can be used to start a login shell. This way users can assume
the user environment of the target user:

FOLD:

Fold is a Unix command used for making a file with long lines more readable
on a limited width terminal.
Most Unix terminals have a default screen width of 80, and therefore reading
files with long lines could get annoying.
The fold command puts a line feed every X characters if it does not reach a
new line before that point. If the -w argument is set, the fold command
allows the user to set the maximum length of a line.

As a usage example, to fold a file named file.txt to have a maximum of 30

characters per line, one could run the following command:
fold -w 30 file.txt

JOIN;

Join command is used to combine two files based on a matching fields in the files.
join [options] file1 file2
Option:
-1 field number : Join on the specified field number in the first file
-2 field number : Join on the specified field number in the second file
-j field number : Equivalent to -1 fieldnumber and -2 fieldnumber
-o list : displays only the specified fields from both the files
-t char : input and output field delimiter
-a filenumber : Prints non matched lines in a file
-i : ignore case while joining
The basic usage of join command is to join two files on the first field. By default the
join command matches the files on the first fields when we do not specify the field
numbers explicitly.

> cat emp.txt

10 mark
10 steve
20 scott
30 chris
> cat dept.txt
10 hr
20 finance
30 db
> cat emp.txt
mark 10 1
steve 10 1
scott 20 2

> join emp.txt dept.txt

10 mark hr
10 steve hr
20 scott finance
30 chris db

> join -1 2 -2 1 emp.txt dept.txt

10 mark 1 hr 1
10 steve 1 hr 1
20 scott 2 finance 2

chris 30 3

30 chris 3 db 3

> cat dept.txt

10 hr 1
20 finance 2
30 db 3
> join -j 3 emp.txt dept.txt
1 mark 10 10 hr
1 steve 10 10 hr
2 scott 20 20 finance
3 chris 30 30 db

Here -1 2 specifies the second field from

the first file (emp.txt) and -2 1 specifies
the first field from the second file
(dept.txt)

two files can also be joined on the third

filed. As the both the files have the
matching join field, you can use the j
option in the join command.

> cat emp.txt

mark:10
steve:10
scott:20
chris:30
> cat dept.txt
10:hr
20:finance
30:db

> join -o 1.1 2.2 -1 2 -2 1

emp.txt dept.txt
mark hr
steve hr
scott finance
chris db
Here 1.1 means in the first file select the
first field. Similarly, 2.2 means in the
second file select the second field

> join -t: -1 2 -2 1 emp.txt

dept.txt
10:mark:hr
10:steve:hr
20:scott:finance
30:chris:db
The -t option is used to specify the
delimiter. The join command for joining the
files is

PASTE:

The paste command merges the lines from multiple files. The paste command
sequentially writes the corresponding lines from each file separated by a TAB
delimiter on the unix terminal.
paste [options] files-list
Option:
-d : Specify of a list of delimiters.
-s : Paste one file at a time instead of in parallel.
> paste file1 file2
> cat file1
Unix
Dedicated server
Unix
Linux
Virtual server
Linux
Windows
Windows
> cat file2
Dedicated server
Virtual server
> cat file3
Hosting
Machine
Operating system
> paste -d"|" file1 file2

> paste file2 file1

Dedicated server Unix
Virtual server
Linux
Windows
By default, the paste command merges the
files in parallel. The paste command writes
corresponding lines from the files as a tab
delimited on the terminal.
> paste -s file1 file2

Unix|Dedicated server
Linux|Virtual server
Windows|

Unix
Linux
Windows
Dedicated server
Virtual server

Paste command uses the tab

delimiter by default for merging the
files. You can change the delimiter to
any other character by using the d
option.

You can merge the files in sequentially using

the -s option. The paste command reads
each file in sequentially. It reads all the lines
from a single file and merges all these lines
into a single line.

SPLIT:

The split command splits the file into multiple files with 1000 lines into each output
file by default.
Since the input file does not contain 1000 lines, all the contents are put into only one
output file "xaa". By default, the output files generated contains the prefix "x", and
the suffix as "aa", "ab", "ac" and so on.

Split the file into 20 MB files as shown below with -b option.

o split -b 20M logdata

Split the file into 50 MB pieces with the given prefix

o split --bytes=50M logdata mydatafile

Split the file and name it with numbers

o

Use -d option to name the files with number suffixes as 00,

01, 02 .. and so on, instead of aa, ab, ac.
$ split -d testfile
$ ls
testfile x00 x01 x02

FILTERS:

CMP COMMAND:
Two files are compared byte by byte, and the location of the first mismatch is echoed
to the screen.
Cmp, when invoked without option doesnt bother about subsequently mismatches.

CUT COMMAND:
Head and tail command used to slice a file horizontally. The slice a file vertically with
the cut command. Cut identifies both column and fields.
-c ---- output only these characters

> cat file.txt

unix or linux os
is unix good os
is linux good os

-f ---- cutting fields

cut command to print characters by

position:
cut -c4 file.txt
x
u
l
To print characters by range
cut -c4-7 file.txt
x or
unix
linu
To print the characters from tenth
position to the end, specify only the start
position and omit the end position.
cut -c10- file.txt
inux os
ood os
good os
The -d option in cut command can be
used to specify the delimiter and -f
option is used to specify the field
position.
cut -d' ' -f2 file.txt
or
unix
linux

print more than one character at a time

by specifying the character positions in a
comma separated list as
cut -c4,6 file.txt
xo
ui
ln
To print the first six characters in a line,
omit the start position and specify only
the end position.
cut -c-6 file.txt
unix o
is uni
is lin
If you omit the start and end positions,
then the cut command prints the entire
line.
cut -c- file.txt

print a range of fields by specifying the

start and end position.
cut -d' ' -f1-3 file.txt

To print the first three fields, you can

ignore the start position and specify only
the end position.
cut -d' ' -f-3 file.txt

To print the fields from second fields to

last field, you can omit the last field
position.
cut -d' ' -f2- file.txt

SORT COMMAND:

sort [options] filename

Option
-t char
-u
-n
-r
-c
-k
-M
-b
-o
file1

Significance
Uses delimiter char to identify field
Remove duplicate lines
Sort numerically
Reverse sort order
Check if the file is sorted
Sorts file based on the data in the specified field
positions.
Sorts based on months. Considers only first 3 letters as
month
Ignores leading spaces in each line
Places output in file file1

-m list

Merges sorted files in list

> cat order.txt

Unix distributed 05 server
Linux virtual 3 server
Unix distributed 05 server
Distributed processing 6 system

> sort order.txt

Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server

-k Option:-

You can specify the field positions using the -k option of sort command.
To sort based on the data in the second field, run the below command:
> sort -k2 order.txt
Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system
Linux virtual 3 server
-u Option:Only unique values in the output using the - u option of the sort command.
> sort -u order.txt
Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server
> sort -t'|' -nrk2 delim_sort.txt
Declast|12
Mayday|4
Janmon|1

Sort the data in the monthwise using the -M option of the sort command. This is shown
below:
> sort -M delim_sort.txt
Janmon|1
Mayday|4
Declast|12

UNIQ COMMAND:

The uniq command can eliminate or count duplicate lines in a presorted file. It reads in
lines and compares the previous line to the current line. Depending on the options
specified on the command line it may display only unique lines or one occurrence of
repeated lines or both types of lines.
uniq [ -options [ input [output] ]

Options

-u Print only lines which are not repeated (unique) in the original file
-d Don't output lines that are not repeated in the input.
-c Generate an output report in default style except that each line is
preceded by a count of the number of times it occurred. If this option is

specified, the -u and -d options are ignored if either or both are also
present.
-i Ignore case differences when comparing lines
-f Ignore a number of fields in a line
-s Skips a number of characters in a line
-w Specifies the number of characters to compare in lines, after any
characters and fields have been skipped
--help Displays a help message
--version Displays version number on stdout and exits.

uniq file Simply fetches one copy of each line and write it to the standard output.
davel
davel
davel
jeffy
jones
jeffy
mark
mark
mark
chuck
bonnie
chuck

You could run uniq on it like this:

% uniq foo
davel
jeffy
jones
jeffy
mark
chuck
bonnie
chuck

% sort foo | uniq

jones
bonnie
davel
chuck
jeffy
mark

That gives you a truly unique list. However, it's also a useless use of uniq since
sort(1) has an argument, -u to do this very common operation:
% sort -u foo
jones
bonnie
davel
chuck
jeffy
mark
d tells uniq to eliminate all lines with

only a single occurrence (delete unique

lines), and print just one copy of repeated
lines:
% sort foo | uniq -d
davel
chuck

-u tells uniq to eliminate all duplicated

lines and show only those which appear
once (only the unique lines):
% sort foo | uniq -u
jones
bonnie

jeffy

mark
-c tells uniq to count the occurrences of
each line:
% sort foo | uniq -c
1 jones
1 bonnie
3 davel
2 chuck
2 jeffy

3 mark

I often pipe the output of "uniq -c" to

"sort -n" (sort in numeric order) to get
the list in order of frequency:
% sort foo | uniq -c | sort -n
1 jones
1 bonnie
2 chuck
2 jeffy
3 davel
3 mark

LINE NUMBERING: nl

The nl command has elaborate schemes for numbering lines.

nl use the tab as the default delimiters.
-w -- To specify the width of the number format.
-s --- delimiters

/home/bliss $ nl design.txt
1 davel
2 davel
3 davel
4 jeffy
5 jones
6 jeffy
7 mark
8 mark
9 mark
10 chuck
11 bonnie
12 chuck

/home/bliss $ cat design.txt

davel
davel
davel
jeffy
jones
jeffy
mark
mark
mark
chuck
bonnie
chuck

[blstst1a]:/home $ nl -w1 -s:

design.txt
1:davel
2:davel
3:davel
4:jeffy
5:jones
6:jeffy
7:mark
8:mark
9:mark
0:chuck
1:bonnie
2:chuck

/unix_work $ nl -w2 -s: design.txt

1:davel
2:davel
3:davel
4:jeffy
5:jones
6:jeffy
7:mark
8:mark
9:mark
10:chuck
11:bonnie
12:chuck

[blstst1a]:/home/bliss/kannan/unix_work $ nl -w2 -s'|' design.txt

[blstst1a]:/home/bliss/kannan/unix_work $ nl -w2 -s"|" design.txt
1|davel
2|davel
3|davel
4|jeffy

5|jones
6|jeffy
7|mark
8|mark
9|mark
10|chuck
11|bonnie
12|chuck
Both command have same output.

FILTERS USING REGULAR EXPRESSION:

GREP:

The name grep is a combination of editor command characters.

It is from the editor command :g/RE/p, which translates to global Regular
Expression print.
In fgrep the f stands for fast.
egrep Achieves by using multiple e option
o egrep bliss|ofr emp.txt

grep [options] pattern [files]

Option
-c
-l
-n
-v
-i
-h
-w
-o
-b
-s
-r
-q

PATTERN
*
.
+
?

Significance
Display count of number of occurrences
Display list of file name only
Display line numbers along with lines
Doesnt display lines matching expression
Ignore case when matching
Omits filenames when handling multiple files
Match complete word only
To show out only the matched string of the pattern
Display the block number at the beginning of each
line
Silent mode
search recursively i.e. read all files under each
directory for a string
Ouput will not display in terminal
[blstst1a]:/home/bliss/ $ grep -q loop *.sh
[blstst1a]:/home/bliss/ $ grep loop *.sh
sample_prt.sh:loop
sample_prt.sh:end loop;
MATCHES

Zero or more occurrences of previous character

A single character
Matches one or more occurrences of the previous
character
Matches Zero or more occurrences of the

^ (caret)
$
[abc]
[a-c]
[^abc]

previous character
For Matching at the beginning
For matching at the end
A single character a,b or c
A character between a to c
A single character which is not a,b or c

SED: STREAM EDITOR

The sed is an instruction. An instruction combines an address for selecting lines with
an action to be taken.

sed options address action files

The address and action are enclosed within single quotes.

Command
i,a,c
d
1,4d
r foo
w bar
3,$p
$!p
/begin/,/end/p
q
s/s1/s2/
-e
[[:space:]]
^ *$

Significance
Insert, append and change text
Delete lines
Delete lines 1 to 4
Places contents of file foo after line
Write address line to file bar
Print line 3 to end
Print all the line except last line
Print lines enclosed between begin and end
Quit after reading up to addressed line
Replace first occurrence of string s1 in all
lines with sting s2
e option to do this multiple substitutions
simply a special keyword that tells sed to match either a TAB or a
space.
'*' indicates 0 or more occurrences of the
previous character. '^ *$' indicates a line
containing zero or more spaces.

dollar sign ($) denotes last line of input file

show only line 3

o sed -n -e '3 p' input-file
show only last line
o sed -n -e '$ p' input-file
substitute endif with fi on line 10
o sed -e '10 s/endif/fi/' input-file

Note:In the above example each regular expression inside the parenthesis would be
back referenced by \1, \2 and so on. Here I used \ to give line break you should
remove this before running this command.
[blstst1a]:/home/bliss/kannan/unix_work $ cat sed1.txt
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
/unix_work $
This is line
This is line
This is line
This is line
This is line
This is line
This is line
This is line
/unix_work $
This is line
This is line
This is line
This is line

sed '1,3p' sed1.txt

1
1
2
2
3
3
4
5
sed -n '$!p' sed1.txt
1
2
3
4

Selecting multiple selection

/unix_work $ sed -n -e '1,2p' -e
'5,7p' -e '$p' sed1.txt
This is line 1
This is line 2
This is line 5
This is line 6
This is line 7
This is line 9

/unix_work $ sed '/^This/d'

sed1.txt
line number 10

/unix_work $
This is line
This is line
This is line

sed -n '1,3p' sed1.txt

1
2
3

-n - To overcome the problem of

printing duplicate lines.
/unix_work $ sed -n '$p' sed1.txt
This is line 5
Deleting lines (d):
/unix_work $ cat sed1.txt
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
This is line 6
This is line 7
This is line 8
This is line 9
line number 10
/unix_work $ sed -n '/.\{14,\}/p'
sed1.txt
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
This is line 6
This is line 7
This is line 8
This is line 9

line number 10
Sed print all the lines on the std output in addition to the lines affected by the action.
So the addressed lines (the first two) are printed twice.
[blstst1a]:/home/bliss/kannan/unix_work $ sed -n '/.\{15,\}/p' sed1.txt
\{m,\} --- m mean length of the character line

Delete the first line AND the last line of a file, i.e, the header and trailer line of a file.
sed '1d;$d' file
With output redirection we can do the same thing.

sed s/baby/dady/ < tem.txt > abc.txt.

How about writing only changes to another file for future reference?
With w option, we will get only changes to the new file

sed s/baby/dady/w abc.txt tem.txt

How about deleting 4 line and update the original file?

sed i 4d tem.txt
Execute multiple commands with e option
sed e s/surendra/bca/ e s/mouni/mca/ e s/baby/bba/ tem.txt
How about reducing it more by using ;(Continuation operator) for the same
question?

sed s/Surendra/bca/;s/mouni/mca/;s/baby/bba/ tem.txt

$ cat file
Cygwin
Unix
Linux
Solaris
AIX
Delete the lines NOT containing the
pattern 'Unix':

$ sed '/Unix/!d' file

Unix

Delete the lines containing the pattern

'Unix' OR 'Linux':
$ sed '/Unix\|Linux/d' file
Cygwin
Solaris
AIX

Delete all lines which are entirely in

capital letters:

Note: The OR condition is specified

using the | operator. In order not to get
the pipe(|) interpreted as a literal, it is
escaped using a backslash.
Delete all lines which are entirely in
capital letters:

$ sed '/^[A-Z]*$/d' file

Cygwin
Unix

$ sed '/^[A-Z]*$/d' file

Cygwin
Unix

Linux
Solaris
Delete the last line ONLY if it
contains either the pattern 'AIX' or
'HPUX':

$ sed '${/AIX\|HPUX/d;}' file

Cygwin
Unix
Linux
Solaris

Linux
Solaris
Delete the line containing the pattern
'Unix' and also the next line:
$ sed '/Unix/{N;d;}' file
Cygwin
Solaris
AIX
Note:
N command reads the next line in the
pattern space. d deletes the entire
pattern space which contains the current
and the next line.

Delete only the next line containing the pattern 'Unix', not the very line:
$ sed '/Unix/{N;s/\n.*//;}' file
Cygwin
Unix
Solaris
AIX

Using the substitution command s, we delete from the newline character till the end,
which effective deletes the next line after the line containing the pattern Unix.
Sample script for sending the report to mail id:
#!/bin/ksh
. /bliss/ofc/script/bliss_env
export FILE_NAME=/home/bliss/kannan/unix_work/report.txt
export OUT_FILE=/home/bliss/kannan/unix_work/output.txt
export REPORT_FILE=/home/bliss/kannan/unix_work/final_report.txt
export MAILLIST="kannkand@in.ibm.com"
rm -f $FILE_NAME
rm -f $OUT_FILE
$ORACLE_HOME/bin/sqlplus -s $ORACLE_LOGIN <<!>>$OUT_FILE
set echo off
set head off
set feedback off
spool $FILE_NAME;
select
CUSTOMER_ORDER_ID||'|'||COE_ID||'|'||LINE_TYPE_CD||'|'||TELEPHONE_NUM
from telephone_number where customer_order_id=198425;
spool off;
exit;
!
sed '1i\
CUSTOMER_ORDER_ID|COE_ID|LINE_TYPE_CD|TELEPHONE_NUM' $FILE_NAME
>$REPORT_FILE
rm -f $FILE_NAME

if [ -s $REPORT_FILE ]; then
echo $REPORT_FILE "File sending to my id"
uuencode $REPORT_FILE $REPORT_FILE |mailx -s "Test Report" $MAILLIST
fi
exit 0
AWK:

awk is one of the most powerful utilities used in the unix world. Whenever it comes
to text parsing, sed and awk do some unbelievable things.
The syntax of awk is:
awk option 'selection criteria {action}' file
The selection criteria (a form of addressing) filter input and selects lines for the
action component to act on.
where the pattern indicates the pattern or the condition on which the action is to be
executed for every line matching the pattern.
In case of a pattern not being present, the action will be executed for every line
of the file.
In case of the action part not being present, the default action of printing the line
will be done.
Built-In Variables:
Variable
Function
NR
Cumulative number of lines read
FS
Input field separator
OFS
Output field separator
NF
Number of fields in current line
FILENAME
Current input file
ARGC
Number of arguments in command line
ARGV
List of arguments
F
Specify the delimiter
Built-In Functions:
Function
Significance
int(x)
Returns integer value of x
sqrt(x)
Returns square root of x
length
Return length of the complete line
substr(stg,m,n)
Returns portion of string of length n, starting from postion m in
string stg.
index(s1,s2)
Returns position of string s2 in string s1
split(stg,arr,ch)
Splits string stg into array arr using ch as delimiter;
system(cmd
Runs UNIX command cmd, and return its exit status.

[blstst1a]:/home/bliss/kannan $ cat >file1

Name Domain
Deepak Banking
Neha Telecom
Vijay Finance

Guru Migration
To omit the header record and get
only the names printed:

$ awk 'NR!=1{print $1}'

file1
Deepak
Neha
Vijay
Guru
FS" - field separator. In place of the
command line option "-F', we can also
use the "FS"

$ awk '{print $1,$3}'

FS="," file1
Name Expertise
Deepak MQ Series
Neha Power Builder
Vijay CRM Expert
Guru Unix

Using field delimiters

$ awk -F","
file1
Name
Deepak
Neha
Vijay
Guru

'{print $1}'

[blstst1a]:/home/bliss/kannan/
$ awk -F"," '{print $1,$3}'
OFS="|" file1
Name|Expertise
Deepak|MQ Series
Neha|Power Builder
Vijay|CRM Expert
Guru|Unix

Pass a variable to awk which contains the double quote. Print the quote,
line, quote.

[blstst1a]:/home/bliss/kannan/unix_work/awk_prg $ awk -v
q="'" -F"," '{print q $1 q}' file1
'Name'
'Deepak'
'Neha'
'Vijay'
'Guru'
To double quote the contents, pass the variable within single quotes

[blstst1a]:/home/bliss/kannan/unix_work/awk_prg $ awk -v
q='"' -F"," '{print q $1 q}' file1
"Name"
"Deepak"
"Neha"
"Vijay"
"Guru"
UUENCODE:

(mail sending)

uuencode [ -m ] [ SourceFile ] OutputFile

The uuencode command converts a binary file to ASCII data. The uudecode command
converts ASCII data created by the uuencode command back into its original binary
form.
The uuencode command takes the named SourceFile (default standard input) and
produces an encoded version on the standard output.
The encoding uses only printable ASCII characters, and includes the mode of the file and
the OutputFile filename used for recreation of the binary image on the remote system.
- Encode the output using the MIME Base64 algorithm. If -m is not specified, the old
m uuencode algorithm will be used
Example:
uuencode $REPORT_FILE $REPORT_FILE |mailx -s "Test Report" $MAILLIST

PROCESS:
Option
-f
-e
-u usr
-a
-l
-t term

Significance
Fill listing showing the PPID of each process
All process including user and system processes
Processes of user usr only
Processes of all users excluding processes not
associated with terminal
A long listing showing memory related information
Processes running on terminal term (tty03)

The & is the shells operator used to run a process in the background.

$ sort o emp.lst &

550 // the jobs ID

NOHUP:

Log out Safety

The hohup permits execution of the process even after the user had logged out.

$ nohup sort emp.lst &

586
Sending output to nohup.out
NICE: Job Execution with Low Priority
Processes are usually executed with equal priority. This is not always desirable since
high-priority jobs must be completed earliest.
To run a job with a low priority.

nice wc l <file> &

nice is a built-in command in the C Shell. Where it has default value of 4. nice
values are system-dependent and typically range from 1 to 19.

Nice -5 wc l <file> &

// Nice value increased by 5 units

SIGNALS:

The following are some of the more common signals you might encounter and want
to use in your programs:
Signal Name

Signal
Number

SIGHUP

Hang up detected on controlling terminal or death of

controlling process

SIGINT

Issued if the user sends an interrupt signal (Ctrl + C).

SIGQUIT

Issued if the user sends a quit signal (Ctrl + D).

SIGFPE

Issued if an illegal mathematical operation is attempted

SIGKILL

If a process gets this signal it must quit immediately and

will not perform any clean-up operations

SIGALRM

14

Alarm Clock signal (used for timers)

SIGTERM

15

Software termination signal (sent by kill by default).

Description

Killing the last background job:

The system variables $! sores the PID of the last background job.
Example:
$ sort o emp.lst &
345
$ kill $!
// kill the sort command
kill -9 %1
// kill the first background job with the signal number 9.

JOB CONTROL:

Relegate a job to the background (bg )

Bring it back to the foreground (fg)
List the active jobs (jobs)
Suspend a foreground job ( [Ctrl-z] )
Kill a job ( kill )

AT AND BATCH: Execute Later

The at and batch command make such scheduling possible.

at takes as its argument the time the job is to be executed.

$ at 14:10
<file name>
[Ctrl-d]

CRONTAB:

The crontab command is used to schedule jobs to be run in the future, usually on
some regular schedule (such as every week). The command is run with one of three
command line arguments:
crontab -l

View crontab file, if any

crontab -r

Remove crontab file, if any

crontab -e

Edit (or create) user's crontab file (starts the editor automatically)

crontab file

Replace existing crontab file (if any) with file

Cron command syntax

Field

Function Minute Hour Day of

month

Month of
year

Day of week

Command

Ranges 0-59

1-12

0-6
Begins with Sunday (0) and
ends Saturday (6).

0-23 1-31

1. Scheduling a Job For a Specific Time

a. 30 08 10 06 * /home/ramesh/full-backup
2. Schedule a Job For More Than One Instance (e.g. Twice a
Day)
a. 00 11,16 * * * /home/ramesh/bin/incremental-backup
3. 30 8 1 * * <command> #Runs at 8:30 am on the first day of
every month
4. 00 0-23/2 02 07 * <command> #Runs every other hour on the
2nd of July
5. 0 */2 * * * /home/username/test.pl run every two hours, at
midnight, 2am, 4am, 6am, 8am, and so on

Entry
@yearly (or
@annually)
@monthly
@weekly
@daily
@hourly
@reboot

Description
Run once a year at midnight in the morning of
January 1
Run once a month at midnight in the morning of
the first of the month
Run once a week at midnight in the morning of
Sunday
Run once a day at midnight
Run once an hour at the beginning of the hour
Run at startup

cron permissions
The following two files play an important role:

Equivalent
To
0 0 1 1 *
0 0 1 * *
0 0 * * 0
0 0 * * *
0 * * * *
@reboot

/etc/cron.allow - If this file exists, it must contain your username for you to use
cron jobs.
/etc/cron.deny - If the cron.allow file does not exist but the /etc/cron.deny file
does exist then, to use cron jobs, you must not be listed in the /etc/cron.deny file.

Default Actions:
Every signal has a default action associated with it. The default action for a signal is
the action that a script or program performs when it receives a signal.
Some of the possible default actions are:
Terminate the process.
Ignore the signal.
Dump core. This creates a file called core containing the memory image of the
process when it received the signal.
Stop the process.
Continue a stopped process
Sending Signals:
There are several methods of delivering signals to a program or script. One of the
most common is for a user to type CONTROL-C or the INTERRUPT key while a script
is executing.
When you press the Ctrl+C key a SIGINT is sent to the script and as per defined
default action script terminates.
The other common method for delivering signals is to use the kill command whose

syntax is as follows:
$kill -signal pid
Here signal is either the number or name of the signal to deliver and pid is the
process ID that the signal should be sent to.

For Example:
$ kill -1 1001
Sends the HUP or hang-up signal to the program that is running with process ID
1001.
To send a kill signal to the same process use the folloing command:

$ kill -9 1001
This would kill the process running with process ID 1001.
Trapping Signals:
When you press the Ctrl+C or Break key at your terminal during execution of a shell
program, normally that program is immediately terminated, and your command
prompt returned.
This may not always be desirable. For instance, you may end up leaving a bunch of
temporary files that won't get cleaned up.

Trapping these signals is quite easy, and the trap command

has the following syntax:
$ trap commands signals
Here command can be any valid Unix command, or even a user-defined function,
and signal can be a list of any number of signals you want to trap.

There are three common uses for trap in shell scripts:

1. Clean up temporary files

2. Ignore signals
Cleaning Up Temporary Files:
To remove some files and then exit if someone tries to abort the program from the
terminal:

$ trap "rm -f $WORKDIR/work1$$ $WORKDIR/dataout$$; exit" 2

From the point in the shell program that this trap is executed, the two files work1$$
and dataout$$ will be automatically removed if signal number 2 is received by the
program.
So if the user interrupts execution of the program after this trap is executed, you can
be assured that these two files will be cleaned up. The exit command that follows
the rm is necessary because without it execution would continue in the program at
the point that it left off when the signal was received.

Signal number 1 is generated for hangup:

Either someone intentionally hangs up the line or the line gets accidentally
disconnected.
You can modify the preceding trap to also remove the two specified files in this case
by adding signal number 1 to the list of signals:
$ trap "rm $WORKDIR/work1$$ $WORKDIR/dataout$$; exit" 1 2
Now these files will be removed if the line gets hung up or if the Ctrl+C key gets
pressed.
The commands specified to trap must be enclosed in quotes if they contain more
than one command. Also note that the shell scans the command line at the time that
the trap command gets executed and also again when one of the listed signals is
received.
So in the preceding example, the value of WORKDIR and $$ will be substituted at the
time that the trap command is executed. If you wanted this substitution to occur at
the time that either signal 1 or 2 was received you can put the commands inside
single quotes:
$ trap 'rm $WORKDIR/work1$$ $WORKDIR/dataout$$; exit' 1 2
Ignoring Signals:
If the command listed for trap is null, the specified signal will be ignored when
received. For example, the command:
$ trap '' 2
Specifies that the interrupt signal is to be ignored. You might want to ignore certain
signals when performing some operation that you don't want interrupted. You can
specify multiple signals to be ignored as follows:

$ trap '' 1 2 3 15
Note that the first argument must be specified for a signal to be ignored and is not
equivalent to writing the following, which has a separate meaning of its own:
$ trap 2
If you ignore a signal, all subshells also ignore that signal. However, if you specify an
action to be taken on receipt of a signal, all subshells will still take the default action
on receipt of that signal.
Resetting Traps:
After you've changed the default action to be taken on receipt of a signal, you can
change it back again with trap if you simply omit the first argument; so
$ trap 1 2
resets the action to be taken on receipt of signals 1 or 2 back to the default.
#!/bin/sh
# trap1a
trap 'my_exit; exit' SIGINT SIGQUIT
count=0

while :
do
sleep 1
count=$(expr $count + 1)
echo $count
done

my_exit()
{
echo "you hit Ctrl-C/Ctrl-\, now
exiting.."
# cleanp commands here if any
}
output:
[blstst1a]:/home/bliss/kannan/unix_work $ ./trap2.sh
1
2
3
4
5
6
7
you hit Ctrl-C/Ctrl-\, now exiting..

SHELL SCRIPT:
Any variable can become an environment variable. First it must be defined as usual;
then it must be exported with the command:
export varnames

File Attribute Checking

Operator
-a file
-d file
-f file
-r file

Significance
File exists
File is a directory
File is a regular file
Have read permission on the file

-f file
-w file
-x file

File exists and is not empty

File have write permission
File have execute permission

If statement:
if [ $# -le 2 ];
then
echo "need 2 argument"
exit 1
fi

Shift:
#!/bin/sh
while [ $# -gt 1 ]; do
echo $1
shift
done

For:
for lst in `ls -l`; do
echo $lst

Case:
case:
#!/bin/sh
set -x

done

read input
case $input in
1) `ls -l`
break ;;
2) `ls`
break ;;
*) `ls -lrt`
exit ;;

esac
DELIM means either a newline or a ; delimiter.
if

if EXPR DELIM then STMT(S) DELIM fi

if else if EXPR DELIM then STMT(S) DELIM elif EXPR ; then STMT(S) DELIM fi
for

for VAR in LIST DELIM do STMT(S) DELIM done

while while EXPR DELIM do STMT(S) DELIM done

case

case VALUE in [[ PATTERN [ | PATTERNS ] ) STMTS ;; ] esac

Conditional Test:
String operations
string1 =
string2

True if the strings are equal.

string1 !=
string2

True if the strings are not equal.

-z string

True if the length of string is zero.

string

True if the length of string is non-zero.

-n string

True if the length of string is non-zero.

string1 ==
string2

(Bash only) True if the strings are equal.

str = regex

(Bash in [[ ]] only) True iff regex matches str. BASH_REMATCH[0] =

entire match, BASH_REMATCH[i] = i-th paren submatch.

-o optname

True if shell option optname is enabled. See the list of options under
the description of the -o option to the set builtin below.
Numeric operations

arg1 OP
arg2

OP is one of -eq, -ne, -lt, -le, -gt, or -ge. These arithmetic binary
operators return true if arg1 is equal to, not equal to, less than, less than
or equal to, greater than, or greater than or equal to arg2, respectively.
Arg1 and arg2 may be positive or negative integers.

string1 <
True if string1 sorts before string2 lexicographi- cally in the current locale.
string2
string1 >
True if string1 sorts after string2 lexicographi- cally in the current locale.
string2

File operations
-e file

True if file exists.

-d file

True if file exists and is a directory.

-f file

True if file exists and is a regular file.

-L file

True if file exists and is a symbolic link.

-r file

True if file exists and is readable.

-w file

True if file exists and is writable.

-x file

True if file exists and is executable.

file1 -nt file2 True if file1 is newer (according to modification date) than file2.
file1 -ot file2 True if file1 is older than file2.
file1 -ef file2 True if file1 and file2 have the same device and inode numbers.
Less frequently used operations
-a file True if file exists.
-b file True if file exists and is a block special file.
-c file True if file exists and is a character special file.
-g file True if file exists and is set-group-id.
-h file True if file exists and is a symbolic link.
-k file True if file exists and its "sticky'' bit is set.
-p file True if file exists and is a named pipe (FIFO).
-s file True if file exists and has a size greater than zero.
-t fd

True if file descriptor fd is open and refers to a terminal.

-u file True if file exists and its set-user-id bit is set.

-O file True if file exists and is owned by the effective user id.
-G file True if file exists and is owned by the effective group id.
-S file True if file exists and is a socket.
-N file True if file exists and has been modified since it was last read.

Debugging Scripts
1. Use echo statements.
2. Run bash -n script to check for syntax errors.

3. Use the command set -v to get a verbose dump of each line the shell reads. Use
set +v to turn off verbose mode.
4. Use the command set -x to see what each command expands to. Again, set +x
turns this mode off.
Running it as ./script.sh will make the kernel read the first line (the shebang), and
then invoke bash to interpret the script. Running it as sh script.sh uses whatever
shell your system defaults sh to
. ./my_script.ksh executes the script within your current (probably login) shell.
sh my_script.ksh creates a new shell in a child process and executes within that.
If you run sh file.ksh , you're running "sh" (which may be linked to ksh or bash or
whatever) with file.ksh as input. It's run in a child process, though, so variables you
set are not available later in your current shell.
If you run ./file.ksh , the first 4 bytes of the file (the file "magic") are read,
determined to be a script, and the executable, if available, identified after the
shbang is run, feeding the file as input. Also run in a child, so no variables are
available after control is passed back to your current login shell.

Ways of Executing a Shell Script in UNIX

Execute Shell Script Using File Name
$ ./scriptfile

If you have the shebang, then it will be executed using the command interpreter specified
in the shebang.
Execute Shell SCript by Specifying the Interpreter
$ sh scriptfile

rrespective of what is being used as shebang, the interpreter which you have specified
will be used for execution. You can use any interpreter (sh, ksh, bash, csh etc.,).
Execute Shell Script Using . ./ (dot space dot slash)
$ . ./scriptfile

This executes the commands specified in the scriptfile in the current shell, and prepares
the environment for you.
Execute Shell Script Using Source Command
$ source ~/.bashrc

What is Library?
A file that contains only functions is called a library. Usually libraries contain no main
code.

GETOPTS:

The parameters to your script can be passed as -n 15 -x 20. Inside the script, you
can iterate through the getopts array as while getopts n:x option, and the variable
$option contains the value of the entered option.

getopts optstring name

optstring - the string which contains the list of options expected in the
command line
name - the variable name which is used to read the command line options
one by one.

Env Variables:
getopts command makes use of 2 environment variables:
OPTARG : contains the argument value for a particular command line option.
OPTIND : contains the index of the next command line option.

TR (TRANSLATE)

tr is an abbreviation of translate or transliterate, indicating its operation of replacing

or removing specific characters in its input data set.

tr [-c] [-d] [-s] [string1] [string2]

-c

Complement the set of characters specified by string1.

-d

Delete all occurrences of input characters that are specified by string1.

-s

Replace instances of repeated characters with a single character.

string1 First string or character to be changed.

string2 Second string or character to change the string1.
Example:
[blstst1a]:/bliss/ $ echo abcdefghijklmnopqrstuvwxyz |tr a-z A-Z
ABCDEFGHIJKLMNOPQRSTUVWXYZ
$ echo "my username is 432234" | tr
-d [:digit:]
Output: my username is

$ echo "the geek stuff" | tr -d 't'

Output: he geek suff

translate all the white-space to tabs:

-s option to squeeze the repetition of

characters

$ echo "This is for testing" | tr

[:space:] '\t'
Output:
This
is
for
testing

$ echo "This is for testing" | tr

-s [:space:] ' '
Output:
This is for testing

To remove all characters except digits

$ echo "my username is 432234" | tr
-cd [:digit:]
Ouput:
432234
tr 'abcd' 'jkmn'

maps all characters a to j, b to k, c to m,

Join all the lines in a file into a single line

$ tr -s '\n' ' ' < file.txt

and d to n.

Unix tar command line options

c -- create, for creating tar file
v -- verbose, display name of files including, excluding from tar command
f -- following, used to point name of tar file to be created. it actually tells tar command that
name of the file is "next" letter just after options.

x -- extract, for extracting files from tar file.

t -- for viewing content of tar file
z -- zip, tells tar command that create tar file using gzip.
j - another compressing option tells tar command to use bzip2 for compression
r -- update or add file or directory in already existed .tar file
wildcards -- to specify patters in unix tar command

Example:
tar -cvf kans.tar *

tar -xvf kans.tar

SCP Command:

scp [options] username1@source_host:directory1/filename1

username2@destination_host:directory2/filename2
The location of the source file is specified by
username1@source_host:directory1/filename1 , which includes the:

Name of the account on the host computer (username1)

Hostname of the computer on which the source file resides (source_host)
Name of the directory containing the source file (directory1)
Filename of the source file (filename1)

The location to which the source file will be copied is specified by

username2@destination_host:directory2/filename2 , which includes the:

Name of the account on the destination computer (username2)

Hostname of the computer to which the source file will be copied
(destination_host)
Name of the directory to which the source file will be copied (directory2)
Filename of the copy (filename2)

use scp with the -r option. This tells scp to recursively copy the source directory and its
contents.

scp -r script_prg
bliss@k584.smst290.att.com:/bliss/home/kannan/
Therefore, to copy all the .txt files from the revenge directory on your deathstar.com
account to your revenge directory on empire.gov, enter:
scp dvader@deathstar.com:"revenge/*.txt" ~/revenge/
without password:
Step 1 : local host
> ssh-keygen -t rsa

and when prompted for pass phrase I hit enter. Then id_rsa , id_rsa.pub files created in
<usershome>/.ssh directory.
Step 2 : local host
> scp id_rsa.pub <remoteIP>:/tmp/mykey

, prompted for password as usual and provided password to copy.

Step 3 : Remote host
> cd ~/.ssh
cat /tmp/mykey >> authorized_keys2

step 4 : Remote host

> ln -s authorized_keys2 authorized_keys

[blstst1a]:/home/bliss/kannan $ ssh-keygen -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/home/bliss/.ssh/id_rsa):
/home/bliss/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/bliss/.ssh/id_rsa.
Your public key has been saved in /home/bliss/.ssh/id_rsa.pub.
The key fingerprint is:
f5:93:0a:d6:d3:57:7e:8c:7d:29:b1:90:6b:a0:4e:6e bliss@blstst1a
[blstst1a]:/home/bliss/kannan $
[blstst1a]:/home/bliss/.ssh $ ls -lrt
-rw------1 bliss
bliss
-rw-r--r-1 bliss
bliss

963 Jul 17 23:50 id_rsa

224 Jul 17 23:50 id_rsa.pub

[blstst1a]:/home/bliss/.ssh $ cat id_rsa.pub

ssh-rsa
AAAAB3NzaC1yc2EAAAABIwAAAIEAxGC2eQlc/3DvFS42hYx8FUhkTvwyHDEnmQTKUn3y12s

IhtvnKfI8Xyhl98eD1eSK2PY4KZ/BbsjCVptpFfGRiQi0Z9C4ApeRukPFGBNqU+X0ComaOj
NHwI5m8/OrwtoWn2osJnQVw9In1u2f8Z5kkgjJCrY/IAS+zMUaR5CUr78=
bliss@blstst1a
[blstst1a]:/home/bliss/.ssh $ scp id_rsa.pub
bliss@k584.smst290.att.com:/bliss/home/authorized_keys
Password:
id_rsa.pub
100% 224
0.2KB/s
00:00
[blstst1a]:/home/bliss/.ssh $
[blstst1a]:/home/bliss/kannan $ scp -r script_prg
bliss@k584.smst290.att.com:/bliss/home/kannan/
Enter passphrase for key '/home/bliss/.ssh/id_rsa':
sample_prt.sh
100% 1600
1.6KB/s
00:00
LOG_OUTP.txt
100% 4000
3.9KB/s
00:00
.txt
100%
16KB 15.8KB/s
00:00
iftesh.sh
100% 126
0.1KB/s
00:00
final_rpt.txt
100%
13KB 12.7KB/s
00:01
shift.sh
100%
63
0.1KB/s
00:00
Sticky bit:It is a user ownership access-right flag that can be assigned to files and directories
on Unix systems.
The sticky bit can be set using the chmod command and can be set using its octal
mode 1000 or by its symbol t (s is already used by the setuid bit).

chmod -t /usr/local/tmp

chmod 0777 /usr/local/tmp

1. T The sticky bit is set (mode 1000), but not execute or search permission.
2. t The sticky bit is set (mode 1000), and is searchable or executable.
The Entry Type:

There are total 10 bits -rw-r--r--:

The file mode printed under the -l option consists of the entry type (1st bit) and the
permissions (9 bits). The entry type character describes the type of file, as follows:
1.
2.
3.
4.
5.
6.
7.
8.

- Regular file.
b Block special file (stored in /dev).
c Character special file (stored in /dev).
d Directory.
l Symbolic link.
p FIFO.
s Socket.
w Whiteout.

setuid and setgid:

setuid and setgid (short for "set user ID upon execution" and "set group ID upon
execution") are Unix access rights flags that allow users to run an executable with
the permissions of the executable's owner or group respectively and to change
behaviour in directories.
Setting the setgid permission on a directory (chmod g+s) causes new files and
subdirectories created within it to inherit its group ID, rather than the primary
group ID of the user who created the file (the owner ID is never affected, only the
group ID).
[root@foo]# find /path/to/directory -type d -exec chmod g+s {} \;
What are the Unix system calls for I/O?
- open(pathname,flag,mode) open file
- creat(pathname,mode) create file
- close(filedes) close an open file
- read(filedes,buffer,bytes) read data from an open file
- write(filedes,buffer,bytes) write data to an open file
- lseek(filedes,offset,from) position an open file
- dup(filedes) duplicate an existing file descriptor
- dup2(oldfd,newfd) duplicate to a desired file descriptor
- fcntl(filedes,cmd,arg) change properties of an open file
- ioctl(filedes,request,arg) change the behaviour of an open file

List the system calls used for process management:

System calls
fork()

Description
To create a new process

exec()

To execute a new program in a process

wait()

To wait until a created process completes its execution

exit()

To exit from a process execution

getpid()

To get a process identifier of the current process

getppid()
nice()
brk()

To get parent process identifier

To bias the existing priority of a process
To increase/decrease the data segment size of a process

Example:
if [[ ${PRODUCT_NAME} = "app_rel" ]]; then
while [[ $I -lt ${#App_Dirset[*]} ]]; do

OUTPUT_LINE="Copying Directory.....: [ ${App_Dirset[$I]} ]"

logit 0 "
${OUTPUT_LINE}
\c"
if [[ ! -d ${DESTINATION}/${App_Dirset[$I]} ]]; then
mkdir -p ${DESTINATION}/${App_Dirset[$I]}
fi
chmod 775 ${DESTINATION}/${App_Dirset[$I]} 2>>$MESSAGE_LOG
let I=$I+1
done
fi
sample sql script:
#!/bin/ksh
. /bliss/ofc/script/bliss_env
sqltmp="/home/bliss/kannan/unix_work/sample.sql"
echo "set feedback off;" > ${sqltmp}
echo "set serveroutput on;" >> ${sqltmp}
echo "select count(*) \"number of TNs\" from telephone_number where
customer_order_id = 198425;">>${sqltmp}
echo "select count(*) \"order elements in coe\" from coe where
customer_order_id=198425;">>${sqltmp}
sqlplus -s $ORACLE_LOGIN @${sqltmp} <<eof
eof
[blstst1a]:/bliss/ofc/script $ uname -n
blstst1a
[blstst1a]:/bliss/ofc/script $ uname
HP-UX
[blstst1a]:/bliss/ofc/script $ uname -a
HP-UX blstst1a B.11.00 U 9000/893 371349291 unlimited-user license
String reverse:

$ echo welcome | rev

emoclew

$ echo welcome | awk '{

for(i=length;i!=0;i--)x=x
substr($0,i,1);}END{print x}'
Out put:
Emoclew

Explain:
The length command gives the length of the argument passed to it.
With no argument, length gives the length of the current line which is $0.
The substr command in awk extracts one character at a time and is appended
to the resultant variable x which is printed at the end using the END label.

--------------------------------#!/bin/bash x="welcome"
len=`echo ${#x}`
while [ $len -ne 0 ] do
y=$y`echo $x | cut -c $len`
((len--))

done
echo $y
--------------------------------cat sample.sh
#! /bin/sh
# reverse a string
[[ -z "$@" ]] && STR=" a b c d e f g h i j k l m n o p q r
s t u v w x y z " || STR="$@"
len=${#STR}
REV=""
for (( i=$len ; i>0 ; i-- ))
do
REV=$REV""${STR:$i-1:$i}
STR=${STR%${STR:$i-1:$i}}
done
echo "Reversed string"
echo $REV
output:

[~/temp]$ ./sample.sh
Reversed string
z y x w v u t s r q p o n m l k j i h g f e d c b a
Print the file content in reverse order:

$ cat file
Linux
Solaris
AIX
$ nl file | sort -nr | cut
-f 2AIX
Solaris
Linux
nl commands numbers the
file contents.
sort sorts the numbered
file in the reverse order
cut command removes the
number and prints the file
contents.

$ tac file
AIX
Solaris
Linux
$ awk '{a[i++]=$0}END{for(j=i1;j>=0;j--)print a[j];}' file
AIX Solaris Linux
Note:
One by one line is read and stored in an
array. At the end, the array is printed in
the reverse order by starting from the last
index.

To print the last line of a file:

$ cat file
Unix
Solaris
Linux
#!/bin/bash
while read line
do
x=$line
done < file
echo $x
$ awk 'END{print}' file
Linux
$ sed '$!d' file
Linux
Note:

$ tail -1 file
Linux

$ sed -n '$p' file

Linux
$ tac file | head -1
Linux

To delete(d) all the lines other than(!)

the last line($) which in turn prints only
the last line.
How to zero pad a number or a variable?

$ x=23
$ printf "%04d\n"
$x

$ typeset -Z4 x
$ x=23
$ echo
$x

Output:
0023

Ouput:
0023
Note:typeset has an option -Z which is

$ x=23
$ echo $x | awk '{printf
"%04d\n", $0;}'

x=23
while [ ${#x} -ne 4 ];
do
x="0"$x
done
echo $x

used for zero padding (only in ksh).

Output:
0023

Output:
0023
Insert a line before or after a pattern:Inserting before the pattern:

$ cat file
Linux

$ awk '/Fedora/{print
"Cygwin"}1' file

Solaris
Fedora
Ubuntu
AIX
HPUX

Linux
Solaris
Cygwin
Fedora
Ubuntu
AIX
HPUX
Note:
The '1' at the end prints every line by
default. On finding the pattern 'Fedora',
'Cygwin' is printed. Hence, first "Cygwin"
gets printed followed by 'Fedora'(1).

$ sed
's/.*Fedora.*/Cygwin\n&/'
file
Output:
Linux
Solaris
Cygwin
Fedora
Ubuntu
AIX
HPUX
Note:
On finding the pattern 'Fedora',
substitute with 'Cygwin' followed by the
pattern matched.

#!/usr/bin/bash
while read line
do
echo $line | grep -q "Fedora"
[ $? -eq 0 ] && echo "Cygwin"
echo $line
done < file
Note:
A line is read. grep -q is silent grep
where the result, if any, will not be
displayed. The status($?) will be 0 if a
match is found and hence 'Cygwin' is
printed.

Inserting After the pattern:

$ awk '/Fedora/{print;print
"Cygwin";next}1' file
Output:
Linux
Solaris
Fedora
Cygwin
Ubuntu
AIX
HPUX

$ sed
's/.*Fedora.*/&\nCygwin/'
file
Output:
Linux
Solaris
Fedora
Cygwin
Ubuntu
AIX
HPUX

Note:
The difference here is the addition of two
extra commands: print and next. print
will print the current line, and the next
command will skip the current line from

Note:
Patten matched(&) followed by "Cygwin".

printing again.

#!/usr/bin/bash
while read line
do
echo $line
echo $line | grep -q "Fedora"
[ $? -eq 0 ] && echo "Cygwin"
done < file
Different ways to display the contents of a file:File name: a

$ cat a
a
b
c
---------------cat < a

$ grep '.*' a
a
b
c

$ paste a
a
b
c
Note:
To paste contents from
multiple files. But, when
paste is used without any
options against a single
file, it does print out the
contents.
$ while read line
do
echo $line
done < a

Note: to match anything(.*)

Output:
a
b
c

$ xargs -L1 < a

a
b
c

$ sed '' a
a
b
c

Note:
xargs takes input from
standard input. By default,
it suppresses the newline
character. -L1 option is to
retain the newline after
every(1) line.

Note:
sed without any command
inside the single quote just
prints the file.

$ sed -n 'p' a
a
b
c

$ sed -n '1,$p' a
a
b
c

$ awk '1' a
a
b
c

$ awk '{print;}' a
a
b
c

Note:
1 means true which is to
print by default. Hence,
every line encountered by awk
is simply printed.
$ awk '{print $0;}' a
a
b
c

note:
The print command of awk
prints the line read, which
is by default $0().