Linux Command Line

Saturday, October 22, 2016

5:52 PM

Command

Meaning

man {cmd}

Show manual for command {cmd}

cd

Go back to HOME dir

cd ..

Go back to previous working dir

pstree -p

Display proc tree

pgrep {name_pattern}

Find/Kill signal proc by name

Shortcuts

- Ctrl-Z: Suspend already running process

- Ctrl-C: Kill running process

pkill {name_pattern}

find . -iname {name_pattern}

Locate file by name in current dir

locate {name_pattern}

Find file by name in anywhere

{cmd} &

Start cmd in background

jobs, bg

See jobs currently running in background

ps

Show all running processes with their PID

kill {pid}

Kill proc based on PID

fg

Bring back most current job running in background

ls -l
head -n={k} {file_list}

- {cmd} {args } > {output}: Overwrite output file

- {cmd} {args } >> {output}: Append output file

- Output of command can be treated like a file with <

Print the first 10 lines of each file

If k provided: Print the first k line

tail -n={k} {file_list}

Input/Ouput Redirection

Print the last 10 lines of each file

If k provided: Print the last k line

less/more {file}

Print content of file

ln -s

Make symbolic link

ln

Make hard link

chown

Change owner of file

chmod

Change file mode bits

COMP 3111H Page 1

diff /etc/hosts <(ssh somehost cat /etc/hosts)

Git
Tuesday, October 25, 2016

10:55 AM

Command

Meaning

git init

Create new git repo

git add {filename}

Add change to index

git add *

Add all changes to index

git commit -m {Message}

Commit, but not in remote repo yet

git checkout - b {branch_name}

Create a new branch

git checkout master

Switch base to master branch

git branch -d {branch_name}

Delete branch

git merge {branch_name}

Assume that you in master branch

Merge changes in {branch_name} to master

Conflict happened: Manually edit conflicts
shown by git. After change, mark them as
merged:
git add {file_name}
git diff {source_branch} {target_branch}

Show difference between source branch and

target branch

git log --author={author_name}

Show commit logs of certain author

git log --pretty=oneline

Show compress log, each commit in one line

git log --graph --oneline --decorate --all Visual history by ASCII art tree
git clone {url}

Check out repo

git push {remote_name} master

Send change to remote repo

Usually, remote_name = origin

git remote add {remote_name} {server}

Add remote repo as {remote_name}

git push {remote_name} {branch_name}

Publish a local branch to server at

{remote_name}

git pull

Update local repo with newest commit from

remote

(Pull = Fetch + Merge)

Github Flow

- Fork:
Copy a repository, allow you to freely experiment with changes
without affecting original repository
Can be used as:

Propose changes to someone else's project

Use someone else project's as starting point
- Add issue: Describe what you are going to work on
- Create a branch
- Add commit

- Open Pull request:

Tell other about changes you've pushed into GitHub repository
Once sent, interested parties can review your changes, discuss
potential modification, event follow-up commits if needed
- Discuss & Review Code
- Test / Deploy

- Merge
* NOTE: Update fork repository
upstream: Name of remote original repository which is forked
origin: Remote storage of your project (forked from upstream
previously)
git fetch upstream

git rebase upstream/master

git push -f origin master

COMP 3111H Page 2

Software Development Process

Tuesday, October 25, 2016

11:50 AM

Overview of Software Development

- 4's P: Process, People, Project, Product

- General stages:
Gather sys requirements
Analyze & Design sys
Implement sys

Test sys
Maintain sys

Conventional Development Process

Name

Illustration

Code & Fix

Pros

Cons

- Easy to kick-start

- Messy code structure: Frequent changes in

implementation
- Unsuitable for large sys:
Difficult to maintain code

Missing/Unmatched requirements
- Unpredictable dev proc
Over schedule, over budget

Waterfall

- Dev proc: More predictable, easy to monitor - Requirement revision is difficult: The cost
(time, human) of changing requirements is
expensive
- Enforce standards:
Documentation (requirements,
implementations) required
Approval required before proceeding
to next steps

- Suitable for junior developers:

Sequential workflow
Details, unambiguous documentations

Prototyping

- Cannot detect code bug early

May introduce bugs which are
difficult to fix (due to completion of sys)
- Next phase cannot be started until previous
phase completed
- Product not available for feedback until end
of dev proc High risk of user dissatisfaction

- Suitable when initial requirements vague,

- A lot of throw out & rebuild:
known & exploration required:
Not emphasize code maintainability &
Get clearer understanding of sys func
reusability

May lead to over schedule, budget

- Different solution can be explored High
chance of finding "right" soltion
- Allow continuous user feedback

Similar to Code & Fix but require:

COMP 3111H Page 3

- Finally built sys may not be stable

 Discipline (Sys design)

Artifacts produced (Requirement Documentation, )
Spiral

- Accommodate good features of Waterfall & - More appropriate for internal dev than
Prototyping:
external contract dev: Lot of dev-client
comm required to get things "right" soon
Waterfall: Enforce discipline
(documentation)
Prototyping: Early feedback

- Rely on expertise on risk assessment

- Focus early attention on reusing code

Dev time
- Focus on early elimination of risks Stable
sys

- Emphasize RISK:

Technical: Use right tech, building right architecture

Non-technical: Training needed, customer approval,
on-schedule
- ACMM: Risk minimization:

Avoid: Replan, change requirement

Confront: Restrict scope of risk's effect

Mitigate: Devise test if detect risk

Monitor: Constantly check dev proc
Phased

- Promote sys modularity:

Well-structured, robust, maintainable
sys
Allow appropriate expertise to be
applied

- Sys components need relatively small (to be

maintainable)
- Difficult in identify common facilities needed
by all pieces

- Solid features added incrementally Risk

of whole project failure
- Early feedback + Frequent release Can
create new market early

Agile Development

Agile Practices
Individual & interaction over processes & tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Response to change over following to plan

- User story: Formulate user requirement

As a {USER ROLE}, I want to do {GOAL} so that {BENEFITS}

- Test Driven Programming (TDD): Write unit test before writing
code

Principle:
1. Highest prio: Satisfy customers through early & continuous
delivery of valuable software

- Standup meeting

2. Welcome changing requirements, event late in dev.

- Planning poker: Estimate dev time of feature

Harness change for customer's competitive advantage

3. Deliver working software frequently, from weeks to month

COMP 3111H Page 4

3. Deliver working software frequently, from weeks to month

Prefer to shorter timescale
4. Business & dev must work together daily

Agile Development Process

5. Build projects around motivated individual.

Give them env

1. SCRUM

Support their need

Trust them to get job done
6. Face-to-face: Most efficient method of conveying info within
dev team
7. Working software: Primary measure of progress
8. Continuous attention to technical excellence & good
design
9. Simplicity: Art of max amount of work not done is essential
10. Best architectures, requirements & design emerge from selforganizing teams

Advantages:
- Low criticality (suitable for developing sys which is not
mission-critical, not require extremely precise engineering)
NOT for rocket control software,
- Suitable for small team of senior devs

Lean analytics

- Analytics: Measurement of movement towards your business goals

- Purpose:
Deal with ambiguous user requirements
Help startup iterate product/market fit before money runs out

COMP 3111H Page 5

SCRUM Meetings: Sprint Planning, Daily Scrum, Sprint Review

2. Extreme Programming (XP)

Requirements
Thursday, October 27, 2016

12:33 AM

Requirement validation

- Complete: Descriptions of all facilities required?

Requirement Overview

- Consistent: No conflics/contradictions in description of sys facilities?

- Def: Features sys must have, constraints it must satisfy Accepted by client

- Realism: Possible to implement sys with available budget + tech?

- Purpose:
Help software dev recall goals, details

- Validity: Sys funcs best support customer' needs?

Means of comm between clients & dev team: Help clarify user expectation & reality

- Verifiability: Can requirements checked?

- Type:
User requirement

System requirement

- "Customer wishes" as understood by - "Solution sketch" of "Solution provider"

"Solution provider"
- Use language of problem domain

- May be part of contract between client &

constractor

- Describe:

- Use language of solution domain

Software requirements documents

- Def:
Statement of what should be built by developers

What services provided by sys

Operational constraints of sys

= User + Sys requiremets

- Describe:

NOT a Software-Architect Design Document

How sys provides its required services

How sys operates under required
constraints
1.1. The MHC-PMS shall generate
monthly management reports showing
the cost of drugs prescribed by each
clinic during that month.

- Style + Content: Depends on sys types, software dev proc used

Document for sys developed incrementally will have less details

1.1 On the last working day of each month, a

summary of the drugs prescribed, their cost and
the prescribing clinics shall be generated.

Industry standards (IEEE) mostly applicable to requirements for large sys

projects
- Structure:
Preface:

1.2 The system shall automatically generate the

report for printing after 17.30 on the last working
day of the month.

Who are target reader?

Software version history:
What are changes made in each version?

1.3 A report shall be created for each clinic and

shall list the individual drug names, the total
number of prescriptions, the number of doses
prescribed and the total cost of the prescribed
drugs.
Functional requirement

Why create new version?

Introduction: Briefly answer
Why sys is needed?
What funcs provided by sys & How they work?

Non-functional requiremet

How sys fits into overall business

Describe:

What contraints are enforced in operations of

- What services must be provided by sys sys services
Service available time
- How these services will be provided:

How sys react to inputs?

I/O device capability

How sys behave in situations?

Sys representation

Glossary: Define technical term (no assumption about reader experience)

User requirement def

1. A user shall be able to search the

appointments lists for all clinics.

System architecture: High-lvl overview of sys structure

1. The MHC-PMS shall be available to all clinics

Sys requirement specification

during normal working hours (MonFri,

083017.30). Downtime within normal working

2. The system shall generate each day, for

hours shall not exceed five seconds in any one day.
each clinic, a list of patients who are
expected to attend appointments that day.
2. Users of the MHC-PMS system shall authenticate
themselves using their health authority identity
3. Each staff member using the system
shall be uniquely identified by his or her 8- card.

digit employee number.

Sys models: Visualize rel between sys components & external env
Sys evolution:
What are assumptions where sys based on?
In future, what are potential sys changes due to hardware,
changing user needs?

3. The system shall implement patient privacy

provisions as set out in HStan-03-2006-priv.

Appendix
Index

Conventional ways of Writing Requirements

1. Natural language:
- Sentences + Supplement diagrams

- Advantages: Expressive, intuitive

- Problems;
Lack of clarify: Precision Difficult to read document
Confusion:
Mix up func & non-func requirements

COMP 3111H Page 6

User Story

(Writing requirements for Agile development)

As a {USER ROLE}, I want to do {GOAL} so that {BENEFITS}

- Problems;
Lack of clarify: Precision Difficult to read document

Confusion:
Mix up func & non-func requirements
Mix up different requirements into single one
-

- Guideline:
Use standard format throughout documents
Be consistent

SHALL = Mandatory
SHOULD = Desirable
Highlight text Identify key parts
Avoid computer jargons

- Def: 3C:
Card: Written on a cue card
Conversation:

Avoid vague terms

Details of requirements exposed in conversation

among dev teams & business users

some, usually, often,

Clear def for terms

User story is JUST starting point for that

conversation

always, never,
Explain uncompleted list (etc, so on) clearly
2. Structured language:
- Writen requirements in standardized format

- Advantages: Clear, unambiguous

- Problems: Too rigid for business sys requirements
- Types: Form-based, tabular,

Confirmation/Condition of satisfaction: Supplement

main story card:
Used by dev team to better understand what to
build & test

Help confirm that user story implemented up to

user satisfaction
- What make a good story:
Independent
Negotiable

Valuable
Estimatable
Small (size appropriately)
Testable

COMP 3111H Page 7

UML

Modeling

Wednesday, October 26, 2016

11:42 AM

- Beneftis:
Describe reality in succinct way

UML

- Visual modeling language for sys

- Standard Object-Oriented (OO) modeling lang
- Process neutral
- Allow dev, architecture & experience users to discuss
inner working of software

Help focus on big picture

Better understanding of requirements Cleaner design,
more maintainable sys

- Type:
Static modeling: Static, code-lvl rel within sys
Class, Components, Package diagram,
Behavioral modeling: Dynamic exec of sys
Use case, Collaboration, State Diagram,

OO Concepts: Object & Class

- OO Modeling steps:
Requirement: Construct problem domain

- Object:
- Physical/abstract thing having boundary

- Can repond to external request

- Properties: Identify, state, behavior

Identify objects, actors,

Analysis & Design: Construct solution domain
Identify how object interacts with each other

Implementation: Implement solution

How to code objects

- Class:
Def: Group of Object having common in: semantics, state,
behavior, relationship:
Template for creating object
Provide abstraction for collection of individual objects

OO Concepts: Link & Association

Complexity in understanding, developing sys

Architecture Design = Choosing right class to represent objects

- Link: Physical/Conceptual rel between objects

Attribute: Data held by object, described by:

- Association: Group of link with common semantics (rel between class)

Name, Type, Initial Value

Visibility (private/public/protected, )
Changeability (readonly?)
Multiplicity (simultaneous existence?)
Operation: Functions provided by object
Described by:
Input, Output
Function arguments
Visibility (private/public/protected, )
Side effect: Operation exec change object state

- Multiplicity:
Restriction on number of ojects of a class that may related to an
object of another class

Instructor can teach 0 Course

Course can be taught by 1 Instructor

Role: 1 end of association

Min/Max Cardinality:

COMP 3111H Page 8

 Role: 1 end of association

Min/Max Cardinality:

Association A
Object of class C1 can related to x objects of class C2
c x d c = MinCard(C1, A)
d = MaxCard(C1, A)
Object of class C2 can related to y objects of class C1
a y b a = MinCard(C2, A)
b = MaxCard(C2, A)

Special symbols:

OO Concepts: Generalization

a *: MaxCard =

(Rel between classess of same kind)

*: MinCard = 0, MaxCard =
1: MinCard = MaxCard = 1

- Inheritance: Assumption of props by subclass from superclass

Description redundancy

MinCard = 0: Optional participation

MinCard > 0: Mandatory participation

Simplify modification
Promote reusability

- Abstract class: No direct instance Container for def

- Coverage:
Overlapping

Disjoint

Complete

A superclass OBJECT can be

member of > 1 subclass

A superclass OBJECT can be

member of 1 subclass

All superclass OBJECTS must be Some superclass OBJECTS is

member of some subclasses
NOT member of an subclass

COMP 3111H Page 9

Incomplete

A Player object can be both of

Tennis & Soccer class

All UniversityStudent objects

must be member of Undergrade
An Account object must be either or Postgrade
of Personal or Business class

COMP 3111H Page 10

Some Tree objects is not member

of Oak, Elm or Birch class

Class Diagram
Wednesday, October 26, 2016

9:00 PM

UML Symbols

Name

Symbol

Meaning

Generalization

A subclass B
= A inherit B

= ClassA "IS A" Class B

Realization

Class A implement Interface B

Dependency

Class A may invoke Class B in its operation

Association

Class A object "HAS A" Class B

Aggregation

- Class A object "owns" x Class B objects

(a x b)
- Class B objects may exist independent of
Class A object

Composition

- Class A object "made of" x Class B objects

(a x b)
- Class B objects may NOT exist independent
of Class A object

Class

Constraint

Assertion about props of model must be

enforced by sys

COMP 3111H Page 11

Constraint

Assertion about props of model must be

enforced by sys

Dynamic Modeling
(For modeling programs)

Static Modeling

(For modeling data)

COMP 3111H Page 12

Use-Case Diagram
Tuesday, October 25, 2016

3:13 PM

Purpose

- Capture sys behavior from users's point of view

- Caputre data & func requirements

- Plan phase/iteration of dev

- Help validate sys

UML Symbols

Name
Actor

User case

Symbol

Meaning

How to identify

- Represent something
outside sys

- Who/What uses/interacts with sys?

- Interact directly with sys

- Who/What receive info from sys?

- Provide input/Receive
output from sys

- Who/What install, start, shutdown,

maintain sys?

- Specific way of using sys

- What tasks actor wants sys to perform?

- Describe what sys must do

- What info actor access?

- Init by actor

- Which external changes actor need to

inform sys?

- What role they play?

- Which event actor need to be informed

about sys?

Assocation

COMP 3111H Page 13

Sequence diagram
Tuesday, October 25, 2016

3:22 PM

UML Symbol
Best practices
Name
Boundary class

Symbol

Meaning
- Interact with:

Actors outside sys

Object within sys
- Represent abstraction of UI elements:
Describe WHAT NOT HOW (no need
to describe every button, menu, )
Entity class

- Represent concepts:
Individual, real-life object, real-life
event,

- Get data from database

- Encapsulate/Isolate changes to info it
represent
Control class

- Model coordination, sequencing,

transaction & control behavior

- Glue that ties other classes together into

use-case realization

COMP 3111H Page 14

- Actor interact only with Boundary object

- Boundary object interact only with Control
object & Actor (initially)
- Entity object interact only with Control object
(initially)
- Control object interact with Boundary, Entity
& other Control objects

State Machine Diagram

Tuesday, October 25, 2016

3:36 PM

Overview

- Purpose: Describe behavior inside an object

- Directed graph:
Node: State

Then when object satisfy some conditions,

wait for event
Can discovered from Sequence diagram
Arce: Event
Something happens at INSTANTANEOUS
point in time
Type:
Call event: Sync call from object
Change event: Specific conditions
come true
Time (Signal) event: Receipt of async
comm from object
- Transition of state:
Take zero time
Can't be interrupted

COMP 3111H Page 15

Design Patterns
Wednesday, October 26, 2016

10:47 PM

Architecture Pattern

Name

Illustration

ModelViewController
(MVC)

Explaination

Benefits

- Model:
Manage sys data

Separate presentation &

interaction from sys data

Associate operations
on data
- View: Define & Manage
how data presented to
user
- Controller:
Manage user
interaction
Pass user interaction
to View & Model

Layered

ClientServer

Organize sys into set of

layers, each provide set of
services

Support incremental dev of

sub-sys in different layers:

- Server: Provide specific

services

Suitable for distributed sys

When layer interface

changes, only adjacent
layer affected

- Clients: Call these service

- Network: Allow client to
access server

Repository

- Repository: Maintain
shared data, can be
accessed by sub sys

Efficient for sys having large

amount of shared data

- Sub sys:
Maintain own data

Pass data explicitly

to others

Pipe-and-

Functional transformation

COMP 3111H Page 16

Easy to combine different

Pipe-andFilter

Functional transformation
process input, produce
output

- Easy to combine different

component to create different
result
- Each component can be
developed indept

COMP 3111H Page 17

Object-Oriented Pattern
Thursday, October 27, 2016

11:47 AM

OO Patterns

Principles:
1. Encapsulation:
Identify the aspects of application that vary
Separate them from what stays the same
2. Program to an interface, not an implementation
3. Favor composition over inheritance (HAS-A can be better than IS-A)

4. Strive for loosely coupled designs between objects that interact

5. Classes should be open for extension, but closed for modificaion

Name

Illustration

Algorithm

Observer

Explaination

Benefits

- Define a family of algorithms

- Encapsulate each

Lets algorithm vary

independently from clients
that use it

- Subject: Publish info

- Principle 1:

- Subscriber: Subscribe & Get info

- One-to-many dependency
between Subject - Subscriber:

Number & Type of

Subscriber vary
Subscriber separated
from state of Subject

When Subject changes state, - Principle 2 & 4: Subjects &

Subscribers only know each
all Subscriber notified &
other via interface
updated automatically
- Principle 3:
Observers composed
with their Subject
No interitance

Mediator

Mediator: Object that

COMP 3111H Page 18

Object reusability:

Mediator

- Mediator: Object that

- Object reusability:
encapsulates how a set of objects Decouple them by Mediator
interact
- Avoid having objects explicitly
know about each other

- Easy maintenance:
Centralize control logic

- Variety of msg sent among

objects (only consider
Set of objects comm in well- object-Mediator msg
format)
defined but complex way

- Use when:

Object reuse is difficult

because it refers to & comm
with many others
Behaviors distributed
among classess and should
be customized without
subclassing

- Many-to-many interaction
1-to-many

Adapter

Convert interface of a class into

another interface of client expect

Let classes work together

that could't otherwise because
of incompatible interfaces

Decorator

- Attach additional responsibilities

to object dynamically

Provide flexible alternative

to subclassing for extending
functionality

- Component: Abstract
Beverage
- ConcreteComponent:
Inherit Component
Dynamically add new
behaviors
HouseBlend, DarkRoast,
Expresso, Decaf

- Decorator:
Abstract class: Provide
foundation for decorator
subclasses
Inherit Component, as need
to be able to wrap around
Component-inheriting
classes
CondimentDecorator
- ConcreteDecorator:
Inherit Decorator
Has instance variable for
thing it decorates (wraps)
Extend existing methods by
combining results returned
by them with some new
computation
public abstract class Beverage {
String description = "Unknown Beverage";

COMP 3111H Page 19

Mocha.getDescription()
Mocha.getCost()

Mocha.getCost()
public String getDescription() {
return description;
}
public abstract double cost();

Can extend state of

component

Milk, Mocha, Soy, Whip

public class HouseBlend extends Beverage {

public HouseBlend() {
// Change "abstract" value of inherited variable with "concrete" one
description = "House Blend Coffee";
}
public double cost() {
return 0.89;
}
}
...

// Condiments need to be interchangable with Beverage, so extend Beverage

public abstract class CondimentDecorator extends Beverage {
public abstract String getDescription();
}
// Mocha -> CondimentDecorator -> Beverage
public class Mocha extends CondimentDecorator {
// Hold beverage we are wrapping
Beverage beverage;
public Mocha(Beverage beverage) {
this.beverage = beverage;
}
// Description needs to include each item decorating beverage
public String getDescription() {
return beverage.getDescription() + ", Mocha";
}
// Cost needs to include each item decorating beverage
public double cost() {
return 0.21 + beverage.cost();
}
}
...

Beverage myBeverage = new Daskroast();

myBeverage = new Mocha(myBeverage);
myBeverage = new Mocha(myBeverage);
myBeverage = new Whip(myBeverage);
System.out.println(myBeverage.getDescription());
System.out.println(" $" + myBeverage.cost());
Singleton

Ensure a class has only instance,

provide global point of access to it

public class Singleton {

// Create instance using static initializer
// -> Guaranteed to be thread-safe
private static Singleton uniqueInstance = new Singleton();

// Private constructor -> Only Single can instantiate itself

private Singleton() { }
public static Singleton getInstance() {
return uniqueInstance;
}
}

COMP 3111H Page 20

Factory

public class SimplePizzaFactory {

public Pizza createPizza(String type) {
Pizza pizza = null;
if (type.equals("cheese")) {
pizza = new CheesePizza();
} else if (type.equals("pepperoni")) {
pizza = new PepperoniPizza();
} else if (type.equals("clam")) {
pizza = new ClamPizza();
} else if (type.equals("veggie")) {
pizza = new VeggiePizza();
}
return pizza;
}
}
public class PizzaStore {
SimplePizzaFactory factory;
public PizzaStore(SimplePizzaFactory factory) {
this.factory = factory;
}

public Pizza orderPizza(String type) {

Pizza pizza;
// Use factory to create pizza
pizza = factory.createPizza(type);
pizza.prepare();
pizza.bake();
pizza.cut();
pizza.box();
return pizza;
}
...
}

COMP 3111H Page 21

Refactoring
Tuesday, December 13, 2016

11:53 PM

Refactoring Category

Techniques

Compose Methods

- Extract/Inline method
- Replace temp with Query

Overview

- Introduce explaining variable

- Split temporary variable
- Remove assignment to parameters
- Replace Method with Method Object

- When to refactor:
New feature added

- Substitute Algorithm

Bug fix needed

Move Features between Objects - Move Method/Field

- Extract/Inline Class

Code review

- Hide Delegate

- When NOT to refactor:

Program too broken

- Remove Middle Man

- Introduce Foreign Method

Deadline close

- Introduce Local Extension

Organize Data

- Encapsulate Field/Collection

- Replace Data Value with Object

- Change Value Reference
- Replace Array with Object
- Duplicate Observed Data
- Change Unidriectional Bidirectional Association
- Replace Magic Number with Symbolic Constant

- Replace Record with Data Class

- Replace Type Code with Class/Subclass/Strategy
- Replace Subclass with Fields
Simplify Condtional Expressions - Decompose/Consolidate expression
- Remove Control Flag
- Replace Nested Conditional with Guard Clauses

- Replace Conditional with Polymorphism

- Introduce Null Object
- Introduce Assertion
Make Method Calls Simpler

- Rename/Parameterize/Hide Method

- Add/Remove Paramter
- Seprate Query from Modifier
- Replace Parameter with Explicit Methods
- Preserve Whole Object
- Replace Parameter with Method
- Introduce Parameter Object
- Remove Setting Method
- Replace Constructor with Factory Method
- Replace Error Code with Exception

- Replace Exception with Test

COMP 3111H Page 22

Defensive Programming & Debugging

Tuesday, December 13, 2016

11:22 PM

Debugging

Find out why program not functioning as intended

Defensive Programming

Program with validation & debugging in mind

- Testing is NOT debugging:

Test: Reveal hidden problems
Debug: Problem exist, find cause

Tip

What to do

Try not to introduce

defects

- Think before code

- Keep simple:
Modularity
Define interfaces with abstract data type
- Write specifications for all modules

Make error impossible - Use language features:

Java makes memory overwrite bugs impossible
- Use protocol/library/module features:

BigInteger guarantee no overflow

- Impose conventions yourself:
Immutable data structure Guarantee behavioral equality

Protect your program

from invalid inputs

- Check data loaded from external source

- Assertion: Check condition that should never occur

Inputs should falls in range

Read-only variables should not changed
Pointer should non-null
Should include in production code (depend on context):
Stop program if checks fail Prevent program doing
something wrong
However:

Bug not have bad consequence

Need program to continue
Avoid putting executable code into assertion

- Unit testing
- Regression testing: Run test whenever code changes

Easily find code fragment reponsible for observed problem

Observe problem earlier, fix more easily
Handle error

Easy Medium bug:

Look at stack-trace in debugger
Print statement (in organized way)

Hard bug:
Explain bug to friend
Rebuild system from scratch, reboot

Find simplest input provoking bug

Binary search + Add checks Narrow down
possible location

- Check routine input parameters

- Decide how to handle bad inputs

Make error
immediately visible

- Techniques:

- Use Exception
- In case of bug:
Return neutral (harmless) value

Return error code

- Log warning message to file
- Show error message
- DONT HIDE BUG: Should make code more robust

COMP 3111H Page 23

Check event log: Very important for debugging in

customer environment (no access to customer use)
Tougher:
Reconsider assumption
OS changed? Hardware changed?
Start documenting system Fresh angle, highlight
area of confusion

Testing

Input Space Partitioning

Sunday, December 11, 2016

5:52 PM

- Principle
Identify sets of inputs with same behavior

Overview of Testing

Test with 1 value from each set

- Purpose: Find differences between expected & actual sys

behavior
Validation: Make sure we build RIGHT product:
Sys functions meets requirements
Verification: Make sure we build product RIGHT
Check implementation quality: Work correctly,
no defect
- Reality of testing:
Cannot prove no error (only quality + confidence)

- What is "same behavior"?:

Sample program
// Should return -x if x < 0, x otherwise
int abs(int x) {
if (x < -2) return -x;
else return x;
}
Nave approach: Execution equivalence:
2 inputs have "same behavior": Make program run same
sequence of steps

Impossible to completely test nontrivial sys

2 exec equivalence:s x < -2, x -2

Destructive activity

Suggest {-3, 3} is good test suite

However, this does not reveal error at x = -2, -1

Difficult of software engineers to effectively test

their own software
- Test design goals:
Highest likelihood of finding defects
Min amoun of time & effort
Test suite: Small enough to run quickly, large
enough to validate + verify software

Better approach: Revealing subdomain

2 inputs have "same behavior":
Give CORRECT result on BOTH, or
Give INCORRECT result on BOTH
Subdomain:
Def: Subset of set of possible inputs

"Revealing" for error E, if:

Elements in subdomain has "same behavior"

- Test case type:

White box ("Testing-in-the-small"):

Program's error E revealed by test

Verify component logic based on data/control

structures

{3, -1}: Best to reveal error

Require source code

Black box ("Testing-in-the-large"):
Verify component functionality based on inputs,
outputs
Not require source code
Regression ("Re-testing"):

Verify no new defects introduced after changes

made

{-3, -2, -1}: Can also reveal error, but not best: Not minimal
- Discovery of input sets:
Require perfect knowledge
Can approx cheaply through heuristics, based on:
Program-dependent info: Actual code
Program-independent info
Requirement spec, data structure,

Selective White + Black box test cases

Regression Testing

- Whenever find bug:

Reproduce
Record input causing bug

Record correct output

Put everything into test suite

Fix bug & Verify

- Why:
Populate good tests
Protect against changes reintroducing bug

COMP 3111H Page 24

White-box Testing
Sunday, December 11, 2016

11:35 PM

Overview

Sample program
Choose subdomains to ensure that we execute all:
- Independent paths, at least 1 Basic Path Testing

1
2
3
4
5
6
7
8
9
10

- Logical Decisions, on both T & F side Condition

Testing
- Loops, at boundaries & within bounds Loop
Testing
- Internal data structure, ensure their validity Data
Flow Testing

Procedure: checkRegistration, Return(result)

Result = "failure"
If (enrollment is open)
If (student doesn't have exclusions)
If (student has prerequisites) OR (student has permission)
Register
Result = "success"
Endif
Endif
End

Basic Path Testing

Step

Example

From code, draw corresponding

flow graph

Determine graph's cyclomatic

complexity
V(G)

= No of region
= No of edges - No of nodes + 2
= No of simple predicate nodes
+1
Region:
- Areas bounded by nodes,
edges
- Area outside graph
V(G) provides upper bound on
no of paths need testing

V(G) = 5
3

Determine basis set of linearly

independent paths based on
cyclomatic complexity

Prepare test cases that force

execution of each path in basis
set

Basis set path

Test case

1 2 3 4a 5 6 7 8 9 10

Enrollment is open

Student doesn't have exclusions

Student has prerequisite
2

1 2 3 4a 4b 5 6 7 8 9 10 Enrollment is open
Student doesn't have exclusions

Student doesn't have prerequisites

Student has permission
3

1 2 3 4a 4b 7 8 9 10

Enrollment is open
Student doesn't have exclusions

Student doesn't have prerequisites

Student doesn't have permission
4

1 2 3 8 9 10

1 2 9 10

Enrollment is open
Student has exclusions

COMP 3111H Page 25

Enrollment is closed

Condition Testing

Loop Testing

- Source of error:
Simple condition: a rel-op b
rep-op {<, , =, , , >}, also NOT
Compound condition: a AND b, a OR b

Types of Loop
Simple (n
iterations)

Illustration

How to test
- 0, 1, 2 passes
- m passes (m < n)
- (n - 1), n, (n + 1) passes

Relational expression: E1 rel-op E2; E1, E2 arthmetic

(a b + c) > (a + b + c)
Boolean (Non-relational expression):

NOT A

Nested

- Outward testing: Test inside out

- Conduct simple loop tests for inner loop
while holding outer at min iteration values

- Branch testing:
For compound condition C, test T & F branch of C
and every simple conditions of C

C = (a > b) AND (c < d), test:

C: T & F

Concatenated

- Independent loops Simple loop testing

- Dependent loops Nested loop testing

a > b: T & F

C < d: T & F
- Domain testing:
For expression E1 rel-op E2, use values that makes

E1 > E2, E1 = E2, E1 < E2

When testing with E1 > E2, E1 < E2: Make |E1 - E2| as
small as possible

Data Flow Testing

- Goal: Ensure value of variable is correct at certain points of execution

- Definition Use (DU) Chain:

Set of [X, S, S'] where X is NOT redefined between S and S'
X: Variable
S: Location of definition statement of X (line number)
S': Location of use statement of X (line number)
- Strategy: Every DU chain must covered once (can be combined with
basis path testing)

COMP 3111H Page 26

Unstructured

Refactor code!

Black-box Testing
Sunday, December 11, 2016

Boundary Testing

11:55 PM

- Motivation: More error occur at "boundaries" than "center" of subdomain

Off-by-one bugs
Forget to handle empty container/null object
Arithmetic overflow
Not handle aliasing of objects

- Principle: Test value at "boundaries" of subdomain

- Heuristic:
Equivalence Partitioning

- Principle: Group inputs by type Subdomain Test coverage of an error

class
Requirement specification Derive equivalent partitions
Divide input/output into partitions of equivalence
Test cases: Cover each equivalence partition 1
- Common heuristic: Select subdomain based on valid & invalid inputs

Input type

How to select subdomain

Range

- 1 valid (in range)

- 2 invalid (< range, > range)

Input

How to select "boundary" value

Range [a, b]

- a, b
- Values just above/below a, b

Set of
discrete
values

- Min, Max value

- Value just above/below them

Set of values - All values in set (if possible)

- 1 value outside set

Specific value - 1 valid (the value)

- 2 invalid (< value, > value)
Boolean

- T, F
- 1 non-boolean value

Data
structure
with
boundaries

- Boundary values
- Value just above/below them

Set of related - 1 valid (set member)

- 2 invalid (non-set member)
values

Boolean

- 1 valid (boolean value: T, F)

- 1 invalid (non-boolean value)

COMP 3111H Page 27

Array with bounds 1 10: Test array index 0, 1, 2, 9, 10, 11

Testing Strategy
Sunday, December 11, 2016

Phase

11:57 PM

Purpose

Unit testing Verify

components
function work
correctly

Responsible parties Test cases

Software engineer
who develops the
code

Issues

- White box

- Driver: Call component to test

- Black box

- Stub: Call by tested component

- Object-oritented testing:
Test every class state

Inheritance & Polymorphism: Test all subclass' methods

Encapsulation: Need methods to reports encapsulated
objects' state (for testing only)

Integration Verify
testing
components
interact
correctly

- Software engineer - White box

- Integration/System
tester

- Black box

Top down testing

Bottom up testing

Sandwich

How

- Top-lvl subsystem tested with

stubs
- New subsystem integrated: Rerun some test

- Group low-lvl subsystems,

tested by driver

- Top-lvl subsystem tested with

stubs
- Low-lvl subsystem grouped,
tested with drivers

Pros

User interface components

tested early

- Easily find interaction faults

- Parallel testing

- Easily design test case

- Total testing time significantly

Illustratio
n

- No stubs needed

Cons

- Low-lvl subsystem tested late

- Need to write stubs

System
testing

Verify system
functions
correctly as a
whole

Integration/System
tester

Black box

User interface components

tested last

Need many drivers & stubs

- Functional: Verify all functions work as specified in requirements

- Performance: Verify design goals met: Stress, Volume, Security, Timing, Recovery

- Pilot: Selected user group verifies common functionality in target environment

Alpha test: Controlled environment, developers can observe users
Beta test: Real environment, bugs uncovered from regular usage patterns
Acceptance Validate
testing
software
against its
requirements

Client, User

Black box

- Validate: function, interface (input/output, design standard), database, performance

- Developers: How to derive acceptance test:
Restate requirements in concise, precise, testable way:
Grouping them
Remove untestable requirements
Add extra requirements gathered from users by looking at:

Use cases
Domain model for database
Performance
For each requirement: Construct evaluation scenario Demonstrate requirementis met

COMP 3111H Page 28

Testing under pressure

Evaluate test
- System's capabilities > Components
- How:
Compare result with goals outlined in plan

Prepare metrics to determine current quality of software

- Metrics:
Testing completeness/coverage: % of test case run, % code tested
Trends in testing error rate: Illustrate reliability

- Post-evaluation possible outcomes:

More test Find more defects
Relax test criteria
Deliver acceptable components, continue revising & testing
unacceptable components

COMP 3111H Page 29

- Old capabilities > new capabilities

Ensure previous functions keep working
- Typical situations > Boundary cases
Ensure system work under normal usage patterns

Automatic Testing
Sunday, December 11, 2016

11:58 PM

Symbolic testing

- Principle:
Execute program symbolically

Adaptive random testing

Use symbols rather than concrete values

Find conditions to get to error conditions

- Main types:

- Definitions:

Random input data generation

Path condition (PC):

Random sequence of data input

Logical formula over symbolic inputs

Random data selection from database

Accumulation of constraints that inputs have to satisfy for particular

path to executed

- Algorithms behind:

Symbolic execution tree:

Uniform random walks

Paths constructed during symbolic execution

Randomized quasi-random sequences

Nodes: Symbolic program states

Edges: Program transitions

Path-oriented random test data generator

Sample program
int foo(int a, int b) {
int x = 0, y = 0, z = 0;
if (a) { x = -2; }
if (b < 5) {
if (!a && c) {
y = -1;
}
z = 2;
}
assert(x + y + z != 3)

Concolic Execution (Concrete + Symbolic)

- Do 2 things simultaneously:
Given concrete inputs, execute program concretely
Also consider inputs as symbols, collect symbolic
conditions along the concrete execution path

Symbolic execution tree: Let a = , b = :

- At each node (on concrete execution path) where branching

happens (if ):
Negate the path condition collected at that node:

Try the opposite branch generated by concrete

execution
Solve the negated path condition Get concrete values
reflecting it
Execute again, from beginning, with the newly found
concrete input values

- Property: Symbolic execution: Program can take any path

(Concrete execution: Program takes 1 path, determined by input)
- Difficulties:
Path explosion: Too many paths
Loops: Number of iteration may unknown

Concrete

Concolic

Symbolic

- Complex programs - Complex program

- Simple program

- Scalable

- Not scalable

- Scalable (lesser)
Need to negate a lot of nodes

- Less coverage

- High coverage

- High coverage

- No false positives

- No false positives

- False positive
Loop programs (we don't know how
many times loop will run)

COMP 3111H Page 30

Joel Testing
Sunday, December 11, 2016

11:58 PM

No

Question

Benefits

Do you use source control?

- Support multiple developers

- Track changes

- Enable roll-back
2

Do you have a one-step build?

- Save time
- Ship new software version faster

Single script:
Full checkout from scratch

Rebuild every line of code

Make binary executable file in all versions,
languages
3

Do you do daily build and test?

- Integration risk, low-quality risk

- Diagnose defect more easily

- Morale: developers, managers, customers

4

Do you use bug database?

- Debug faster (in case of bug)

- Avoid reintroducing bug

Need to record:
How to reproduce bug
Expected & actual behavior
Responsible party, status, priority
5

Do you fix bugs before writing new code?

- Integration risk: Later code may depend on this

- Hard to find & fix later: Forget "old" code meaning
- Bugs may reveal fundamental problems

Do you have up-to-date schedule?

- Keep expectation realistic:

Team, customer, stakeholders

- Prevent feature creep: Dont take on anything without checking schedule first
7

Do you have a spec?

- Conceptual integrity for your project

- You know what you are trying to build, so are your teammates & customer

- You build right thing: Pieces fit together

- Design stage: Fix problem more easily

- Undocumented code has no commercial value

Netscape Navigator
8

Do you have quiet working conditions?

Do you use best tools money can buy?

10

Do you have testers as part of the team?

11

Do you have interview candidates write code?

12

Do you do hallway usability testing

Hallway usability test:

Grab random person, force him to try using your code

Do this to 5 people: You will learn 95% of usability problem in your code

COMP 3111H Page 31