SOFTWARE QUALITY ASSURANCE (SW9121)

NAME REG NO DEPT SEMESTER

: JASPER J WESSLY : 2012246015 : M.TECH (IT) : II

REGRESSION TESTING : The purpose of regression testing is to confirm that a recent program or code change has not adversely affected existing features. Regression testing is nothing but full or partial selection of already executed test cases which are re-executed to ensure existing functionalities work fine. This testing is done to make sure that new code changes should not have side effects on the existing functionalities. It ensures that old code still works once the new code changes are done. Need of Regression Testing

Regression Testing is required when there is a Change in requirements and code is modified according to the requirement New feature is added to the software Defect fixing Performance issue fix

Regression Testing Techniques Software maintenance is an activity which includes enhancements, error corrections, optimization and deletion of existing features. These modifications may cause the system to work incorrectly. Therefore, Regression Testing becomes necessary. Regression Testing can be carried out using following techniques:

Retest All

This is one of the methods for regression testing in which all the tests in the existing test bucket or suite should be re-executed. This is very expensive as it requires huge time and resources. Regression Test Selection Instead of re-executing the entire test suite, it is better to select part of test suite to be run Test cases selected can be categorized as 1) Reusable Test Cases 2) Obsolete Test Cases. Re-usable Test cases can be used in succeeding regression cycles. Obsolete Test Cases cant be used in succeeding cycles. Prioritization of Test Cases Prioritize the test cases depending on business impact, critical & frequently used functionalities . Selection of test cases based on priority will greatly reduce the regression test suite. Selecting test cases for regression testing It was found from industry data that good number of the defects reported by customers were due to last minute bug fixes creating side effects and hence selecting the test case for regression testing is an art and not that easy. Effective Regression Tests can be done by selecting following test cases Test cases which have frequent defects Functionalities which are more visible to the users Test cases which verify core features of the product Test cases of Functionalities which has undergone more and recent changes All Integration Test Cases All Complex Test Cases Boundary value test cases Sample of Successful test cases Sample of Failure test cases Regression Testing Tools If your software undergoes frequent changes , regression testing costs will escalate. In such cases , Manual execution of test cases increases test execution time as well as costs. Automation of regression test cases is the smart choice in such cases. Extent of automation depends on the number of test cases that remain re-usable for successive regression cycles.

Following are most important tools used for both functional and regression testing: Quick Test Professional (QTP): HP Quick Test Professional is automated software designed to automate functional and regression test cases. It uses VbScript language for automation. It is a Data driven, Keyword based tool. Rational Functional Tester (RFT): IBMs rational functional tester is a java tool used to automate the test cases of software applications. This is primarily used for automating regression test cases and it also integrates with Rational Test Manager. Selenium: This is an open source tool used for automating web applications. Selenium can be used for browser based regression testing. Regression Testing and Configuration Management Configuration Management during Regression Testing becomes imperative in Agile Environments where code is being continuously modified. To ensure effective regression tests , observe the following : Code being regression tested should be under a configuration management tool No changes must be allowed to code , during the regression test phase. Regression test code must be kept immune to developer changes. The database used for regression testing must be isolated . No database changes must be allowed Difference between Re-testing and regression testing: Retesting means testing the functionality or bug again to ensure the code is fixed. If it is not fixed, defect needs to be re-opened. If fixed, defect is closed. Regression testing means testing your software application when it undergoes a code change to ensure that the new code has not affected other parts of the software.

Challenges in Regression Testing:

Following are the major testing problems for doing regression testing: With successive regression runs, test suites become fairly large. Due to time and budget constraints, the entire regression test suite cannot be executed Minimizing test suite while achieving maximum test coverage remains a challenge Determination of frequency of Regression Tests , i.e., after every modification or every build update or after a bunch of bug fixes, is a challenge.

Conclusion: An effective regression strategy, save organizations both time and money. As per one of the case study in banking domain, regression saves upto 60% time in bug fixes(which would have been caught by regression tests) and 40% in money.

ADHOC TESTING Adhoc testing is an informal testing type with an aim to break the system. This testing is usually an unplanned activity . It does not follow any test design techniques to create test cases. In fact is does not create test cases altogether! This testing is primarily performed if the knowledge of testers in the system under test is very high. Testers randomly test the application without any test cases or any business requirement document.

Adhoc Testing does not follow any structured way of testing and it is randomly done on any part of application. Main aim of this testing is to find defects by random checking. Adhoc testing can be achieved with the testing technique called Error Guessing. Error guessing can be done by the people having enough experience on the system to geuss the most likely source of errors. This testing requires no documentation/ planning /process to be followed. Since this testing aims at finding defects through random approach, without any documentation, defects will not be mapped to test cases. Hence, sometimes, it is very difficult to reproduce the defects as there are no test-steps or requirements mapped to it. When execute Adhoc Testing? Adhoc testing can be performed when there is limited time to do elaborative testing. Usually adhoc testing is performed after the formal test execution. And if time permits, adhoc testing can be done on the system).Adhoc testing will be effective only if the tester is knowledgeable of the System Under Test. Types of adhoc testing There are different types of Adhoc testing and they are listed as below: Buddy Testing Two buddies mutually work on identifying defects in the same module. Mostly one buddy will be from development team and another person will be from testing team. Buddy testing helps the testers develop better test cases and development team can also make design changes early. This testing usually happens after unit testing completion. Two testers are assigned modules, share ideas and work on the same machines to find defects. One person can execute the tests and another person can take notes on the findings. Roles of the persons can be a tester and scriber during testing. Comparison Buddy and Pair Testing

Pair testing

Buddy testing is combination of unit and system testing together with developers and testers but Pair testing is done only with the testers with different knowledge levels.(Experienced and non-experienced to share their ideas and views) Monkey Testing Randomly test the product or application without test cases with a goal to break the system.

Best practices of Adhoc testing Following bestpratices can ensure effective Adhoc Testing. Good business knowledge Testers should have good knowledge of the business and clear understanding of the requirements- Detailed knowledge of the end to end business process will help find defects easily. Experienced testers find more defects as they are better at error guessing. Test Key Modules Key business modules should be identified and targeted for adhoc testing.. Business critical modules should be tested first to gain confidence on the quality of the system. Record Defects All defects need to be recorded or written in a notepad . Defects must be assigned to developers for fixing. For each valid defect ,corresponding test cases must be written & must be added to planned test cases. These defect findings should be made as lesson learnt and these should be reflected in our next system while we are planning for test cases. Conclusion: The advantage of Adhoc testing is to check for the completeness of testing and find more defects then planned testing. The defect catching test cases are added as additional test cases to the planned test cases. Adhoc Testing saves lot of time as it doesnt require elaborate test planning , documentation and test case design.

WEB TESTING

What is Web Testing? Web Testing in simple terms is checking your web application for potential bugs before its made live or before code is moved into the production environment. During this stage issues such as that of web application security, the functioning of the site, its access to handicapped as well as regular users and its ability to handle traffic is checked. Web Application Testing Checklist: Some or all of the following testing types may be performed depending on your web testing requirements. 1. Functionality Testing : This is used to check of your product is as per the specifications you intended for it as well as the functional requirements you charted out for it in your developmental documentation.Testing Activities Included: Test all links in your webpages are working correctly and make sure there are no broken links. Links to be checked will include Outgoing links Internal links Anchor Links MailTo Links Test Forms are working as expected. This will include Scripting checks on the form are working as expected. For example- if a user does not fill a mandatory field in a form a error message is shown. Check default values are being populated Once submitted , the data in the forms is submitted to a live database or is linked to an working email address Forms are optimally formatted for better readability

Test Cookies are working as expected. Cookies are small files used by websites to primarily remember active user sessions so you do not to log in every time you visit a website. Cookie Testing will include Testing cookies (sessions) are deleted either when cache is cleared or when they reach their expiry. Delete cookies (sessions) and test that login credentials are asked for when you next visit the site.

Test HTML and CSS to ensure that search engines can crawl your site easily. This will include Checking for Syntax Errors Readable Color Schemas Standard Compliance.Ensure standards such W3C, OASIS, IETF, ISO, ECMA, or WS-I are followed. Test business workflow- This will include Testing your end - to - end workflow/ business scenarios which takes the user through a series of webpage's to complete. Test negative scenarios as well , such that when a user executes an unexpected step , appropriate error message or help is shown in your web application.

Tools that can be used: QTP , IBM Rational , Selenium

2. Usability testing: Usability testing has now become a vital part of any web based project. It can carried out by testers like you or a small focus group similar to the target audience of the web application. Test the site Navigation: Menus , buttons or Links to different pages on your site should be easily visible and consistent on all webpages Test the Content: Content should be legible with no spelling or grammatical errors. Images if present should contain and "alt" text

Tools that can be used: Chalkmark, Clicktale, Clixpy and Feedback Army 3.Interface Testing: Three areas to be tested here are - Application , Web and Database Server Application: Test requests are sent correctly to the Database and output at the client side is displayed correctly. Errors if any must be caught by the application and must be only shown to the administrator and not the end user. Web Server: Test Web server is handling all application requests without any service denial. Database Server: Make sure queries sent to the database give expected results. Test system response when connection between the three layers (Application, Web and Database) can not be established and appropriate message is shown to the end user.

Tools that can be used: AlertFox,Ranorex 4.Database Testing: Database is one critical component of your web application and stress must be laid to test it thoroughly. Testing activities will include Test if any errors are shown while executing queries Data Integrity is maintained while creating , updating or deleting data in database. Check response time of queries and fine tune them if necessary. Test data retrieved from your database is shown accurately in your web application

Tools that can be used: QTP 5. Compatibility testing. Compatibility tests ensures that your web application displays correctly across different devices. This would includeBrowser Compatibility Test: Same website in different browsers will display differently. You need to test if your web application is being displayed correctly across browsers , javascript , AJAX and authentication is working fine. You may also check for Mobile Browser Compatibility. The rendering of web elements like buttons , text fields etc changes with change in Operating System. Make sure your website works fine for various combination of Operating systems such as Windows , Linux , Mac and Browsers such as Firefox , Internet Explorer , Safari etc. Tools that can be used: NetMechanic 6.Performance Testing: This will ensure your site works under all loads. Testing activities will include but not limited to Website application response times at different connection speeds Load test your web application to determine its behavior under normal and peak loads Stress test your web site to determine its break point when pushed to beyond normal loads at peak time. Test if a crash occurs due to peak load , how does the site recover from such an event Make sure optimization techniques like gzip compression , browser and server side cache enabled to reduce load times Tools that can be used: Loadrunner, JMeter 7. Security testing: Security testing is vital for e-commerce website that store sensitive customer information like credit cards.Testing Activities will include-

Test unauthorized access to secure pages should not be permitted Restricted files should not be downloadable without appropriate access Check sessions are automatically killed after prolonged user inactivity On use of SSL certificates , website should re-direct to encrypted SSL pages.

Tools that can be used: Babel Enterprise, BFBTester and CROSS 8.Crowd Testing: You will select a large number of people (crowd) to execute tests which otherwise would have been executed a select group of people in the company. Crowdsourced testing is an interesting and upcoming concept and helps unravel many a unnoticed defects. Tools that can be used: People like you and me . And yes , loads of them!

This concludes almost all testing types applicable to your web application. As a Web-tester its important to note that web testing is quite an arduous process and you are bound to come across many obstacles. One of the major problems you will face is of course deadline pressure. Everything is always needed yesterday! The number of times the code will need changing is also taxing. Make sure you plan your work and know clearly what is expected of you. Its best define all the tasks involved in your web testing and then create a work chart for accurate estimates and planning.

BLACK BOX TESTING what is Black Box Testing? Black box testing is a software testing techniques in which functionality of the software under test (SUT) is tested without looking at the internal code structure, implementation details and knowledge of internal paths of the software.This type of testing is based entirely on the software requirements and specifications. In Black Box Testing we just focus on inputs and output of the software system without bothering about internal knowledge of the software program.

The above Black Box can be any software system you want to test. For example : an operating system like Windows, a website like Google ,a database like Oracle or even your own custom application. Under Black Box Testing , you can test these applications by just focusing on the inputs and outputs without knowing their internal code implementation. Black box testing - Steps Here are the generic steps followed to carry out any type of Black Box Testing. Initially requirements and specifications of the system are examined. Tester chooses valid inputs (positive test scenario) to check whether SUT processes them correctly . Also some invalid inputs (negative test scenario) are chosen to verify that the SUT is able to detect them. Tester determines expected outputs for all those inputs. Software tester constructs test cases with the selected inputs. The test cases are executed. Software tester compares the actual outputs with the expected outputs. Defects if any are fixed and re-tested. Types of Black Box Testing There are many types of Black Box Testing but following are the prominent ones Functional testing This black box testing type is related to functional requirements of a system; it is done by software testers. Non-functional testing This type of black box testing is not related to testing of a specific functionality , but non-functional requirements such as performance, scalability, usability. Regression testing Regression testing is done after code fixes , upgrades or any other system maintenance to check the new code has not affected the existing code.

Tools used for Black Box Testing: Tools used for Black box testing largely depends on the type of black box testing your are doing. For Functional/ Regression Tests you can use - QTP For Non-Functional Tests you can use - Loadrunner

Black box testing strategy: Following are the prominent test strategy amongst the many used in Black box Testing Equivalence Class Testing: It is used to minimize the number of possible test cases to an optimum level while maintains reasonable test coverage. Boundary Value Testing: Boundary value testing is focused on the values at boundaries. This technique determines whether a certain range of values are acceptable by the system or not.It is very useful in reducing the number of test cases. It is mostly suitable for the systems where input is within certain ranges.

Decision Table Testing: A decision table puts causes and their effects in a matrix. There is unique combination in each column.

Comparison of Black Box and White Box Testing:

While White Box Testing (Unit Testing) validates internal structure and working of your software code, the main focus of black box testing is on the validation of your functional requirements. To conduct White Box Testing , knowledge of underlying programming language is essential. Current day software systems use a variety of programming languages and technologies and its not possible to know all of them. Black box testing gives abstraction from code and focuses testing effort on the software system behaviour. Also software systems are not developed in a single chunk but development is broken down in different modules. Black box testing facilitates testing communication amongst modules (Integration Testing) . In case you push code fixes in your live software system , a complete system check (black box regression tests) becomes essential. Though White box testing has its own merits and help detect many internal errors which may degrade system performance Black Box Testing and Software Development Life Cycle (SDLC) Black box testing has its own life cycle called Software Test Life Cycle (STLC) and it is relative to every stage of Software Development Life Cycle. Requirement This is the initial stage of SDLC and in this stage requirement is gathered. Software testers also take part in this stage. Test Planning & Analysis Testing Types applicable to the project are determined. A Test Plan is created which determines possible project risks and their mitigation. Design In this stage Test cases/scripts are created on the basis of software requirement documents Test Execution- In this stage Test Cases prepared are executed. Bugs if any are fixed and re-tested. Equivalence partitioning & Boundary value analysis:

 We have already learned that exhaustive testing is not possible due to time and budget considerations We need certain testing techniques to select a few test cases out of the many with most likelihood of finding a defect Lets look into Equivalence Partitioning & Boundary Value Analysis testing techniques . Equivalence Partitioning - is a black box technique which can be applied to all levels of testing like unit , integration , system etc. A black box technique where the code is not visible to the tester. In Equivalence Partitioning , you divide set of test conditions into partitions that can be considered the same. To understand this better , lets consider the behavior of tickets in the Flight reservation application , while booking a new flight. Ticket values 1 to 10 are considered valid & ticket is booked. Values 11 to 99 are considered invalid and a error message "Only ten tickets may be ordered at one time" is shown On entering values 100 and above , the ticket # number defaults to a two digit number On entering values 0 and below , the ticket # defaults to 1 We can not test all the possible values , because if done , number of test cases will be more than 100 .To address this problem we use equivalence partitioning where we divide the possible values of tickets into groups or sets where the system behavior can be considered the same. The divided sets are called Equivalence Partitions or Equivalence Classes. Then we pick only one value from each partition for testing. The hypothesis behind this technique is that if one condition/value in a partition passes all others will also pass. Likewise , if one condition in a partition fails , all other conditions in that partition will fail. In Boundary Value Analysis , you test boundaries between equivalence partitions In our earlier example instead of checking, one value from each partition you will check the values at the partitions like 0,1,10,11 and so on As you may observe, you test values at both valid and invalid boundaries Boundary Value Analysis is also called range checking. Equivalence partitioning and boundary value analysis are closely related and can be used together at all levels of testing.

TESTING METHODOLOGY What is a Software Methodology? A methodology is a package of methods. In simple words , its a bundle of practical ideas and proven practises which help in efficient software project management.

Software Test Methodology : The important Software Test Methodologies are discussed below Waterfall model

What is it? In the waterfall model ,software development progress through various phases like Requirements Analysis , Design etc - sequentially. In this model, next phase begins only when the earlier phase is completed. What Is The Testing Approach? The first phase in waterfall model is the requirements phase in which all the project requirements are completely defined before starting the testing. During this phase , the test team brainstorms the scope of testing , test strategy and drafts a detailed test plan. Only once the design of software is complete, the team will move on to execution of the test cases to ensure that the developed software behaves as it expected. In this methodology, the testing team proceeds to the next phase only when the previous phase is completed. Advantages This model is very simple to plan and manage. Hence, projects where requirements are clearly defined and stated beforehand can be easily tested using waterfall model.

Disadvantages In the waterfall model , you can begin with the next phase only once the previous phase is completed. Hence , this model cannot accommodate unplanned events and uncertainty. This methodology is not suitable for projects where the requirements change frequently. Iterative development

What is it? In this model , a big project is divided into small parts , and each part is subjected to multiple iterations of the waterfall model. At the end of iteration, a new module is developed or an existing module is enhanced. This module is integrated into the software architecture and the entire system is tested all together What is the testing Approach? As soon as iteration is completed, the entire system is subjected to testing. Feedback from testing is immediately available and is incorporated in next cycle. The testing time required in successive iteration can be reduced based on the experience gained from past iterations. Advantages The main advantage of iterative development is the test feedback is immediately available at the end of each cycle. Disadvantages This model increases communication overheads significantly since at the end of each cycle, feedback about deliverables , effort etc must be given. Agile methodology

What is it ? Traditional software development methodologies work on the premise that software requirements remain constant throughout the project. But with increase in complexity , the requirements undergo numerous changes and continuously evolve. At times, the customer himself is not sure what he wants. Though iterative model addresses this issue, its still based on the waterfall model. In Agile methodology , software is developed in incremental, rapid cycles. Interactions amongst customers, developers and client are emphasized rather than processes and tools. Agile methodology focuses on responding to change rather than extensive planning. What Is The Testing Approach? Incremental testing is used in agile development methods and hence, every release of the project is tested thoroughly. This ensures that any bugs in the system are fixed before the next release. Advantages It is possible to make changes in the project at any time to comply with the requirements. This incremental testing minimizes risks.

Disadvantages Constant client interaction means added time pressure on all stake holders including the client themselves , software development and test teams . Extreme programming

What is it? Extreme programming is a type of agile methodology which believes in short development cycles. A project is divided into simple engineering tasks. Programmers code a simple piece of software and get back to customer for feedback. Review points from the customer are incorporated and the developers proceed with the next task. In extreme programming developers usually work in pairs. Extreme Programming is used in places where customer requirements are constantly changing. What Is The Testing Approach? Extreme programming follows a Test-driven development which is described as follows 1. Add a test case to the test suite to verify the new functionality which is yet to be developed 2. Run the all tests and obviously the new test case added must fail since the functionality is not coded yet 3. Write some code to implement the feature/functionality

4. Run the test suite again .This time , the new test case should pass since the functionally has been coded Advantages Customers having a vague software design in mind could use extreme programming Continuous testing and continuous integration of small releases ensure software code is delivered is of high quality Disadvantages Meetings amongst the software development team and clients add to time requirements. Which Software Methodology to choose ? There are tons of methodologies available for software development and its corresponding testing. Each methodology is designed for a specific purpose and has its relative merits and demerits. Selection of a particular methodologies depends on many factors such as the nature of project, client requirement, project schedule , etc. From a testing perspective, some methodologies push for testing input early in the development life cycle , while others wait until a working model of the system is ready. How to setup software testing methodologies? Software testing methodologies should not be setup just for the sake of testing software code. The big picture should be considered and the prime goal of the project should be satisfied by the testing methodology. Scheduling Realistic scheduling is the key to the implementation of successful testing methodology and the schedule should meet the needs of every member of the team. Defined deliverables In order to keep all the members of the team on the same page, well defined deliverables should be provided. The deliverables should contain direct content without any ambiguity. Test approach Once scheduling is complete and defined deliverables are made available, the testing team should be able to formulate the right test approach. Definition documents and developer meetings should indicate the team about the best test approach that can be used for the project. Reporting Transparent reporting is very difficult to achieve, but this step determines the effectiveness of the testing approach used in the project.