LoadRunner interview questions

By admin | May 2, 2005

1.

What is load testing? - Load testing is to test that if the application works fine with the loads that result from large number of simultaneous users, transactions and to determine weather it can handle peak usage periods. What is Performance testing? - Timing for both read and update transactions should be gathered to determine whether system functions are being performed in an acceptable timeframe. This should be done standalone and then in a multi user environment to determine the effect of multiple transactions on the

2.

3. 4.

timing of a single transaction. Did u use LoadRunner? What version? - Yes. Version 7.2. Explain the Load testing process? Step 1: Planning the test. Here, we develop a clearly defined test plan to ensure the test scenarios we develop will accomplish load-testing objectives. Step 2: Creating Vusers. Here, we create Vuser scripts that contain tasks performed by each Vuser, tasks performed by Vusers as a whole, and tasks measured as transactions. Step 3: Creating the scenario. A scenario describes the events that occur during a testing session. It includes a list of machines, scripts, and Vusers that run during the scenario. We create scenarios using LoadRunner Controller. We can create manual scenarios as well as goal-oriented scenarios. In manual scenarios, we define the number of Vusers, the load generator machines, and percentage of Vusers to be assigned to each script. For web tests, we may create a goal-oriented scenario where we define the goal that our test has to achieve. LoadRunner automatically builds a scenario for us. Step 4: Running the scenario. We emulate load on the server by instructing multiple Vusers to perform tasks simultaneously. Before the testing, we set the scenario configuration and scheduling. We can run the entire scenario, Vuser groups, or individual Vusers. Step 5: Monitoring the scenario. We monitor scenario execution using the LoadRunner online runtime, transaction, system resource, Web resource, Web server resource, Web application server resource, database server resource, network delay, streaming media resource, firewall server resource, ERP server resource, and Java performance monitors. Step 6: Analyzing test results. During scenario execution, LoadRunner records the performance of the application under different loads. We use LoadRunners graphs and reports to analyze the applications performance.

5.

When do you do load and performance Testing? - We perform load testing once we are done with interface (GUI) testing. Modern system architectures are large and complex. Whereas single user testing primarily on functionality and user interface of a system component, application testing focuses on performance and reliability of an entire system. For example, a typical application-testing scenario might depict 1000 users logging in simultaneously to a system. This gives rise to issues such as what is the response time of the system, does it crash, will it go with different software applications and platforms, can it hold so

6.

many hundreds and thousands of users, etc. This is when we set do load and performance testing. What are the components of LoadRunner? - The components of LoadRunner are The Virtual User Generator, Controller, and the Agent process, LoadRunner Analysis and Monitoring, LoadRunner Books Online.

7.

What Component of LoadRunner would you use to record a Script? - The Virtual User Generator (VuGen) component is used to record a script. It enables you to develop Vuser scripts for a variety of application types and communication protocols. What Component of LoadRunner would you use to play Back the script in multi user mode? The Controller component is used to playback the script in multi-user mode. This is done during a scenario run where a vuser script is executed by a number of vusers in a group.

8.

9.

What is a rendezvous point? - You insert rendezvous points into Vuser scripts to emulate heavy user load on the server. Rendezvous points instruct Vusers to wait during test execution for multiple Vusers to arrive at a certain point, in order that they may simultaneously perform a task. For example, to emulate peak load on the bank server, you can insert a rendezvous point instructing 100 Vusers to deposit cash into their accounts at the same time.

10. What is a scenario? - A scenario defines the events that occur during each testing session. For example, a scenario defines and controls the number of users to emulate, the actions to be performed, and the machines on which the virtual users run their emulations. 11. Explain the recording mode for web Vuser script? - We use VuGen to develop a Vuser script by recording a user performing typical business processes on a client application. VuGen creates the script by recording the activity between the client and the server. For example, in web based applications, VuGen monitors the client end of the database and traces all the requests sent to, and received from, the database server. We use VuGen to: Monitor the communication between the application and the server; Generate the required function calls; and Insert the generated function calls into a Vuser script. 12. Why do you create parameters? - Parameters are like script variables. They are used to vary input to the server and to emulate real users. Different sets of data are sent to the server each time the script is run. Better simulate the usage model for more accurate testing from the Controller; one script can emulate many different users on the system. 13. What is correlation? Explain the difference between automatic correlation and manual correlation? - Correlation is used to obtain data which are unique for each run of the script and which are generated by nested queries. Correlation provides the value to avoid errors arising out of duplicate values and also optimizing the code (to avoid nested queries). Automatic correlation is where we set some rules for correlation. It can be application server specific. Here values are replaced by data which are created by these rules. In manual correlation, the value we want to correlate is scanned and create correlation is used to correlate. 14. How do you find out where correlation is required? Give few examples from your projects? Two ways: First we can scan for correlations, and see the list of values which can be correlated. From this we can pick a value to be correlated. Secondly, we can record two scripts and compare them. We can look up the difference file to see for the values which needed to be correlated. In my project, there was a unique id developed for each customer, it was nothing but Insurance Number, it was generated automatically and it was sequential and this value was unique. I had to correlate this value, in order to avoid errors while running my script. I did using scan for correlation. 15. Where do you set automatic correlation options? - Automatic correlation fromweb point of view can be set in recording options and correlation tab. Here we can enable correlation for the entire script and choose either issue online messages or offline actions, where we can define rules for that correlation. Automatic correlation for database can be done using show output window and scan for correlation and picking the correlate query tab and choose which query value we want to correlate. If we know the specific value to be correlated, we just do create correlation for the value and specify how the value to be created. 16. What is a function to capture dynamic values in the web Vuser script? - Web_reg_save_param function saves dynamic data information to a parameter. 17. When do you disable log in Virtual User Generator, When do you choose standard and extended logs? - Once we debug our script and verify that it is functional, we can enable logging for errors only. When we add a script to a scenario, logging is automatically disabled. Standard Log Option: When you select Standard log, it creates a standard log of functions and messages sent during script execution to use for debugging. Disable this option for large load testing scenarios. When you copy a script to a scenario, logging is automatically disabled Extended Log Option: Select extended log to create an extended log, including warnings and other messages. Disable this option for large

load testing scenarios. When you copy a script to a scenario, logging is automatically disabled. We can specify which additional information should be added to the extended log using the Extended log options. 18. How do you debug a LoadRunner script? - VuGen contains two options to help debug Vuser scriptsthe Run Step by Step command and breakpoints. The Debug settings in the Options dialog box allow us to determine the extent of the trace to be performed during scenario execution. The debug information is written to the Output window. We can manually set the message class within your script using the lr_set_debug_message function. This is useful if we want to receive debug information about a small section of the script only. 19. How do you write user defined functions in LR? Give me few functions you wrote in your previous project? - Before we create the User Defined functions we need to create the external library (DLL) with the function. We add this library to VuGen bin directory. Once the library is added then we assign user defined function as a parameter. The function should have the following format: __declspec (dllexport) char* <function name>(char*, char*)Examples of user defined functions are as follows:GetVersion, GetCurrentTime, GetPltform are some of the user defined functions used in my earlier project. 20. What are the changes you can make in run-time settings? - The Run Time Settings that we make are: a) Pacing - It has iteration count. b) Log - Under this we have Disable Logging Standard Log and c) Extended Think Time - In think time we have two options like Ignore think time and Replay think time. d) General - Under general tab we can set the vusers as process or as multithreading and whether each step as a transaction. 21. Where do you set Iteration for Vuser testing? - We set Iterations in the Run Time Settings of the VuGen. The navigation for this is Run time settings, Pacing tab, set number of iterations. 22. How do you perform functional testing under load? - Functionality under load can be tested by running several Vusers concurrently. By increasing the amount of Vusers, we can determine how much load the server can sustain. 23. What is Ramp up? How do you set this? - This option is used to gradually increase the amount of Vusers/load on the server. An initial value is set and a value to wait between intervals can be specified. To set Ramp Up, go to Scenario Scheduling Options 24. What is the advantage of running the Vuser as thread? - VuGen provides the facility to use multithreading. This enables more Vusers to be run per generator. If the Vuser is run as a process, the same driver program is loaded into memory for each Vuser, thus taking up a large amount of memory. This limits the number of Vusers that can be run on a single generator. If the Vuser is run as a thread, only one instance of the driver program is loaded into memory for the given number of Vusers (say 100). Each thread shares the memory of the parent driver program, thus enabling more Vusers to be run per generator. 25. If you want to stop the execution of your script on error, how do you do that? - The lr_abort function aborts the execution of a Vuser script. It instructs the Vuser to stop executing the Actions section, execute the vuser_end section and end the execution. This function is useful when you need to manually abort a script execution as a result of a specific error condition. When you end a script using this function, the Vuser is assigned the status "Stopped". For this to take effect, we have to first uncheck the Continue on error option in Run-Time Settings. 26. What is the relation between Response Time and Throughput? - The Throughput graph shows the amount of data in bytes that the Vusers received from the server in a second. When we compare this with the transaction response time, we will notice that as throughput decreased, the response time also decreased. Similarly, the peak throughput and highest response time would occur approximately at the same time. 27. Explain the Configuration of your systems? - The configuration of our systems refers to that of the client machines on which we run the Vusers. The configuration of any client machine includes its hardware

settings, memory, operating system, software applications, development tools, etc. This system component configuration should match with the overall system configuration that would include the network infrastructure, the web server, the database server, and any other components that go with this larger system so as to achieve the load testing objectives. 28. How do you identify the performance bottlenecks? - Performance Bottlenecks can be detected by using monitors. These monitors might be application server monitors, web server monitors, database server monitors and network monitors. They help in finding out the troubled area in our scenario which causes increased response time. The measurements made are usually performance response time, throughput, hits/sec, network delay graphs, etc. 29. If web server, database and Network are all fine where could be the problem?- The problem could be in the system itself or in the application server or in the code written for the application. 30. How did you find web server related issues? - Using Web resource monitors we can find the performance of web servers. Using these monitors we can analyze throughput on the web server, number of hits per second that occurred during scenario, the number of http responses per second, the number of downloaded pages per second. 31. How did you find database related issues? - By running Database monitor and help of Data Resource Graph we can find database related issues. E.g. You can specify the resource you want to measure on before running the controller and than you can see database related issues 32. Explain all the web recording options? 33. What is the difference between Overlay graph and Correlate graph? -Overlay Graph: It overlay the content of two graphs that shares a common x-axis. Left Y-axis on the merged graph shows the current graphs value & Right Y-axis show the value of Y-axis of the graph that was merged. Correlate Graph:Plot the Y-axis of two graphs against each other. The active graphs Yaxis becomes X-axis of merged graph. Y-axis of the graph that was merged becomes merged graphs Y-axis. 34. How did you plan the Load? What are the Criteria? - Load test is planned to decide the number of users, what kind of machines we are going to use and from where they are run. It is based on 2 important documents, Task Distribution Diagram and Transaction profile. Task Distribution Diagram gives us the information on number of users for a particular transaction and the time of the load. The peak usage and offusage are decided from this Diagram. Transaction profile gives us the information about the transactions name and their priority levels with regard to the scenario we are deciding. 35. What does vuser_init action contain? - Vuser_init action contains procedures to login to a server. 36. What does vuser_end action contain? - Vuser_end section contains log off procedures. 37. What is think time? How do you change the threshold? - Think time is the time that a real user waits between actions. Example: When a user receives data from a server, the user may wait several seconds to review the data before responding. This delay is known as the think time. Changing the Threshold: Threshold level is the level below which the recorded think time will be ignored. The default value is five (5) seconds. We can change the think time threshold in the Recording options of the Vugen. 38. What is the difference between standard log and extended log? - The standard log sends a subset of functions and messages sent during script execution to a log. The subset depends on the Vuser type Extended log sends a detailed script execution messages to the output log. This is mainly used during debugging when we want information about: Parameter substitution. Data returned by the server. Advanced trace. 39. Explain the following functions: - lr_debug_message - The lr_debug_message function sends a debug message to the output log when the specified message class is set. lr_output_message - The lr_output_message function sends notifications to the Controller Output window and the Vuser log file. lr_error_message - The lr_error_message function sends an error message to the LoadRunner

Output window.lrd_stmt - The lrd_stmt function associates a character string (usually a SQL statement) with a cursor. This function sets a SQL statement to be processed.lrd_fetch - The lrd_fetch function fetches the next row from the result set. 40. Throughput - If the throughput scales upward as time progresses and the number of Vusers increase, this indicates that the bandwidth is sufficient. If the graph were to remain relatively flat as the number of Vusers increased, it would be reasonable to conclude that the bandwidth is constraining the volume of data delivered. 41. Types of Goals in Goal-Oriented Scenario - Load Runner provides you with five different types of goals in a goal oriented scenario: The number of concurrent Vusers

The number of hits per second The number of transactions per second The number of pages per minute The transaction response time that you want your scenario

42. Analysis Scenario (Bottlenecks): In Running Vuser graph correlated with the response time graph you can see that as the number of Vusers increases, the average response time of the check itinerary transaction very gradually increases. In other words, the average response time steadily increases as the load increases. At 56 Vusers, there is a sudden, sharp increase in the average response time. We say that the test broke the server. That is the mean time before failure (MTBF). The response time clearly began to degrade when there were more than 56 Vusers running simultaneously. 43. What is correlation? Explain the difference between automatic correlation and manual correlation? - Correlation is used to obtain data which are unique for each run of the script and which are generated by nested queries. Correlation provides the value to avoid errors arising out of duplicate values and also optimizing the code (to avoid nested queries). Automatic correlation is where we set some rules for correlation. It can be application server specific. Here values are replaced by data which are created by these rules. In manual correlation, the value we want to correlate is scanned and create correlation is used to correlate. 44. Where do you set automatic correlation options? - Automatic correlation from web point of view, can be set in recording options and correlation tab. Here we can enable correlation for the entire script and choose either issue online messages or offline actions, where we can define rules for that correlation. Automatic correlation for database, can be done using show output window and scan for correlation and picking the correlate query tab and choose which query value we want to correlate. If we know the specific value to be correlated, we just do create correlation for the value and specify how the value to be created. 45. What is a function to capture dynamic values in the web vuser script? - Web_reg_save_param function saves dynamic data information to a parameter.

2. What is a protocol?
Sat, 06/18/2011 - 07:52 Srinivasa Bittla

Set of rules defined for transmitting the information between the two or multiple systems is called as a protocol. Protocol is used for recording the test scripts in most of performance and load testing tools. So, as a performance / load test engineer, you need to know the protocols used for accessing the application, so that you can choose the right protocol for recording the script. You can get the protocol related information from the architects, not from the developers.

3. Why load testing tools requires protocol information for capturing the test script?
Sat, 06/18/2011 - 07:52 Srinivasa Bittla

Unlike functional testing tools, load testing tools are not intended for capturing the GUI operations. Instead, they be will used for request generation on the servers. The request needs to be posted on the servers with optimum resource utilization on the client side, then only multiple user simulation will happen with low infrastructure costs. To post the request on the servers, you do not have to use GUI of application. Instead, you can use the other modes of request generation. This is possible through posting the request directly through protocol. That's the reason we capture traffic related to the requests which goes by a protocol using the load testing tools. If you use GUI (like browser), multiple instances of the browser should be running for hundreds and thousands of users. Simulation of hundreds and thousands of users is not possible by a single machine if you use browser. Whereas, if you are using request simulation by using a thread (virtual user) with a protocol, this is pretty much possible. The main intention of load testing is to generate the load on the servers and see the behavior of the servers. Underlying assumption is that clients are adequately configured with enough hardware and software. Since the client is used by single user, mostly there wont be any issues. Whereas the servers will be serving to the multiple users, most of the issues will come from the servers. Based on the type of application, and its communication we need to choose the protocol. For example, for any web / HTML based applications, we need to choose HTTP protocol for capturing the traffic as part of test script. Similarly if you want to capture, FTP (file transfer) traffic you need to choose FTP protocol. Protocol is used for recording the test scripts in most of performance and load testing tools. So, as a performance / load test engineer, you need to know the protocols used for accessing the application, so that you can choose the right protocol for recording the script. You can get the protocol related information from the architects, not from the developers.

4. What is concurrent user hits in load testing?

Sat, 06/18/2011 - 07:53 Srinivasa Bittla

When the multiple users are running the load test, to hit a common step in the application without any millisecond difference is called a concurrent user hit. The concurrency point needs to be added as a one of the step for the virtual users. So, that the virtual users will wait at the concurrency point, if they reach early. Once all the users reached to the concurrency point, they start hitting the requests.

6. What is performance tuning?

Sat, 06/18/2011 - 07:54 Srinivasa Bittla

A mechanism followed to improve the systems performance is called performance tuning. To improve the systems performance there are two types of tuning followed. 1) Hardware tuning 2) Software tuning. Hardware tuning: Optimizing, adding or replacing the systems hardware components and infrastructure level improvements to improve the systems performance is called hardware tuning. Software tuning: Identifying the software level bottlenecks by profiling the code, database etc. Fine tuning or modifying the software to fix the bottlenecks is called software tuning.

7. What is hardware tuning in load testing?

Sat, 06/18/2011 - 07:54 Srinivasa Bittla

If any hardware specific bottleneck is identified during the load test, the process of eliminating the bottleneck is called as hardware tuning. For example, it is observed that CPU usage of Application server is going very high (Avg Processor Time >95%) during the load test, and all other server resources are in acceptable limits. After profiling of the code, it is observed that there is no issue found at software level. In this case, the bottleneck is the CPU usage of the Application Server. To support the existing users or to support more users, we need to replace the current CPU with the high end CPU or keep few more application servers in cluster environment. This is the way hardware tuning will be done. Improving the systems infrastructure to support the futuristic load, hardware tuning/ improvements need to be done. This is also called as one way of capacity planning. Hardware tuning: Optimizing, adding or replacing the systems hardware components and infrastructure level improvements to improve the systems performance is called hardware tuning.

8. What is software tuning in load testing?

Sat, 06/18/2011 - 07:54 Srinivasa Bittla

The process of fixing the software bottlenecks is called software tuning. For example, as a performance test engineer you identified high CPU utilization or memory utilization on the application server. Then you need to use a code profiling tool to identify the issue and report the issue to the performance tuning team. To fix the identified bottleneck the performance developers, will use the best possible code to reduce the high CPU / memory utilization. Something like thread prioritization or String Buffer instead of String something like that. Software tuning: Identifying the software level bottlenecks by profiling the code, database etc. Fine tuning or modifying the software to fix the bottlenecks is called software tuning.

9. Who will do the tuning after load testing?

Sat, 06/18/2011 - 07:55 Srinivasa Bittla

Software tuning will be done by the developers, who are responsible for improving the systems performance. Performance Testers can give recommendations / suggestions to the development team for tuning software. But, tuning will be done by only developers.

10. What is profiling?

Sat, 06/18/2011 - 07:55 Srinivasa Bittla

A process of pin pointing a performance bottleneck at the minute level is called profiling. Profiling will be done by the performance testers or developers (Performance engineering teams). Profiling will be done in any layer (presentation, business or data base layers) of application under test. If we need to do the profiling at the application server (business layer), we may need to use application server performance profiling tools. When the application server is been profiled, you identify the code level issues such as high CPU or memory intensive APIs. If you are profiling the database using the database profiling tools, you can identify the issues such as the number of SQLs executed, high cost queries, full table scan queries etc.

11. How to do application server profiling?

Sat, 06/18/2011 - 07:56 Srinivasa Bittla

When the Application Servers needs to be profiled. You need to identify the profiling tools, which is compatible with the respective application server. For example, if you want to profile the application which is deployed on weblogic application server. You can use the tools like Optimize It, JProbe, JProfiler. Before you start profiling, you need to configure the application server with the respective code profiling tool jars and restart the application server. And start the profiling tools and start

accessing the application from the related UIs. Then you can start seeing the APIs in the profiling tool in a sorted order, whichever is taking high CPU or memory utilization. Identify the problematic APIs and report them to the development team, to get it runed. Profiling will be done in any layer (presentation, business or data base layers) of application under test. If we need to do the profiling at the application server (business layer), we may need to use application server performance profiling tools. When the application server is been profiled, you identify the code level issues such as high CPU or memory intensive APIs.

12. How to do database profiling?

Sat, 06/18/2011 - 07:56 Srinivasa Bittla

To identify the database specific bottlenecks, we do the database profiling. Choose the database profiling tool (for Ex, Quest Central, SQL Analyzer etc). Provide the necessary information (such as database machine, schema, admin user and password etc) to the database profiling tool. Start the database profiling tool. Flush the cache in the database profiling tool before start profiling. Execute the problematic scenario of your application and stop profiling. Now you can see the SQLs captured as part of database profiling tool. Analyze the information and report the issues. If you are profiling the database using the database profiling tools, you can identify the issues such as the number of SQLs executed, high cost queries, full table scan queries etc. Profiling will be done in any layer (presentation, business or data base layers) of application under test. If we need to do the profiling at the application server (business layer), we may need to use application server performance profiling tools. When the application server is been profiled, you identify the code level issues such as high CPU or memory intensive APIs.

13. How the network can be profiled?

Sat, 06/18/2011 - 07:56 Srinivasa Bittla

Network is profiled, using the network monitoring or profiling tools. Install the network monitoring tool within the network and start the network profiling tool. Network will be profiled to identify the issues related to bandwidth, high network traffic generated machine, and also identify the relevant traffic sources.

14. How to identify a memory leak?

Sat, 06/18/2011 - 07:57 Srinivasa Bittla

To identify a memory leak, we need to run the load test for longer duration. When the load test runs for long duration, if there is any memory leak, the memory of the respective server will be growing gradually.

For example, you are running the load test for 10 days on an application server which is set maximum heap size of 2GB. If the memory usage of the application server is 1024 MB initially. On day 1, you observed that the memory is 1124MB, on day 2 it is 1224.... Finally on the last day the application utilizing the maximum heap. If you draw the chart of the memory you can see the gradual increase in the heap size of the application server. The heap size of the application server is growing due to a leak in the memory. If you are using a java based application server, the garbage collection may not be happening properly. You need to identify the code which is having the memory leak, by using profiling tools.

15. How to identify a CPU bottleneck on the application?

Sat, 06/18/2011 - 07:57 Srinivasa Bittla

To identify the CPU bottleneck we need to add the counters like %Processor Time. This counter give the percentage of time the processor is busy serving the processes. Identify which process is taking highest CPU, if you are running multiple applications on a single machine. Sort the processes based on the processor time and identify the application which is taking maximum processor time. If you are using windows, go to the task manager and sort the processes based on the CPU usage and identify the application process which is taking maximum processor time. For example, you identified as application server is taking maximum processor time. Then use the application profiling tools to identify the exact API which is causing the high CPU usage. Report the issue with profiling snapshot to the development team.

16. How to identify a disk read or write bottleneck?

Sat, 06/18/2011 - 07:58 Srinivasa Bittla

Disk read or write bottleneck are seen in the servers, where high data intensive operations performed. Mostly these issues are seen, database and file servers. These bottlenecks are identified in windows machines using different set of counters like % Disk Write Time, % Disk Read Time, Avg. Disk Write Queue Length and Avg. Disk Read Queue Length. If you observe % Disk Write Time, % Disk Read Time near to 100%, that means disks are too busy in reading and writing the content to the disks. If it is a database, you may need to use the database profiling tools, to check if there are any SQLs doing full table scan. If there are any indexes missing or SQLs needs to be rebuilt. You may need to give the necessary recommendation to fix the issues. You can also check, Avg. Disk Read Queue Length and Avg. Disk Write Queue Length. If the avg. queue length is greater than 2, that means there are more requests in queue. To overcome this problem, you may need to replace the slow disks with high speed disk arrays.

To check for the disk usage you can use the below mentioned counters in windows from LogicalDisk object: % Disk Write Time is the percentage of elapsed time that the selected disk drive was busy servicing write requests. % Disk Read Time is the percentage of elapsed time that the selected disk drive was busy servicing read requests. Avg. Disk Read Queue Length is the average number of read requests that were queued for the selected disk during the sample interval. Avg. Disk Write Queue Length is the average number of write requests that were queued for the selected disk during the sample interval.

17. How to identify a bottleneck using cpu or disk queue length?

Sat, 06/18/2011 - 07:58 Srinivasa Bittla

If the average of Avg. queue length is greater than 2, that means on an average there are more than 2 requests in queue. When there are more number of requests in queue, that means there is a bottleneck with the respective resource. If the Avg. CPU queue length is more than 2, that means there are more than 2 processes are always waiting to get processed. In this case the CPU is busy serving existing process. This can be concluded as there is a bottleneck with CPU. Similarly if the average of Avg. Disk queue length is greater than 2, there is a bottleneck with disk speed.

18. Whom to interact with, to get the monitoring counters or measures information?
Sat, 06/18/2011 - 07:58 Srinivasa Bittla

To get the application specific monitoring counters, we may need to talk to the respective server administrators. As a performance tester, you may not be the expert of every server which is used as part of the application architecture. If you do not know, what are the counters need to be added as part of the load testing, you may need to talk to the administrators of each server to get the counters. For example, if you want to monitor the DB2 counters. You need to talk to the DB2 database administrators to get the monitoring counters. If you want to monitor the weblogic application servers, you need to talk to the weblogic administrator to get the counters.

19. On what basis you define the thresholds or service level agreements for the load test?
Sat, 06/18/2011 - 07:59 Srinivasa Bittla

Thresholds or service level agreements for the load test will be defined based on the following inputs. 1) Talk to the customer, and try to get the expected thresholds for the application scenarios. 2) Talk to the business analyst, ask him to get the thresholds. 3) You analyze the competitive websites / applications of your customer. And you derive the thresholds and share with your customer and business analyst and get the approval before you do the load test. 4) Analyze any regulatory guidelines for thresholds, if it is applicable to the customer application, and set the thresholds according to the guidelines. 5) If the application is completely new, the you measure the application performance and show the response times to the customer. Ask him to derive, or you decide on the thresholds and get the approval.

20. How do you compute the number of users supported by a load generator or load agent?
Sat, 06/18/2011 - 07:59 Srinivasa Bittla

The number of users supported by a load generator or load agent will be computed based on the Virtual user foot print. Virtual user foot print. is computed based the resource utilization of the virtual user on the respective load generator. We also consider the kind of business operation performed by the virtual user, during the virtual user foot print computation. Before you run the load test, run a dry run with minimal number of virtual users and check the load generator resource utilization. For example, you are running the 20 virtual users through a load generator. You observed that the resource utilization. You have noted that the CPU utilization is increased by 10% and memory utilization is increased by 5%. In this case CPU usage is increasing faster than the memory. So, we take CPU as base to compute the number of virtual users. As per CPU usage guidelines, we will accept CPU to max of 70% and memory up to 120%. In this case for 20 users 10% of CPU utilized. To utilize the max of 70% of CPU, you can run up to 140 virtual users on the specified load generator.

21. What is hits or http hits per second in load testing?

Sat, 06/18/2011 - 08:00 Srinivasa Bittla

The number of hits or requests, that are reached to the web server called as http hits. When a page request sent from the load testing tool or a browser, the sub requests such as .css, .jpg, .js, .png, .gif and .html will be automatically computed and sent to the web server. If you look into the web server logs, you can observe all the web element specific requests, which are reached web server.

22. What is throughput in load testing?

Sat, 06/18/2011 - 08:00 Srinivasa Bittla

The amount of data transferred in bytes during the load test is called throughput. When the load is generated with any load testing tool. You start monitoring the throughput graph. Throughput gives the data like the number of bytes transferred, between the load generators and the application under test.

23. What to conclude by looking into the standard deviation in load test results?
Sat, 06/18/2011 - 08:01 Srinivasa Bittla

The deviation from mean is called Standard deviation. The higher the deviation in average response time, the response times are more inconsistent. The lower the deviation the response times are very consistent. We always expect the response times to be consistent. So, we expect the standard deviation to be lower. If the standard deviation is too high, we conclude that there may be a bottleneck in the application architecture. Note: Before you analyze the standard deviation for response times, ensure that the think time is eliminated from the average transaction response time.

24. What to conclude by looking into the 90th percentile in load test results?
Sat, 06/18/2011 - 08:01 Srinivasa Bittla

Compare the 90th percentile value with maximum value. If the difference between the 90th percentile value and maximum value is huge, that means there are 10% of total long duration response times are inconsistent and there is a huge gap between the maximum and 90 percentile value.

For example, if the maximum response time is 12 sec and 90th percentile is 7 sec. There is 10% of response times between 7 sec to 12 Sec. So, there are only few (10%) response times taking longer durations. So, we suspect that the server is not responding quickly for few cases. In this case we may need to rerun the test or we need to profile folr lower levels to identify the bottleneck For example if the maximum response time is 12 Sec and 90th percentile is 11 Sec. That means the largest response times are consistent. A percentile (or centile) is the value of a variable below which a certain percent of observations. For example, the 90th percentile is the value (or score) below which 90 percent of the observations may be found.

25. What is the difference between http hits, page hits, transaction hits, and user hits?
Sat, 06/18/2011 - 08:01 Srinivasa Bittla

User Hits: The number of user visited from different IP addresses in a given period of time is called user hits. Page Hits: The number of page requests made by all users are called page hits. HTTP Hits: The number of HTTP hits, that reached to the web server. The HTTP requests can be of .gif, .jpg, .css, .js or .png file.

28. What are http status codes?

Sat, 06/18/2011 - 08:03 Srinivasa Bittla

HTTP Status codes are used for determining the response of the server. Especially when the request is sent from the load testing tools, the tools will be evaluating the response based on the response codes. There are three digits in HTTP codes. The first digit of the status code specifies one of five classes of response; the bare minimum for an HTTP client is that it recognises these five classes.

29. Explain about the http status codes?

Sat, 06/18/2011 - 08:03 Srinivasa Bittla

The http status codes are: 1xx Informational - Request received, continuing process 2xx Success - This class of status codes indicates the action requested by the client was received, understood, accepted and processed successfully 3xx Redirection - The client must take additional action to complete the request

4xx Client Error - The 4xx class of status code is intended for cases in which the client seems to have erred. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. 5xx Server Error - The server failed to fulfill an apparently valid request When the request is sent from the client or browser or virtual user. The difference between the first byte of request sent, till the first byte of the response received is called as Time to First Buffer In loadrunner terms: Displays each Web page components relative server/network time (in seconds) for the period of time until the first buffer is successfully received back from the Web server.

30. What is time to first buffer?

Sat, 06/18/2011 - 08:04 Srinivasa Bittla

When the request is sent from the client or browser or virtual user. The difference between the first byte of request sent, till the first byte of the response received is called as Time to First Buffer In loadrunner terms: Displays each Web page components relative server/network time (in seconds) for the period of time until the first buffer is successfully received back from the Web server.

31. What is round trip time?

Sat, 06/18/2011 - 08:04 Srinivasa Bittla

When the request is sent. The total time taken to process the complete request and till it get the complete response is round trip time. This includes: 1) DNS Resolution Time 2) Connection Time 3) SSL Handshaking Time 4) FTP Authentication Time 5) First Buffer Time 6) Receive Time 7) Client Time 8) Error Time

32. What is network time?

Sat, 06/18/2011 - 08:05 Srinivasa Bittla

The total time spent in receiving the response from server is called network time. In loadrunner terms: Network time (in seconds) for the period of time until the first buffer is successfully received back from the Web server.

33. What is server time?

Sat, 06/18/2011 - 08:05 Srinivasa Bittla

The total time taken by the AUT servers to process the request is called Server Time. For the enterprise level applications, this includes web server processing at the presentation layer, business logic processing at the application server and data fetching and processing at the data layer.

34. What is receive time?

Sat, 06/18/2011 - 08:05 Srinivasa Bittla

Displays the amount of time that passes until the last byte arrives from the server and the downloading is complete. The Receive measurement is a good indicator of network quality (look at the time/size ratio to calculate receive rate).

35. What are the acceptable limits for CPU or %Processor Time usage in load testing?
Sat, 06/18/2011 - 08:06 Srinivasa Bittla

The acceptable average limit for CPU or %Processor Time usage is 85%. If it less than 70%, it is good. Even it is between 70% to 85%, we still say it is OK. If the Processor Time is more than 85% continuously more than 10min. We say there is a bottleneck with the CPU

36. What are the acceptable limits for Memory usage in load testing
Sat, 06/18/2011 - 08:06 Srinivasa Bittla

The acceptable limits for Memory (Physical memory) usage is up to 135%. If the memory usage less than 120%, it is good. If it is between 120% to 135% we still accept it. Why memory usage 135% is acceptable?. Because, 100% from physical memory and 35% from Virtual memory.

But if the memory usage is more than 135%, there will be lots of disk swap, which will hamper the performance. So, we will not accept if the memory usage is more than 135%.

37. What are the acceptable limits for Disk queue length in load testing
Sat, 06/18/2011 - 09:07 Srinivasa Bittla

The acceptable limit for average Disk queue length is less than or equal to 2. If the maximum of avg. disk queue length exceeds 2, you dont have to worry. Take a look at the average of avg. disk queue length, if it is less than 2, you do not have any issues. If Avg. Disk Queue Length counter exceeds 2 for continuous periods (over 10 minutes or so during your 24 hour monitoring period) for each disk drive in an array, then you may have an I/O bottleneck for that array. Like the Physical Disk: % Disk Time counter, if this happens occasionally in your 24 hour monitoring period, dont worry too much, but if it happens often, then start looking into finding ways to increase the I/O performance on the server

38. What are the best practices to keep the number of counters per graph?
Sat, 06/18/2011 - 09:07 Srinivasa Bittla

Less the number of counters on a graph the better it is. For example, if you are keeping 10 counters on a graph it is very difficult understand. Because the graph looks so bulky and it is difficult to differentiate the graphs. In my view the optimum number of counters per graph could be 3, and at the max 5 counters per graph. If you keep more than 5 it is very difficult to analyze the results.

39. Whom to contact for analyzing the server side measures?

Sat, 06/18/2011 - 09:08 Srinivasa Bittla

As a performance or load test engineer, you may not be master to analyze the server side measures. If you are unable to conclude some of the measures, better contact the respective server administrators. For example if you are not able to conclude database specific bottleneck based on a certain measure, better talk to the DBA and take a decision.

40. What is the significance of doing parameterization in load testing?

Sat, 06/18/2011 - 09:08 Srinivasa Bittla

To Resolve the issues like:

When the load test requests are sent to the AUT, the requests are not sent through AUT. As most of the validations take place at UI. You may not get any errors for wrong input data. 1) Data Dependency issues like booking flight ticket with depart and arrival cities as the same city 2) Date Dependency issues like booking a flight ticket with past data, giving the future date of birth 3) Caching issues at web server, application server and database server 4) Unique constraint issues like creating an item with same id. Where id is the primary key.

41. Why parameterization of load test script using database or excel sheet is not recommended?
Sat, 06/18/2011 - 09:09 Srinivasa Bittla

When you parametrize or data drive the load test scripts, we need to be very careful in terms of choosing the mode of test data. The default or recommended mode of parametrization is text files. Something like .txt, .csv or .dat files. For reading the excel files, you need to have an excel driver. If at all you are using excel file, for each virtual user the excel driver needs to be loaded to read the data. Which will be an additional over head for the load generator. So that's the reason it is not recommended to use excel file for parametrization. To retrieve the data from the database, you need connect to the database. If you are running 1000 users with database parametrization, then 1000 connections need to be opened from each virtual user to the database. Each database connection is an expensive operation in terms of the database. Your database may be crashing or not responding due to too many connections. This is due to the connections for the data processing by virtual users. So, we don't recommend fetching the data from the database for parametrization in load tests. 5. How the performance bottlenecks can be identified?

14. How to identify a memory leak? 15. How to identify a CPU bottleneck on the application? 16. How to identify a disk read or write bottleneck?

26. How to identify the web server bottleneck? 27. How to identify the network bottleneck?