Webinar: Testing Automation of Mobile Apps - The Best Practices April 13, 2012 Question and Answer Session

Q: When you mention automatic network switching, does this imply that we can simulate, say a Brazilian network while having the device in the US or in India (so we hit geo specific download store), etc.? Answer : No this will not simulate a particular network in any other country, but runs on the network which is available on the devices. Regarding the Geo-specific download store, our framework mAutomate captures the Latitude and Longitude values while recording the test case and then while playing, it gets simulated on the device. So, lets suppose, you recorded your application in a Brazilian network and based on that, your application shows where the nearby ATMs are located. Now if you are playing that application in US or India then in that case, the device would get the LAT/ LONG coordinates of its current location and your app would not show the ATMs located in Brazil. Now how an offshore QA will test this scenario? mAutomate simulates the location attributes from the point where you recorded the test case. So if the offshore team would test the app in India, it would get the list of ATMs in Brazil as the device assumes that it is in Brazil itself. Q: What do you mean by network switching? Is it testing against a variety of access bearers, such as WiFI, 3G, 4G etc? Answer: Network switching is a practice that we recommend to understand the application behavior in different scenarios. Our tool mAutomate helps automatically switch the network like Wi-Fi to 3G and 3G to Wi-Fi.

Impetus Proprietary

Page 1

Q: What type of tools enables Automation Testing of Mobile Apps? Answer: As defined in our Webinar, we can categorize the tools in 3 categories, which is Whitebox QA, Image-based comparison tools and Instrumentation-based comparison tools.. Q: Does mAutomate support iOS? Answer : For the release 1.0, we are supporting Android but version 2.0 which is already under way will also support iOS. Q: Do you advocate the service application remaining in the final commercial device after testing is complete? Answer : We think that the framework injection into the system is ideal. Once testing has been done, you can directly use your own original source code for deployment on the Android market. So, the service application will not be a part of final commercial device. Q: Any way to record cases OUTSIDE the device or is the use of a wrapper completely necessary? Answer : No, you cant record test cases outside the device and yes for recording and playing purpose, our wrapper is required. Q: Does a service application need jailbreaking/rooting? Answer : As discussed in the webinar, our framework provides cloud based infrastructure where you really do not need to see actual device but can still perform the testing. In our case, we provide automatic installation of application Over the Air to devices for testing. If you would like to completely automate system, even perform installation without manual intervention, then the phone should be rooted. Else, automation can be run without rooting the phone.

Impetus Proprietary

Page 2

Q: If recording is made on a LOCAL device, is the use of the cloud absolutely necessary? Answer : No, the use of the Cloud is not necessary in this scenario. If you have all the devices handy with you, then test can be run directly on them, as well. Cloud is only required if you want to test your application across geographies. Like in the scenario where few devices are placed in UK, US India, etc. and then you want to test the application behavior on all of them without deploying a QA resource at each location. Q: Whats the strategy for dynamic screen changes? How to automate those cases? Answer : This framework provides automation of dynamic screen changes by observing the internal elements/objects of the system and would periodically check if there is any difference on those. If there is any difference found, it concludes the dynamic operation on the screen and would then adjust itself automatically to scale this behavior. Q: Looks like the automation framework you have is as per record & playback. How do you maintain the scripts for changes - re-record ? Answer : Yes we purely provide automation through Record and Playback, but would also provide complete script if one can change those. We advise our users to first acquaint themselves with the system and then only try to edit the script. Once the users are trained in that, they can directly play along with the script and do necessary changes as required. Q: Do you have to jailbreak the iPhone? Answer : As discussed on the webinar, our framework provides cloud based infrastructure where one do not need to see actual device but can still perform the testing.

Impetus Proprietary

Page 3

In our case, we provide automatic installation of application Over the Air to devices for testing. If one wants to completely automate the system, including installation, then the phone should be rooted. Otherwise, automation can be run without rooting the phone. Answer is from Android perspective however question is related with iOS. Q: Do we have to just record on one device to test the same test script on different devices? Answer : Yes, we just need to record once to run it on multiple devices. In our case, while recording the test cases, this framework automatically generates script in the background and then can assign these scripts to multiple devices for automatic execution. Q: How does your tool compare to Selenium ? Answer : In mAutomate, we dont even need to write a single line of code and it manages the complete execution automatically. It also covers playing of test cases over cloud and have option to simulate various hardware -specific features. Q. What about playability speed? On my testing and evaluation problem we've noticed that the playability speed of tools like SeeTEST (the only tool that provides access to non-Jailbroken iOS devices) is very slow. Answer : mAutomate intelligently plays all test cases with a much faster speed as compared to the record time. For example, typing values in the entry boxes while recording may take you a long time. However, mAuomate enables quick playing of these test cases, therevy, saving total execution time of the test cases. Q: Does it mean for environmental testing, we need to keep updating the devices? Answer: No, its not necessary to update the devices, but we recommend to use the updated versions. mAutomate does not restrict users on any such environmental testing scenarios. Even you can record your test cases on Android 2.2 and can run them on Android 3.0.

Impetus Proprietary

Page 4

Q: What are some tools that you suggest to run environmental tests? Answer : We suggest using mAutomate as it already covered all factors of the environmental tests. Q: I'd love a list of the tools mentioned, can we make that available? Answer : Traceview and Eclipse Memory Analyser for Android; J2ME profiler for Java phones; JDE memory Analyser for BlackBerry; and Instruments for iOS are already publicly available on the internet. Q: Can this mobile automation test be used for tablets as well? Answer : Yes this can be used for Tablets and does not require special recording of the test cases for larger device. It implies that if you are running the same application on both devices and tablets, then the same recorded test cases can be played on both. Q: Do you have any specific project or client examples using this tool? Answer : We have tried it with few of our customers and infact also shared a couple of case studies in the webinar. Q: Can existing test cases be imported into mAutomate? Answer : That would not be possible. Q: Is it possible to simulate weak connectivity? Answer : This framework currently does not support simulating weak connectivity option.

Impetus Proprietary

Page 5

Q: How do your system test camera-based test cases? Answer :mAutomate supports opening camera interface automatically, but the only pre-condition in this kind of test cases is that the device should be placed somewhere on stand where it can actually take the photo of required object. Q: I am specifically interested in testing the performance of the application not the device. How does mAutomate takes care of it? Answer : This framework help you get performance analysis of your application and even provides reports per test cases. This way, one can understand the CPU and memory usage of an application while running a particular test case. For ex : While testing an application your Memory would increase at some point of time while running a test cases. Now, this information can be easily provided by mAutomate. Our framework would also share the details such as which activity is causing the problem. So, instead of looking into the complete application code, you can directly jump on this activity. Q: It was said that mAutomate was used by a banking company. Banking apps are usually using only WebView. Does the mAutomate support custom UIs? Answer : Web views are not supported by mAutomate but yes, it do support custom UIs. So, if your application has multiple custom UIs and layouts, then you can automate your entire application for those as well. The banking example we have given in webinar is for the customer who was creating native application for banking. Write to us at inquiry@impetus.com for more information.

Impetus Proprietary

Page 6