1

1. Introduction

The Wide Area Video Communication Application is a mobile video call

application that offers features namely Video Call, Video Chat, and File Transfer.
1.1 Purpose
The primary purpose of the test was to test the functionality of the application and
investigate whether users are comfortable with the applications features and
functionalities.
Executive Summary
The researchers conducted a usability test to selected students from Cor Jesu
College on November 8, 2014. The students were provided with the applications
Android Project Kit (APK) and Wireless Fedility (Wi-Fi) connection to test the
functionality of the application.
All the selected students are from the Computer Studies Division (CSD). There
are only 10 selected CSD students to perform the test due to some of the students dont
have android devices or smart phones.
The test identified some of the problems stated below:

Stability of sessions
The sessions are unstable due to the availability of the web
server. The website where the sessions are made is not always
available. There are times when the website is timed out without

certain reason.
Quality Output
The video and voice quality is sometimes poor due to the
internet connections performance. The quality output depends on

how fast and reliable the internet connection is.

Incompatibility of the application to any Android mobile Operating
System (OS).

Methodology
Sessions
The researchers recruited 10 CSD students to perform the test. The researcher
gives a brief introduction to what the application is about and the functionality of the
application. Then, the researchers gives proper instruction on how to use the application.
The students were gathered before giving the instructions. The session lasted for
15 minutes including the giving of the instructions and the test questionnaires for
evaluation.
After each task, the administrator asked the participant to rate the interface on a 2-point
Likert Scale with measures ranging from Disagree and Agree. Post-task scenario subjective
measures included (see Attachment B):

the application is easy to use;

the application is modernize and offers friendly user interface;
the application is functional due to new technologies involved namely Android;

and
the application is useful to the society.

After the Post-task scenario subjective measures, the students were asked to rate the
application overall by using a 5-point Likert scale (Strongly Disagree to Strongly Agree) by the
eight subjective measures mentioned below:

ease of use;

frequency of use;

learning ability - How quick would it be for the users to learn the application
and its features;

graphical Appeal User likes the User Interface of the application;

application content Applications contains sufficient functions for

the needs of the user; and

organized Application.

Lastly, the researchers asked the students the following overall application
questions listed below:

what the participants likes the most in the application?;

what the participants likes the least in the application?; and
comments and Recommendations for improvement.

Refer to Attachment C for the subjective and overall questionnaires.

Participants
The participants were selected students from Computer Studies Division. The
participants completed the test on November 8, 2014 at ITR1 class room. Among the
selected participants, each has an android phone.
Table 1: The table below shows the registration information needed from each users to
login and use the application.
Username
Password

Input users desired username

Input users desired password

Email Address

Input users valid Email Address

Session ID and Token

Auto-generated by the server upon

successive registration

API key

Given by the server upon successive

registration

Evaluation Tasks/Scenarios
The selected students attempt to complete the eight tasks given by the researchers
listed below:

fill up the registration fields to complete the registration;

after successive registration, Login using the registered accounts;
the application now directs users to the main interface of the application to

provide selection of contacts;

after selecting the desired contact, the application directs users to the feature

options the application provides;

the application offers features namely video call and video chat;
after selecting the desired option, the application will now connect to the other

pair of user to perform the specific feature selected;

wait for the applications response; and
after the application responded, the two pair of users are now connected and ready
to perform the specific feature.

Results
Task Completion Success Rate
All the selected students completed all the given tasks. Task Completion
Rates Table is given below.

Table 2: Task Completion Success Rates

Participa
nt
1

Task
1

Task
2

Task
3

Task
4

Task
5

Task
6

Task
7

Task
8

Success

Completi
on Rates

100
%

100
%

100
%

100
%

100
%

100
%

100
%

100
%

Task Ratings
The selected students completed each tasks given and rated the ease and difficulty
of completing the tasks given the factors listed below:

i was able to do the register and login function with ease and without

further questions;
as I go on with the application, I was able to understand each feature and

completed the task given; and

i was able to understand the whole function of the application.

The five-point rating scale ranged from 1 (Strongly disagree) to 5 (Strongly

agree). Agree ratings are combined with the strongly agreed (5) and agreed (4) with a
mean agreement ratings of 5.0 considering that all the participants agrees that they were

able to do the register and login functions without further questions, they were able to
understand each features the application provides and understand the whole functionality
of the application.

Ease of completing the Register and Login requirements

All the selected participants agreed that it was easy to perform the Register and
Login Functions

Understands the provided features the application offers

The selected participants agreed that the features were easy to understand and
they were able to complete the task with ease.

Understands the whole function of the Application

The selected participants agreed that they understand the whole application and
how it works.

Table 3: Mean Task Ratings & Agree Percentage

Task

Ease of

Understands

Understands

Completing

the Features

the whole

Register and
Login

Functionality

Overall

7
1

5.0 (100%)

5.0 (100%)

5.0 (100%)

5.0

5.0 (100%)

5.0 (100%)

5.0 (100%)

5.0

5.0 (100%)

5.0 (100%)

5.0 (100%)

5.0

5.0 (100%)

5.0 (100%)

5.0 (100%)

5.0

5.0 (100%)

5.0 (100%)

5.0 (100%)

5.0

5.0 (100%)

5.0 (100%)

5.0 (100%)

5.0

5.0 (100%)

5.0 (100%)

5.0 (100%)

5.0

5.0 (100%)

5.0 (100%)

5.0 (100%)

5.0

*Percent Agree (%) = Agree & Strongly Agree Responses combined

Time on Task
The testing of the application and performing the tasks were recorded for each
participant. All the tasks given were completed without difficulties.
The selected participants were given task to fill up the information asked from
register and login, test the features the application offers, and judge if the application was
easy to use and whether they understand the flow of the application. Completion of the
tasks given ranged from 2 minutes minimum time to 10 minutes maximum time.

Table 4: Time on Task

P1

P2

P3

P4

P5

P6

P7

P8

Avg.
Total

Task

(mins)
4.63

10

6.38

Task

10

5.25

7.25

Task

5.88

3.5

Task

5.5

5.25

Task
5
Task
6
Task
7
Task
8

Summary of Data
The table below displays a summary of the test data. Low completion rates and
satisfaction ratings and high errors and time on tasks.
Table 5: Summary of Completion, Errors, Time on Task, Mean Satisfaction
Task

Task

Errors

Time on

Satisfaction

Completion
8

Task
4

*
4.7

4.7

4.7

4.7

4.7

4.7

4.7

4.7

10

* Satisfaction = Mean combined rating across three post-task measures: ease of finding
the information, ability to keep track of location in site, and site information prediction
accuracy.
Overall Metrics
Overall Ratings
After task session completion, participants rated the application for eight
overall measures. These measures include:

Table 6: Post-Task Overall Questionnaire

Strong

Disagr

Neutr

ly

ee

al

Agree

Disagr
Thought

Strong

Mean

Perce

ly

Rating

nt

Agree

Agree

ee
0

3.9

100%

3.9

92%

3.9

100%

application
was easy to
use
Would use
application
frequently
Found it
fast and
easy to
learn the
application
and its
features

11
Graphical

3.9

85%

3.9

90%

3.9

95%

User
Interface
makes me
want to
explore
more of the
application
Application
content
would keep
me using it
Application
is well
organized

Likes and Dislikes

Upon completion of all the tasks, selected participants were asked about their
comments and provided the researchers feedback of what they like the most and what
they like the least in the application.
Liked Most
The participants were asked what they liked the most and give the following
replies:

Liked Least

i like doing the video call and voice call feature;

i liked graphical output of the video call and chat feature;

12

The participants were asked what they liked the least and give the following
replies:

it takes several seconds to display the video stream;

Conclusion
Most of the selected participants found that the application is very useful to the
school and to the work groups within the society which lessen their time contacting
others specially when its an emergency due to the application is available anytime as
long provided with fast and reliable internet connection to process the video feeding.

ATTACHMENTS

Attachment A Background Questionnaire

Name: _____________________________________________________
School: ____________________________________________________
Age: ______
Gender: __ Male
__ Female
Status: __ Single
__ Married
__ Separated

13
__ Widowed
Occupation: _________________

Attachment B Post-Task Questionnaire

Directions: In answering the questions below, use this scale (From standard form for evaluation):

5 = Strongly Agree/Always
2 = Disagree

4 = Agree

3 = Neutral/Often

1 = strongly Disagree/Never

For each question, please circle the number that best represents your opinion.
Table 7. Post-task scenario subjective measures
A. Post-task scenario subjective measures
The application is easy to use.

The application is functional due to new

technologies involved namely Android.

The application is useful to the society.

The application is modernize and offers friendly

user interface.

14

Table 8. Task Ratings

Task Ratings

It was easy to do the register and login

function.

The features are easy to use and

understand.

The applications whole functionality

is very understandable.

15
Attachment C
Table 9. Post-session Overall Subjective Questionnaire
Overall Application Questions:
What you liked most?
What you liked least?
Any recommendations for
improvement of the
application?

Attachment D Task Scenarios

Directions: In answering the questions below, use this scale (From standard form for evaluation):

5 = Strongly Agree/Always
2 = Disagree

4 = Agree

3 = Neutral/Often

1 = strongly Disagree/Never

For each question, please circle the number that best represents your opinion.

16

Table 10. Post-Task Overall Questionnaire

Post-Task Overall Questionnaire
1. Thought application was easy to use

2. Would use application frequently

3. Found it fast and easy to learn the

application and its features.

4. Graphical User Interface makes me want
to explore more of the application
5. Application content would keep me using
it
6. Application is well organized