DEDA at ITU

Spring 2010

MOUSE EXPERIMENT GUIDE

Mouse Pad Dimensions and Their Effects on User Performance

Contents
Quick overview .............................................................................................................................................. 1 Introduction................................................................................................................................................... 2 Experimental design ...................................................................................................................................... 2 Experimental procedure................................................................................................................................ 4 Data and how to analyse them...................................................................................................................... 5 Appendix. Tables for data collection and survey templates ......................................................................... 9

Quick overview
In this experimental exercise, you will replicate an experiment originally conducted by Tara Morgan and Shane Shaffer at the University of Maryland in 1998, as described in the online report published on http://otal.umd.edu/SHORE98/bs03/ You will conduct a user-oriented experiment, measuring the user performance in two types of tasks, and evaluate whether the speed with which these tasks are completed depends on the size of the mouse pad used. When you have run the experiment and analysed the data, you will write a report using the standard template of Field & Hole (2003) and hand it in.

DEDA at ITU

Spring 2010

The work will be carried out partially in the exercise classes, partially as preparation, as follows: Week 5: Starting up the mouse experiment, planning and preparation. Week 6: Running the experiment (there will be other tasks to do in the class). Week 7 and 8: Writing the report. Week 9: Hand-in deadline (hand it in during the lecture or exercise on the 24/3). Running the Mouse Experiment will give you the practical skills you need for running the free experiment in the last half of the semester (this is where you choose what you want to investigate).

Introduction
The purpose of this experiment is to examine the influence of mouse pad size on user performance. We want to figure out what the ideal dimensions for a mouse pad is, based on user performance in two types of tasks. The experiment illustrates the type of work being carried out in an industrial context, and is equally relevant for academic research work. In essence, the participants will be subjected to different interactive tasks, repeating them three times using different sizes of mouse pads. Their performance in completing these tasks (measured as completion time) is recorded, as is their responses to a series of subject, survey-based questions. You will perform some simple calculations on the completion time data to evaluate if mouse pad size has any effect on task completion times, and evaluate whether the participants responses provide any additional knowledge. Combining qualitative and quantitative data in this way is often done because each approach has different strengths and weaknesses they supplement each other. As a preparation, you should read the three first sections (i.e. the sections Introduction, Experiment, and Results) of the report by Morgan and Shaffer, published on http://otal.umd.edu/SHORE98/bs03/ You should also download the executable Visual Basic program, which is available through a link in the Experiment section this program plays a fundamental part in the experiment!

Experimental design
Since we are going to replicate an experiment which has already been conducted and reported, we will closely follow the description given in the report. Hypothesis Begin by formulating a hypothesis what do you expect will happen? Let us for example consider the following hypothesis: H1: Mouse pad size is inversely proportional to task completion time for both clicking tasks and drag and drop tasks. This hypothesis states that we would expect to see a significant difference in performance related to the size of the mouse pad. The relationship will be inversely proportional, with performance improving (completion time decreasing) as the size of the mouse pad is increased due to the user's less frequent need to reposition the mouse to avoid falling off the edge of the mouse pad. We may not expect this relationship to be linear it may, for example, be that the improvement between medium and large mouse pads is smaller than the improvement between small and medium

DEDA at ITU

Spring 2010

mouse pad sizes. We may also expect this relationship between size and completion time to be seen in both the clicking and drag and drop tasks in our experiment. Now that we have formulated our research hypothesis, lets design an experiment to test it! Variables First, we should decide which variables to include in the experiment. In this experiment, we have just one independent variable, namely the size of the mouse pad, with possible values SMALL, MEDIUM, LARGE. (Consider which type of variable this is.) Note that each participant should be presented with the mouse pads in a different order: SML, SLM, MSL, MLS, LSM, LMS. (Why is this important?) We have one dependent variable, the performance time of the user to complete clicking and dragging tasks for each size mouse pad. (Consider which type of variable this is.) In addition to the performance times, each subject will be asked to fill in a subjective questionnaire for each pad-size and then a final questionnaire after using all three pads. This is to give us some user-feedback to compare with our quantitative results (the numbers). Participants You need at least 15 participants. These can be people from your own team (normally we do not participate in our own experiments, but we will cheat here), people from the class or people you know at ITU or elsewhere. The more participants the better. In the Background Information Survey (see Appendix) you will record information about the participants, notably how much they use a mouse. Be mindful of this: if some of your participants have substantially less or more experience using the mouse than others, try and see if their test results were also different. Who we are and our experiences can have a huge impact on experiment results. Materials You need a computer/laptop with the executable Visual Basic program written by Morgan and Shaffer (you can download it through a link in the Experiment section of their report). Furthermore, you need a mouse and 3 custom mouse pads. The mouse pads should have the following dimensions: 200*230 mm (medium); 300*345 mm (large) and 150*172,5 mm (small). Note that all have the same aspect ratio. The small size is 75% the length/width of the medium size, the large size is 150% the length and width of the medium size. The larger large size is because commercial gaming mouse pads are around this size. Make sure your setup is identical for each user: monitor position must be fixed, as must mouse pad position, with the center of the mouse pad positioned consistently for all 3 sizes of mouse pads. The mouse should be centred on the mouse pads at the start time of each of the three procedures. Questionnaires Have your participants complete the questionnaires in the appendix. There is one questionnaire to complete before the test begins (the Background Information Survey). Then there is one questionnaire to complete after each of the three test rounds (the Task Survey) remember, there is one round for each of the three mouse pad sizes. Finally, there is one survey to be filled in after the test (the Post-Task Survey).

DEDA at ITU

Spring 2010

In the appendix you will also find an overview of the information you should give your participants before they begin the test (Mouse Experiment Participant Information).

Experimental procedure
Make sure you run through the tests beforehand, so that you know what to do in case the participant has problems understanding the tasks. Use yourselves as participants also, to get as much data as possible (normally, experimenters should not participate in their own experiments, but we will cheat this time). It is imperative that you do not inform the participants about the purpose of the study until after they have completed the tasks doing so may bias their performance. Here is a check list for the procedure: 1. When your participant enters your laboratory, instruct her or him to sit at the workstation running the test software. Ask the participant to complete the Background Information Survey (age, experience, etc.). 2. Give the participant verbal instructions on what to do during the experiment. Ask if there are any questions. 3. Have the participant start the first set of tasks, using the first of the assigned mouse pads. Make sure that you randomize the order that the three mousepad sizes are used between each participant. With three sizes of mouse pads (SMALL, MEDIUM, LARGE) there are six possible permutations of order: SML, SLM, MSL, MLS, LSM, LMS. Make sure that you run through each of these, ideally with an identical number of participants, to alleviate any unwanted systematic effects. The tasks will be carried out using a custom script written in Visual Basic. The program includes two basic screens, one with a clicking task, another with a drag and drop task. Clicking task: In the clicking task screen, 25 buttons appear on screen. One button will appear red and clicking on the red button will cause another button to turn red. The order that the buttons turn red is random. After all twenty-five buttons have been clicked in the proper order, a number, the subject's completion time in seconds, will appear on the screen along with instructions on how to continue. The completion time should be recorded! Drag and drop task: The drag and drop screen is similar to the clicking task screen, but with the center button on the clicking screen replaced by a larger black box. Instead of clicking on the red button, subjects drag the button to the black box, where it is dropped. As before, upon completion of the task the subject's completion time appear on the screen. The completion time should be recorded! You can download the executable file through a link at the original report website! Download it and save it on your test computer. To activate it, just double-click the file - this will run the test program. There are three versions of each of the two task screens. These will run automatically as the user progresses through the three rounds of tasks. Each version has a different order of activating the buttons, which alleviate the problem or learning i.e. we avoid that the user when repeating a task becomes faster because he or she has already tried them before. All three versions require the same amount of movement in the same order, to ensure that tasks are comparable.

DEDA at ITU

Spring 2010

The program also includes several screens with brief instructions to the participant. Clicking OK on the screen launches the next task and activates the timer. After each set of tasks (one clicking screen, one drag and drop screen), a screen will appear that instructs the subjects to wait while the experiment supervisor switches the subject's mouse pad before continuing with the program. The check list continues thus: 4. Ask the participant to start the first test, following instructions on the screen. 5. When the participant has finished the clicking task, a number, the time used for the task, will appear in the window have the user record this in the space allocated in the Task Survey. 6. Allow the user to follow the instructions on the screen to continue to perform the dragging task. Upon completion, record the time in the Task Survey. 7. Ask the user to complete the questions of the Task Survey. 8. Repeat this procedure with the other two mouse pad sizes. 9. Have the user complete the Post-Task Survey. 10. Thank the participant for his/her involvement in the experiment, and ask if there are any questions.

Data and how to analyse them

For each participant, you will record six numbers: their completion time for the clicking task and the drag and drop task for the three mouse pad sizes. Record these in the schemas found in the appendix of this guide. Also enter the raw data in Excel or SPSS, since youll probably prefer to do calculations and drawing graphs using software rather than doing it by hand. Following this, calculate the descriptive statistics for each of the data sets, using the tables found in the appendix (and reproduced below).
Small Medium Large Mean Standard deviation Median Mode Standard error Range Minimum Notes

Hockey experiment guide Hockey experiment guide Field & Hole Field & Hole Field & Hole Range from smallest to largest number Smallet number

DEDA at ITU

Spring 2010

Maximum

Largest number

Table 1. Template for calculation of descriptive statistics. Clicking tasks results Now it is time to report the results. For each of the two tasks, report the mean value for each of the three mouse pad sizes in a table like the one below. Include standard deviations in a parenthesis after each mean value.
Small Medium Large

Clicking Task X (STD DEV) X (STD DEV) X (STD DEV)

Table 2. Average completion time (with standard deviation) for each mouse pad size. In SPSS or EXCEL, draw a suitable diagram, for instance like the one in Figure 1.

Figure 1. Example of a completion time vs. mouse pad size diagram. Now evaluate the data and the diagram: 1. How do the results between the three conditions vary? Are they similar or different? Does mouse pad size appear to make a difference? 2. Is the difference in the mean values of the three test conditions larger than the standard deviation of each mean? Now follow exactly the same procedure for the drag and drop task. Mouse repositionings During the experiment you recorded how many mouse repositionings that were necessary during the tasks. Calculate the mean for each category and list them as follows: Small Medium Large

DEDA at ITU

Spring 2010

All subjects X Adjusted X

X X

X X

Table 3. Average number of repositionings for each mouse pad size. If you have very few repositionings, consider whether there are any outliers. If, for example, one participant had 20 repositionings for the small mouse pad, while all other participants had less than 2, consider adjusting the mean score by removing that participant from consideration. Be careful how you define outliers, and try and figure out why the outliers occured. Outliers may also contain valuable information! Subjective preferences favourite and least favourite Report the result of the subjective question regarding favourite and least favourite mouse pad in a table. Include the percentage of the total in a paranthesis after the result. For example, if 10 of 100 preferred the small mouse pad size, write: 10 (10%). Report which mouse pad size that was the least favoured by the users in the same fashion. Small Medium Large Favourite X (%) X (%) X (%) X (%)

Least Favourite X (%) X (%)

Table 4. Participants votes for favourite and least favourite mouse pad size (with percentage of total). Subjective preferences - suggestions for size alteration Report the mean results, with standard deviations, for responses to the subjective question as to how the subject would like to change the size of the mouse pad in question. Small Medium Large

X (STD DEV) X (STD DEV) X (STD DEV) Table 5. Average ratings for suggested change of mouse pad size (with standard deviations). Consider other information you may gain from the surveys what information can you extract from them? Do your quantitative results match with the perceptions of the users? Describe your findings in the Results section of the report, and add a Discussion section where you summarise the results and extract meaning from them. The most important question to address is whether the hypothesis was proven or disproven: did the mouse pad size affect the task time completion rates? Address questions such as: Which mouse pad provided the fastest task completion time? Which mouse pad size did the users like the best (and the least)? What would you recommend in terms of mouse pad size for users that utilize the mouse to accomplish the kinds of tasks that were tested?

DEDA at ITU

Spring 2010

The list is by no means exhaustive address any questions you find relevant. It is up to you what you want to focus on, but report results broadly.

DEDA at ITU

Spring 2010

Appendix. Tables for data collection and survey templates

Clicking Task Raw Data Small Medium Large 1

2 3 4
Mean Clicking Task Statistics Small Medium Large

5
Standard Error

6
Median

7
Mode

8
Standard Deviation

9
Range

10
Minimum

11
Maximum

12 13 14 15

DEDA at ITU

Spring 2010

Dragging Task Raw Data Small Medium Large 1

2 3 4
Mean Dragging Task Statistics Small Medium Large

5
Standard Error

6
Median

7
Mode

8
Standard Deviation

9
Range

10
Minimum

11
Maximum

12 13 14 15

10

DEDA at ITU

Spring 2010

MOUSE EXPERIMENT PARTICIPANT INFORMATION

Participant identification Number: ____________ First off, we would like to thank you for taking the time to participate in our experiment. For the experiment, we ask that you please do the following: 1. Do not discuss the experiment with anyone else after you have left the lab.

2. Please fill out all forms completely. 3. Do not talk to anyone apart from the experiment supervisor during the experiment. 4. Avoid spending too much time on one question. 5. Please be as specific as possible when filling out the questionnaire. 6. Do not touch the mouse or the computer until instructed to do so.

11

DEDA at ITU

Spring 2010

Background Information Survey

1. How often do you use a mouse? (Please circle the right answer.) Never 1 Few times a year Once a Month At least Once a Week Daily 2 3 4 5 6 7

2. Do you know how to click and drag icons using a mouse? Yes 3. Your Age (please in full years): 4. Your Gender: (please circle the right answer): Male Female No

If, at anytime during the experiment, you feel that you have to leave, you may do so. You are a volunteer participant and completely free to leave if you wish without any repercussions. If you have completely read and filled out this page, please proceed to the next page.

12

DEDA at ITU

Spring 2010

Task Survey
You may complete this questionnaire only after you have finished the computer tasks: 1. Time for First Task _______ Time for Second Task _______

2. How would you rate the size of this mouse pad? Too small 1 Ok, but prefer Bigger 2 Perfect 3 Ok, but prefer smaller 4 Too large 5

3. How is the size of this mouse pad compared to the one you normally use? Don't normally use one Smaller than what you use Same size as what you use Bigger than what you use

4. At anytime during the experiment did you have to reposition the mouse to complete your task? Yes If yes, about how many times ________ 5. Rate how you would change the size of the mouse pad: Leave the same 3 No

Definitely smaller 1

A bit smaller 2

A bit larger 4

Definitely larger 5

When you have filled in the questionnaire, please remain seated quietly until the experiment supervisor tells you to continue. Please DO NOT start the second round of tasks before instructed to do so by the experiment supervisor.

13

DEDA at ITU

Spring 2010

Post-Task Survey
Fill in this questionnaire only after you have completed all three sets of computer tasks and all three corresponding questionnaires: 1. Which size mouse pad did you like best? Small Medium Large

Why (please be as specific as possible) ________________________ 2. Which size mouse pad did you like the LEAST? Small Medium Large

Why (please be as specific as possible) _______________________

Information about the experiment Now that you have completed the experiment, we will give you some information concerning what the experiment was all about: We were testing to see how the size of the mouse pad affects the users performance when doing clicking or dragging tasks. Please do not discuss this experiment with anyone else or you may affect the results of our experiment. Thank you for taking the time to complete this experiment!

14