D4.10.3 SmartSkiGoggles Experiment Results and Evaluation v1.0

This document describes the final evaluations of the experiment "Smart Ski Goggles".
It
furthermore contains detailed information about the methodology and the resulting insights.
Additionally, the technological implementation and the generation and discussion of business
model scenarios is described.
D4.10.3
Smart Ski Goggles Experiment Results and
Evaluation
2014-05-31

Gerald Binder (evolaris next level)

www.experimedia.eu
EXPERIMEDIA Dissemination level: PU
Copyright evolaris next level GmbH and other members of the EXPERIMEDIA consortium 2014 1

Project acronym EXPERIMEDIA
Full title Experiments in live social and networked media experiences
Grant agreement number 287966
Funding scheme Large-scale Integrating Project (IP)
Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation
(FIRE)
Project start date 2011-10-01
Project duration 36 months
Activity 4 Experimentation
Workpackage 4.10 EX10: Smart Ski Goggles
Deliverable lead organisation evolaris next level GmbH
Authors Gerald Binder (evolaris next level GmbH)
Reviewers Stefan Prettenhofer (Infonova)
Version 1.0
Status Final
Dissemination level PU: Confidential
Due date PM32 (2014-05-31)
Delivery date 2014-05-31


Table of Contents
1. Executive Summary ............................................................................................................................ 4
2. Introduction and Experiment Objectives ....................................................................................... 5
3. Methodology: Co-Creation Process ................................................................................................. 7
3.1. Focus Groups ............................................................................................................................ 7
3.1.1. Setup ....................................................................................................................................... 7
3.1.2. Results ..................................................................................................................................... 8
3.2. Online Survey .......................................................................................................................... 10
3.2.1. Setup ..................................................................................................................................... 10
3.2.2. Results ................................................................................................................................... 10
3.3. Pilot 1 ........................................................................................................................................ 13
3.3.1. Setup ..................................................................................................................................... 13
3.3.2. Results ................................................................................................................................... 14
3.4. Pilot 2 ........................................................................................................................................ 16
3.4.1. Setup ..................................................................................................................................... 16
3.4.2. Results ................................................................................................................................... 16
3.5. Discussion ................................................................................................................................ 19
4. Software Development .................................................................................................................... 21
4.1. System Architecture ................................................................................................................ 21
4.2. Client App ................................................................................................................................ 22
4.2.1. Features ................................................................................................................................ 23
4.2.2. Logging ................................................................................................................................. 26
4.3. Gateway App ........................................................................................................................... 27
4.4. Backend Server and CMS ...................................................................................................... 27
4.5. Integration of Experimedia Components ........................................................................... 28
4.5.1. Integration of ECC ............................................................................................................. 28
4.5.2. Integration of SCC .............................................................................................................. 30
4.5.3. Integration of AVCC .......................................................................................................... 32
5. Business Model Generation and Exploitation ............................................................................. 33
5.1. Business model generation .................................................................................................... 33
5.1.1. Methods ................................................................................................................................ 33
5.1.2. Results ................................................................................................................................... 34
5.2. Discussion ................................................................................................................................ 37

6. Dissemination ................................................................................................................................... 39
7. Conclusion ......................................................................................................................................... 41


1. Executive Summary
This deliverable presents the finals results of the Smart Ski Goggles experiment at Schladming. It
provides information focused on the three major parts of the project. First, we describe our
methodological approach based on co-creation to develop a system displaying real-time
information in ski goggles equipped with a micro-display. We will discuss the results and explain
why we think this approach was suitable for such kind of service. Second, we explain the
technological implementation of Smart Ski Goggles and the integration of external services as
well as the integration of the EXPERIMEDIA baseline components. We discuss the
technological hurdles and will summarize that from a pure technological perspective the service
could be implemented in real-life. Third, we provide information about possible business model
scenarios for Smart Ski Goggles. Here, the summary is not definite. On the short-term and if
such a service is solely focussing on data ski goggles it won't finance itself as the market potential
is too small at the moment. But on the other hand a ski resort could use this service for
marketing reasons. Furthermore, parts of the services features could be used within smartphone
apps or info terminals. The experiment fulfilled the venue-oriented goals of the experiment to
give the venue a good indication on whether real-time information on data ski goggles would be
a service their guests like and to provide the venue with business model scenarios to decide how
such a service could be operated. Apart from that the other major experiment goal to research
how a real-time information system could be implemented into a wearable data ski goggle from a
technical and feature-oriented perspective was reached and the results are discussed in this
report.

2. Introduction and Experiment Objectives
This document describes the results of the experiment "Smart Ski Goggles". It describes not
only the integration of EXPERIMEDIA software components, but also the general
implementation of the software as well as the co-creation process within the experiment and the
way how business scenarios and exploitation ideas were generated.
The experiment Smart Ski Goggles aimed at the experimentation of a real-time information
system implemented into a wearable data goggle (Oakley Airwave). The displayed information
about lifts, slopes, weather, hospitality, community activities and the resort in general support
users with congestion monitoring and basic navigational hints. Smart Ski Goggles is aiming to
enhance the visitor experience while skiing on the mountain.
The experiment builds upon the outcomes of a field study conducted during the FIS Alpine
World Ski Championships 2013 by the competence centre for mobile communication and
innovation evolaris. The experimentation will implement an application for the Oakley Airwave
digital data goggle displaying and processing information, which was found most useful by skiers
in a real-life test. Thus the Smart Ski Goggles app will integrate real-time information about
current load and basic navigation for lifts, slopes and hospitality points of interest in the ski area,
as well as social media features. Current temperature, actual weather forecasts and avalanche
warnings will be implemented in the app to keep the users well informed about the current
conditions on the slope.
The experiment integrates the following EXPERIMEDIA software components:
ECC for experiment control & monitoring
SCC for acquiring a filtered Twitter stream
AVCC for image analysis of visual data of cameras
Evolaris as active member of the European Network of Living Labs (ENoLL) followed a co-
creation approach in the app development and applied a design science methodology to derive a
tailor-made solution right from the end users' needs and requirements as starting point. A
targeted dissemination and exploitation strategy (including business model scenarios) under
strong involvement of the local stakeholders and business partners in the Schladming resort at all
stages of the experimentation will guarantee to meet the expectations and assure a maximum of
sustainability of the proposed solutions, also beyond the projects lifetime. This is underlined by
the composition of LoS partners (resort operator, local retail expert, tourism organization and
specialized technology provider).
From a pure technical perspective the experiments goals are:
to experiment with a real-time information system implemented into a wearable data ski
goggle;
to understand the practical usage potential of real-time information displayed in smart
glasses;

to investigate an emerging technological solution, bring the user in at an experimental
stage of an upcoming commercial solution.
The general objectives of this experiment are to:
analyse and define the stakeholder needs in the Schladming venue & the user needs/
requirement profile for a Smart Ski Goggles app;
implement a real-time information system for a wearable ski data goggle (Oakley
Airwave) in the Schladming international ski resort;
pilot the developed app in real-life settings, disseminate the results and define suitable
business models to assure long-term sustainability of project results.

3. Methodology: Co-Creation Process
A co-creation approach is being applied to integrate all the relevant actors and stakeholders
(users, venue operator, technology provider, etc) into the conception, implementation and
evaluation of this experiment. To maximize the impact of the proposed technical solutions end-
users and stakeholders are continuously involved in the development process.
The recruiting of the participants was done via our Living Lab database, where hundreds of
potential participants for studies are stored and our social media as well as other communication
channels (e.g. poster, flyer). Additionally, our local partners at Schladming helped acquiring the
participants. For the online survey (step 2) we used a commercial service providing a
representative sample of the Austrian population.
All four steps of the co-creation process are tightly linked together to gain a maximum of valid
insight. Figure 1 shows an overview of the different steps and their timing.

Figure 1: Co-Creation timeplan
3.1. Focus Groups
3.1.1. Setup
The focus groups should basically answer the question: What features should be
implemented and how?
Thus, two focus groups were conducted to discuss user requirements, screen designs and
interaction concepts. This was the basis for the conception of the Smart Ski Goggles software
and a representative online survey (the second step in the Co-Creation approach). Each of the
focus groups consisted of seven participants and lasted for around 120 minutes.

3.1.2. Results
The most wanted features were lift waiting time, navigation, notifications (warnings), general
resort information, as well as weather information and forecast. Figure 2 shows the outcome of a
feature idea discussion in focus group 2.

Figure 2: Mindmap of feature ideas
Before we showed the used ski goggle "Oakley Airwave" to the participants, we asked them to
draw how they would like to have information displayed in a ski goggle (Figure 3). There were
also ideas about Augmented Reality information directly projected into the goggles "glass". Of
course, we had to tell the participants afterwards, that the used ski goggle has got a built-in
micro-display. So, we asked them to think, how the GUI elements should be arranged in that
small display. Figure 4 shows some outcomes.

Figure 3: Sketch of information position in goggle

Figure 4: Ideas for positioning of GUI elements
In general it can be concluded that context is very important factor for software running in smart
glasses. For example, a participant brought up the idea that the screen is changing to a screen
with reduced content as soon as the user is moving on the slopes. This was discussed in detail
and finally also implemented. Important alerts should be visually highlighted in the GUI.
Furthermore, personalization and the possibility to configure the software to personal needs
were requested features. Finally, we decided that such features could be part of a future software
version but are not essential for providing real-time information in an experimental setup.

We also asked the participants to evaluate whether and how the menu and status elements
should be arranged. It finally turned out that a combined menu and status bar is ok.
3.2. Online Survey
3.2.1. Setup
The online survey should basically answer the questions: Which features would you use and
when? and How much are you willing to pay to rent or buy a smart ski goggle?
This second step examined the user requirements on a representative level and served as a basis
for a detailed target group specification for the pilot runs. We used computer assisted web
interviews with a for Austria representative sample of 1005 participants. The target sample
(people who use smartphones, have downloaded apps and were skiing at least once in the last 2
years) was 382 people. The analysis was segmented by target groups "Sport", "Spass" ("Fun")
and "Genuss" ("pleasure") as well as by age and gender.
3.2.2. Results
One result of major importance is the data shown in Figure 5. We asked which feature is
interesting during skiing or in a break, as well as, whether a feature is not interesting at all. A very
interesting detail is that the speed was almost evenly evaluated as interesting and not interesting.
Apart from that the survey confirmed our expectation that navigation, lift waiting, weather
information and warnings are top requested features. Additionally, it clearly shows that showing
the pulse in the screen (an idea of the early concept phase) is of comparably low interest.
Showing text messages, e-mail and other social media contents was also rated as not interesting.

Figure 5: How interesting is a certain feature during skiing or in a break?
Asked which contents notifications pushed from the operation into the system should have, the
participants who found notifications interesting in general wished most warnings and
information about weather changes, see Figure 6. Again, it showed that information from other
common communication channels (telephone calls, e-mails, text messages, Facebook postings)
are of much lower interest.

Figure 6: Which contents should notifications have
The analysis of the target groups interest in using features like warnings, navigation and lift
waiting time during skiing, during a break or not at all showed, that the groups "Spass" ("Fun")
and "Sport" are most interested in using these features, see Figure 7. The group "Genuss"
("Pleasure") was ok with warnings, but everything else was clearly less interesting for it. This
difference is also reflected in the basic interest in buying a smart ski goggle, see Section 5 and
Figure 32.

Figure 7: Interest in features segmented by target groups
In general, 63,4 % of the surveys target sample are very interested or interested in using a smart
ski goggle, see Figure 8. There is a clear majority gender-wise. Men are more likely to use it (68,6
%) than women (57,4 %). Almost all age groups are evenly interested, a bit surprising was the
value of 66,1 % in the age group of 50-65 years.

Figure 8: How many people are interested in using a smart ski goggle
Asked about the price the participants would be willing to pay for a smart ski goggle with these
features the average was 160 Euro. On the one hand this is much higher than the price for
average ski goggles (below 100 Euro), but on the other hand it is still very much less than the
current price of the Oakley Airwave (650 Euro).
The average price the participants would be willing to pay to rent a smart ski goggle for one day
was 11,30 Euro, see Figure 9. Very interesting details are that women and people between 15 and
29 years old are much more likely to pay more than 15 Euro a day.

Figure 9: Interest in rental of a smart ski goggle and price per pay

The results of this survey were used to further improve the used software for Pilot 1 and Pilot 2
and of course for the generation of business model scenarios.
3.3. Pilot 1
3.3.1. Setup
The focus for Pilot 1 was on usability aspects. At the same time the test runs during Pilot 1 were
also used to evaluate whether the implemented features were considered to be as useful as the
results of the focus groups and the online survey suggested.
Pilot 1 was conducted in Schladming on two days with in total 15 participants. We used the
Thinking-Aloud method combined with observation and interviews. The total duration of the
test for one participant was around three hours. At the beginning of the test the participants
were briefed about the system setup und provided features, see Figure 10.

Figure 10: Briefing of participants (Pilot 1)
It was not necessary to install the Gateway App on the participants' smartphones as we could use
our own company-owned smartphones. After the briefing the participants were equipped with
the ski goggles and could use it freely for the whole test run, see Figure 11. At the end of the
pilot run the users were interviewed to gain qualitative feedback.

Figure 11: Testrun on the slopes (Pilot 1)
3.3.2. Results
The results of the focus groups and the online survey regarding the feature set were basically
confirmed. As expected the features lift waiting time, weather (current and forecast), information
about huts and navigation were rated as most useful. The notification feature could not be tested
extensively due to technical reasons. So, it could only one message be sent to the participants
(drink voucher), but this was rated as a positive feature.
The feature lift waiting time could only be tested usability-wise, as the installation and integration
of the video cameras and the access to turnstile usage date could not be finished in time due to
organizational and resource reasons. But still, the feature was rated as very useful.
The weather feature was rated as making the impression of a completely finished feature and
should be kept like it was. The Twitter stream feature was not rated that good, as it was not clear
for many participants at all, that this would be a filtered Twitter stream and what the difference
to the notification feature was. It was suggested to use the Twitter logo at least.
Most interesting was the feature navigation, because on the one hand the participants found it
very useful and liked the feature very much. On the other hand there were still many routing
errors in the navigation (which made many navigation results useless) and also the visual
representation of the routing information was not good enough to clearly give helpful navigation
hints. The former point is out of control of the software Smart Ski Goggles, but we informed the
owner of the navigation algorithm and data about errors we found and some of them were fixed.
The latter point was tackled by us for Pilot 2 with many improvements in the visual presentation
of routing information, e.g. with using static arrows and automatic photographic hints.

From a usability perspective we got a lot of useful feedbacks from the participants. Most of the
were related to contrast (see Figure 12), used colours, font sizes, where GUI elements are placed
and how the interaction in lists should work.

Figure 12: Low contrast for selected list element
The Pilot test showed that usability aspects even more crucial in smart glasses software than in
conventional desktop software. The reasons are manifold. First, the display size and resolution is
limited. This leads to necessary reductions of labels, which in consequence makes it harder for
the user to understand the displayed data. Second, the interaction possibilities are limited. On the
remote there are only six buttons: left, right, up, down, enter, back. Actually, this makes the work
on an interaction concept harder and some useful interaction principles could not be
implemented (e.g. free selection of menu item). But, on the other hand due to these limitations
the final interaction concept was easy and clear to the participants. Third, the frame conditions
when using the smart ski goggle have to be considered. The display has to be readable also with
bright sunlight and even during moving on the slope. Especially the second case is important.
The user must not be forced to focus his eye longer than a fraction of a second onto the display
and away from the slope. So the displayed information must be very easy recognizable. Another
idea would be to fully switch off the display as soon as the user is moving faster than a defined
speed.
Figure 13 shows an overview of recommendations to improve the usability of Smart Ski Goggles
resulting from the insights we could gain during Pilot 1. Many of them were implemented into
the software for Pilot 2.

Figure 13: Recommendations to increase usability

Priority Effort Recommendations
Mittel Niedrig
nderung Kontrast Listenelement
Mittel Niedrig
nderung Kontrast Farbehinterlegung und Symbol
Mittel Niedrig
nderung Gelbton / Gelb-Wei-Kontrast bei Navigationslistenelement
Niedrig Niedrig
Testen des Farbkonzepts bzgl. Fehlsichtigkeiten
Niedrig Mittel
Bei Farbschema-nderung: Testing auf existente Designrichtlinien
Mittel Mittel
Reihung der Menpunkte bzw. Menhierarchie berdenken (Twitter vs. Nachrichten)
Niedrig Niedrig
Durchscrollen in der App ermglichen
Mittel Niedrig
berschrift fr den Startpunkt der Navigation integrieren
Hoch Hoch
Entscheidung berarbeitung Navigation oder Umbenennung der Funktion
Niedrig Hoch
Integration Liftausfall in Navigation
Niedrig Niedrig
Verstrkung der roten Umrandung bei ungelesenen Nachrichten
Niedrig Mittel
Andere Darstellungsform der Durchschnittsgeschwindigkeit, z.B. Durchschnittszeichen
Mittel Hoch
Community-Funktion berdenken bzw. berarbeiten
Niedrig Mittel
Symbol im Hauptmen ndern, Httensymbol oder Messer&Gabel-Symbol verwenden
Mittel Niedrig
Anzeige fr Liftfahrdauer als solche betiteln


For example, the one item with high priority was discussed in detail, because the term
'navigation' was interpreted by many participants as describing the same functionality as the
navigation in cars. Lacking a better term it was decided to stick with the term 'navigation'.
3.4. Pilot 2
3.4.1. Setup
The focus for Pilot 2 were user experience aspects. To get feedback from as many participants as
possible we used self-administered questionnaires before and after the test run. So the
participants had to fill out the questionnaires, from which we could analyse quantitative data.
Additionally, short interviews with all participants and one focus group (n=5) were conducted to
get more qualitative data. Pilot 2 was conducted on 5 consecutive days in which 54 participants
tested the software during a timeframe of 2-5 hours. During this timeframe they had to fulfill
two navigation tasks. Apart from the quantitative data measured via the logfile and ECC, there
will be a lot of qualitative feedback from the participants as well.
After a discussion about methodology the General Assembly in January in Graz, we decided to
integrate instant feedback. The participants had to give feedback (good / not good) about the
quality of the displayed lift waiting time.

Figure 14: Instant feedback screen for lift waiting time
3.4.2. Results
As the participants had to fulfil two navigation tasks we got the most feedback about this
feature. To fulfil the navigation tasks the participants had to go to a specific start point and select
the target of the navigation. After that the route was calculated and displayed on the screen, see
Figure 15.

Figure 15: Route list in the navigation feature
Basically, the navigation feature was improved a lot compared to the version tested at Pilot 1.
But still the participants missed some functionality like the usage of a completely free starting
point (at the moment a specific POI must be used) and the re-calculation of the route in case
someone drives in wrong direction. It can clearly be seen that such expectations come from the
fact that many people are already very much used to the navigation features of cars and
smartphones. The qualitative feedback of the participants showed that it was not clear to them
that a slope is by far not as structured as streets and that the movement on a slope is very
different to that on a street. A functionality which was evaluated as very positive was the
automatic display of a photo with a arrow in it to indicate the direction at an important point on
the slope, see Figure 16.
In average each participant started 5 navigations, so the participants not only tried to fulfil the
navigation tasks but also used the navigation with their own start and target points. For 75 % of
these navigations the route could be loaded. This means, that in one of four cases the mobile
network connection was not available. Furthermore, from the loaded routes 49 % were finished,
i.e. the selected target was reached. 39 % of the navigations were cancelled by the user and the
remaining 12 % were not finished (e.g. because the test run ended before). The usefulness of the
navigation feature was rated with 3,2 (1 very useful, 5 not useful at all). This value underlines the
need for improvement if this feature should be further developed.

Figure 16: Automatic navigation hint with photo and arrow


Testing of lift waiting time difficult, as not so many people were on the slopes. This has to be
taken into consideration when analysing the very positive value of the instant feedback for the
lift waiting time. 214 of 238 feedbacks (90%) confirmed a correct waiting time.
The sent notifications were considered to be useful and not too frequent. During the test runs
we sent 4 notifications in total (sent every 30 mins). The contents of the notifications were a
voucher for a drink, a hint to use the instant feedback for lift waiting time, the announcement of
an event (Musikanten WM) and some advertisement for a hut. Especially the notifications about
huts very interesting for the participants. 75 % of all send messages were read by the participants.
The usefulness of this feature was rated with 2,3 (1 very useful, 5 not useful at all).
The weather feature was again considered to be the feature which made the most impression of
being finished. Also the information about huts, their services and photos of the huts very found
useful by the participants. This feature was used in average 7 times per test run. The usefulness
of this feature was rated with 2,2 (1 very useful, 5 not useful at all).
The filtered Twitter stream was, like in Pilot 1, a feature not to be considered interesting. One
reason is the missing interactivity know from Twitter, as it was not possible to reply or forward a
tweet. This leads to the general topic of personalisation. Many participants wished to be able to
have more personalised content, e.g. posting from social networks like Facebook. This is a little
bit surprising as the focus groups and online survey showed a different picture. On the other
hand many participants had IT background and so have more affinity to social networks. In
average 12 tweets were received per test-run, 40 % of them were read by the participants. The
usefulness of this feature was rated with 3,9 (1 very useful, 5 not useful at all).
In summary Pilot 2 showed that the most qualitative feedbacks from participants can be
clustered into three areas: Security aspects, technical problems, the smart factor. Regarding
the security aspects the major concerns are centred around the fact that moving your eyes away
from the slope to the display is a potential risk. Unfortunately, there were many different
technical problems which influenced the test result. This was especially a problem, when
participants very skiing together, because the problem of one goggle was transparent to the other
test persons as well.
The Bluetooth connection problems can be divided into a lost pairing between remote control
and the goggle or between the smartphone and the goggle. Some participants then
unintentionally triggered a factory reset which deleted our software. Furthermore, the GPS signal
was harder to catch on some devices than on others. As this was no problem in earlier tests and
in Pilot 1, it seems a firmware update might be the reason for that. Another problem was the
sometimes not working connection to the mobile network (this was limited and provoked by
some used smartphone models). Finally, the connectivity problems with the ECC were not
directly visible to the participants, but the so induced stress at the test team also influenced the
participants. The last of the three areas concerns the smart factor of the goggles. Obviously,
many participants had higher expectations about the display (projected directly into the glasses)
and the feature set (e.g. map-based navigation like in a car). A reason for that might be that many
of the participants had an IT background. Another reason might be that the media coverage of

the project was extensive and the software was thus more considered to be market-ready than
being a prototype (what it was).
Apart from the above, we have learned from a Quality of Service (QoS) perspective that the
energy consumption of our app the goggles is comparable to that of the original software. We
couldn't discover significant lower run times with Smart Ski Goggles. This means, it was no
problem to run the software 5 hours without any break. However, we have seen, that due to a
firmware problem the recharging of the battery must be done with the software running,
otherwise the battery is not being fully loaded. Additionally, in very cold conditions (below
minus 10 C) the battery life will be much shorter, it is recommended to use an additional
external battery back in this case. The average roundtrip time (time between user acquire data
from the backend and the requested data arrives at the client) was around 2 sec, whereas the
acquisition of a navigation route data needed more time compared to all the other data, as the
routing server had to calculate the route in real-time.
3.5. Discussion
The approach to co-create software for smart glasses in a multi-step approach basically proved
to be very useful. With integrating potential users from the beginning the features could be
implemented as close as possible to real needs and requirements. Of course, some of the
requirements and wished features could not be implemented due to technical, conceptual or
resource reasons.
On the other hand one could argue that integrating potential users in the conception of features
for a device which is so new that it barely hit the market is not a good idea. This has to be
confirmed if it comes to the fact that many people really believed the glass shield of the goggle
would be the display (like a HUD in a car). So, it is very important to make it clear to the
participants of such a study how the display works. This can be achieved best by letting them try
the goggles by themselves.
Some delays in getting access to the turnstile data and the installation of the video cameras could
be caught up until Pilot 2. Though, it was initially planned to have both finished for Pilot 1, so
the user feedback for the lift waiting time feature was not as valuable as it would have been when
testing under more crowded slope conditions during Pilot 1.
An important lesson learnt in this project is that, again, less is more. The integration of too many
features forced us to reduce the detail level or bug fixing of a certain feature due to time and
resource constraints. Especially in Pilot 2 it turned out that the testers were very keen to test the
software thoroughly. The user experience could have been increased with less functionality but
better tested before.
A crucial point are system tests in which the complete system is being testing in real life
conditions, as we have a complex system with many components (goggle, remote, smartphone,
backend server, several other servers) with unpredictable real-time performance (mobile network
availability, GPS availability and accuracy, etc) In a future project of this kind more focus has to
be put on system testing compared to feature development.

Another crucial point is the acquisition of suitable participants for testing and the study
guidelines. We have seen in Pilot 1 and especially Pilot 2 that the expectations of participants are
a major factor in their evaluation of the user experience of the device and the installed software.
And if participants are testing together they are influencing each other which leads to a not so
clear test result. So, the guidelines how to conduct such a study are of major importance as well
as the execution of the study itself.

4. Software Development
4.1. System Architecture
The Smart Ski Goggles system consists of three components. First, the Oakley Airwave is
defined as the frontend. It will be used to display real-time information, which is gathered from
the integrated sensors and from the attached smartphone. The application running on the Oakley
Airwave is referred to as the Client Application.
The Bluetooth-connected smartphone runs the Smart Ski Goggles Gateway App. The Gateway
App is responsible for connecting to the EXPERIMEDIA components and exchange data from
the client application to the Smart Ski Goggles Backend.
The Smart Ski Goggles backend processes information from number different external sources
such as weather service, resort information service, and navigation information provider. In
addition, it receives lift utilization statistics from external components like video cameras and
embedded IR-beams sensors. The lift utilization statistics is analysed and used for providing lift
capacity utilization information to the clients. Furthermore, the Smart Ski Goggles backend also
receives turnstile usage information from each lift which is also used for capacity utilization
information for the clients. Finally, the Smart Ski Goggles backend will be used for pushing
notifications to the client. Examples will be urgent weather warnings, social updates like new
tweets, hospitality offers, or unplanned lift open/close events.
The Smart Ski Goggles backend also provides an interface for administration. This is labelled as
the admin cockpit. In the admin cockpit, the administrators can configure send broadcast
notifications to all connected users. These notifications can be weather warnings, hospitality
notifications, event notifications, and tweet notification. Unplanned lift open / close events are
automatically propagated to all connected clients.
How do they interact?
The Oakley Airwave is connected to the attached smartphone by Bluetooth. It uses a custom
developed message protocol to request information and receive the related responses. The
gateway application also communicates with the EXPERIMEDIA components and the Smart
Ski Goggles Backend server over 3G cellular networks. The Smart Ski Goggles Backend server is
connected via high speed internet connection (100mbit/s) to the internet.


Figure 17: Overall system architecture
4.2. Client App
The Client App runs on the Oakley Airwave, in which the technological components are
manufactured by Recon Instruments. Oakley Airwave runs Android 4.1.2. Recon Instruments
provides a software development kit (SDK), which is used to access the integrated sensors and
to receive skiing statistics. The Client App was developed in an agile approach throughout the
project and was available in v0.2 at Pilot 1, whereas v0.5 was used for Pilot 2. This is final
prototype version of the app.
Basically the user interaction is limited to the means the remote control of the data ski goggle
offers (buttons for up, down, left, right, enter, back). So, we implemented a menu-based
navigation. The horizontal menu of the top of the screen indicates by icons the available
features. The currently used feature is being highlighted. With the "left" and "right" buttons the
user can switch between the features, whereas the "up" and "down" buttons are used to scroll
through lists (e.g. lift waiting time) or switch between screens, in case a certain feature consists of
more than one screen (e.g. weather). With the "enter" button a selected lift element can be
opened, i.e. a detail screen appears.

The Client App was configured in a way that it loads automatically on boot-up of the device and
cannot be exited. This was to ensure that the participants of Pilot 1 and 2 are really only using
Smart Ski Goggles and not accidentally the original software installed on the Oakley Airwave.
4.2.1. Features
Based on the general goal to provide real-time information to the user the following features
were implemented:
Lift waiting time
Navigation to huts, lifts
Current weather and forecast
Notifications, e.g. warnings
Hospitality, e.g. special offers
Filtered Twitter stream
Individual info like speed and distance
The lift waiting time feature is implemented as a list, sorted in a way that the closest lift is on the
top position. The lift waiting time is indicated by coloured icons (green = no waiting time, yellow
= short waiting time, orange = long waiting time), see Figure 18.

Figure 18: Lift waiting time
The navigation feature was being improved substantially after the results from Pilot 1. The major
challenge was to transform the result from the routing server, which was provided by an XML
list, into a format which is displayable and understandable for the user. A huge problem here was
that the routing information (from the routing server) itself many times was not correct. Some
POIs had wrong GPS coordinates or were wrongly connected, sometimes connections between
POIS were missing. For Pilot 2 we tried to improve the displayed routing information with
adding arrow icons into the routing list, see Figure 19. Furthermore, the usage of photographs
with arrows on it (see Figure 20) helped users in Pilot 2 to identify an important point on the
slope where he or she has to turn. These photographs were automatically shown in the display as
soon as the user was as close as 50 meters to the displayed POI.


Figure 19: Routing list in the navigation feature

Figure 20: Automatic photographic hint in the navigation feature
A weather service (wetter.at) was integrated to provide current weather conditions as well as a
forecast for the next hours and days, see Figure 21 and Figure 22. As the screen space is limited
we only displayed the most important information (temperature, weather icon and wind speed).

Figure 21: Weather screen (current weather)


Figure 22: Weather screen (hourly forecast)
With the notification feature the service operator has got the possibility to push messages into
the Client App and inform skiers in real-time about important informations. Figure 23 shows an
example where a event is being announced. The feature is implemented in a way that a
notification can contain a link to another feature or screen in the Client App. If, for example, a
special offer for a hut is being sent, the message can contain a link to the hut's detail page in the
app.

Figure 23: Notification announcing an event
Within the hospitality feature all huts in the ski area are being shown in a list. After selecting a
certain hut, there are several detail pages with text, photo (see Figure 24) and opening hours.
Furthermore, it is possible to start a navigation to the hut in question.

Figure 24: One of a huts details screens
The feature "Individual Information" contains some basic data about the skier's activity, i.e.
average and top speed (see Figure 25), as well as the distance and height meters the skiers drove
during the last run. This feature was implemented as we knew from our first study that these
informations are of high interest to the users and were simply expected. Actually, these
informations are already part of the original software of the Oakley Airwave.

Figure 25: Individual information (average and maximum speed)
An interesting featured was derived from the focus groups: If the speed is higher than 20 km/h
the screen shows only the speed, time and a notification icon (if a new notification has arrived),
see Figure 26. This is to not distract the user. If a navigation is active only the routing
information to the next POI is being displayed (instead of the speed).

Figure 26: Speed, time and notification icon, if user is in motion above 20 kph
4.2.2. Logging
For analysis purposes a debug mode and a logfile has been implemented. The debug mode
displays debugging information which is normally hidden from client. By default, the debug
mode is deactivated.
The logfile is written at a regular interval and protocols the user activity as well as environment
properties such as speed, GPS, battery level, and the related time stamp. Figure 27 shows an
extract of a logfile. The logfile is a comma-separated values (CSV) file format. Each entry starts
with current date, current time, latitude, longitude, altitude, and followed by logging parameter
name and a variable number of log values for this parameter. For example, the fourth line is
logging the "Weather-Start" event. It is triggered when a user enters the weather screen. Chapter
4.5.1 covers the semantics of logfile in greater detail.

Figure 27: Small part of a logfile from a Pilot 1 testrun
4.3. Gateway App
The Gateway App uses a network data caching system in order to counteract mobile network
outages or network reception drops. In addition, it implements a prefetching mechanism so that
often used data such as lift waiting time indication, and weather service are available in advance.
Our caching approach proved to be very reliable. Only a navigational route can only be
calculated if we have live access to the routing server. In future work this necessary live access
could be avoided if we pre-fetch most used or even all possible navigation routes.
The Gateway App provides several configuration options, which affect the functionality. For
example, it provides options to enable enhanced navigation mode (more information), alter
cache time out, skiing mode speed activation limit, or the geo-fence radius.
The Gateway App (on the smartphone) was incrementally improved based on the experience we
have during testing. In general the Bluetooth connection between Client App and Gateway App
was very stable and can easily be re-established if lost. However, during Pilot 2 a few
smartphones had mobile network connectivity problems. This resulted in devices failing to fetch
data from the Smart Ski Goggles backend. Since the problems occurred randomly distributed, it
assumed that it is due to a device specific problem.
4.4. Backend Server and CMS
Due to the fact that the installation of light beams in the area of the lift entrance is only possible
with huge efforts we concluded that we will only use the Skidata turnstile usage data and the
video analysis from the AVCC component to indicate the waiting time at a lift.
The Smart Ski Goggles backend can be used for pushing notifications to the client. Examples
will be urgent weather warnings, social updates like new tweets, hospitality offers, or unplanned
lift open/close events. Unplanned lift open / close events are automatically propagated to all
connected clients. This feature is already finished, see Figure 28, and got very good feedback
during Pilot 1. In addition, chalet owners are having the possibility to promote special meal
offers when there are any new or updated offers. This is being done manually in the Smart Ski
Goggles backend (Admin Cockpit). Basic information about the chalet is per default integrated
into the Smart Ski Goggles software and can be administered via the backend.

Figure 28: Notification screen
The Smart Ski Goggles backend implements an ECC client. It is used for dispatching metrics
such as roundtrip time to external service, AVCC performance data, etc. The user interface of
the ECC client is reduced as the ECC dashboard provides advanced facilities for monitoring
system events.
The Smart Ski Goggles backend was developed using Panama, an open-source Lightweight Web-
Application Framework. The backend uses a MySQL database for storing weather, lift, slopes,
lift utilization and tweet information.
The Smart Ski Goggles backend provides a set of interfaces, which are exposed to the Gateway
App. The Smart Ski Goggles interfaces deliver data in a JSON data format. Since some external
services (Feratel, wetter.at) deliver data in XML format, the data is being converted and
optimized for the Gateway App.
4.5. Integration of EXPERIMEDIA Components
Initially it was also intended to use PCC for retrieving POIs. However, during inspection of PCC
it was revealed that the interface does not meet our requirements for saving POIs. In addition,
the navigation provider also has a set of POIs. Due to technical reasons the navigation provider
cannot use external POIs. As a result the PCC component was not integrated in the experiment.
However, the SCC (SAD service) and AVCC (VAS service) were used, as well as of course the
ECC.
4.5.1. Integration of ECC
The ECC should enable the near real-time monitoring of important experiment parameters (e.g.
status of 3G connectivity). Furthermore it was planned that provenance data will be reported to
the ECC, so it would have been possible to monitor in real-time at the ECC dashboard some
determined actions the participants are doing during Pilot 2, e.g. starting a navigation or using a
specific lift. Unfortunately, the provenance feature was not finished by the baseline component
and thus could not be used in the experiment.
The following metrics were implemented and integrated into the ECC as well as written to the
logfile on the data ski goggles:
QoS

o Battery levels of Oakley Airwave and smartphone (Overall power consumption by
application)
o 3G network reception level (Service availability)
o GPS availability
o Round trip time per feature call (Service performance measurement)
o Lift waiting time provided by AVCC
QoE
o Application Feature Call (To see which features are used where and when)
o Application Feature Usage Duration (How long is a certain feature used?)
o Application Feature Usage Count (How often were which feature used?)
o Navigation Usage Details (Which starting points and targets were used? Was the
navigation cancelled or finished?)
o GPS Position (To send location-dependent notifications)
o Time between reception and reading of notification (Analysis of usage behaviour)
o Instant user feedback on quality of lift waiting time indication
In the preparation and during the Pilot 2 test runs a few problems with the dashboard appeared.
The dashboard responded very slowly and it was difficult for the operators to monitor the
experiment. It was very hard to really monitor the experiment with ten client testing in parallel.
The reason was most probably the unstable mobile network connections of the clients to the
ECC RabbitMQ. Further investigation is required to determine the root cause of the problem.
Figure 29 shows the number of measurement sets which were exported during Pilot 2 using
ECC dashboard export function.

Figure 29: ECC measurement sets (the y axis shows their number)
During the usage of the ECC dashboard in Pilot 2, a number of issues came up:
0
50
100
150
200
250
300
350
31/03/2014 01/04/2014 02/04/2014 03/04/2014 04/04/2014

Each Smart Ski Goggles client reports 62 metric measurement sets. Since there were 11
clients, there is theoretical limit of 682 measurement sets reported. However, Figure 29
displays the number of actual measurement sets reported. It seems that not all available
metrics were reported to the ECC dashboard based on the assumption that there was no
fundamental change of user behaviour.
Changing a new user in the ECC dashboard was very slow. This might be due to the high
number of ECC metrics reported by each Smart Ski Goggles client.
On April, 1st, no clients appeared in the ECC dashboard. In order to fix this problem,
the ECC dashboard had to be shut down and the RabbitMQ queues had to be deleted
before starting a new dashboard.
One operator of the ECC dashboard reported that only few graphs showed actual data.
However, this might be related to first issue reported earlier.
Sometimes an error happened and the dashboard became out of sync. As a result, the
dashboard operator had to perform a browser refresh to fix this problem.
The ECC dashboard is an interesting technology to monitor events and user behaviour. Based
on our experiences above, we identified some recommendations for a future version of the ECC
dashboard:
The responsiveness of the ECC dashboard should be improved. The time to select new
users or adding new graphs should be minimized in order to make the observation easier.
The stability of the ECC dashboard should be improved and sync issues should be
avoided.
The dashboard should support multiple instances of the same experiment to make the
observing easier. This also provides the ability for multiple operators.
The dashboard should support observing measurements of the same type from multiple
users in one single graphs.
4.5.2. Integration of SCC
The SCC supported the experiment in providing a filtered stream of Twitter tweets with the
following filter criteria:
#Schladming or the word Schladming in the text body (no case sensitivity)
#Planai, @planai, or Planai in the text body (no case sensitivity)
#Skiing
#Dachstein or Dachstein in the text body (no case sensitivity)
#Bluetomato
#SkiAmade
#SkiData
@Bluetomato
@SchladDachstein
@skiamade_info
#SSG2014

Here it is important that not too many tweets are pushed into the system. The SAD analysis was
configured to provide new tweets not more often than every hour. The Smart Ski Goggles
backend polled new tweets regularly and new tweets were automatically pushed to every single
connected Gateway (and of course Client) App.
The SCC service worked very reliably (apart from single downtimes due to power outages of the
server infrastructure).


4.5.3. Integration of AVCC
The Smart Ski Goggles experiment was using a new AVCC component (VAS - Video Analytics
Service) from Joanneum Research (JRS) to provide information about lift usage by skiers. The
necessary lift cameras were connected to a processing node provided by the Smart Ski Goggles
experiment, which ran the Video Analysis software. The processing node used the LAN network
hosted by the Schladming lift operator to connect to the Smart Ski Goggles backend. The
cameras were installed in the ski area especially for the experiment. Each VAS processing node
(i.e. every lift) pushed a key performance indicator (percentage of waiting area covered by
people) every 10 seconds to the Smart Ski Googles backend, where it was processed. The
integration of the AVCC/VAS was easy and the performance and reliability was very good.
Figure 30 shows how a lift camera is mounted to observe the waiting area at a lift entrance.

Figure 30: Lift camera

5. Business Model Generation and Exploitation
The business model in general describes which key resources are necessary to produce a certain
value proposition (i.e. the product, service). Furthermore, it describes via which channels this
product or service is being offered and distributed to the customer. Finally, it opposes financial
costs to expected turnover.
For "Smart Ski Goggles" it was necessary to elaborate different alternative business model
scenarios, because there is not really a market in place so far for data ski goggles. So, it is of
special importance to evaluate different scenarios.
5.1. Business model generation
5.1.1. Methods
The general basis for our approach is to use the Effectuation method combined with the
business model framework of Osterwalder/Pigneur, see Figure 31Figure 31. This is based on the
fact, that there are a lot of uncertainties with new technology like data ski goggles, as e.g. the
technology acceptance of smart goggles at all is not very well researched.

Figure 31: Business Model Canvas (Osterwalder/Pigneur)
In general the scenarios we were working on provide alternative approaches from users buying
the data ski goggles with "Smart Ski Goggles" software pre-installed to the ski resort operators
renting the data ski goggles with "Smart Ski Goggles" software on it to skiers.

To work out these scenarios of course some basic data was necessary. This was collected via the
online survey (see chapter 3.2), from which we e.g. learned how much people are willing to pay
as a rental fee (in average roughly eleven Euro), see Figure 9. For the same purpose we
conducted a series of interviews we have done with typical stakeholders of a service like "Smart
Ski Goggles". Thus, we gained insights into the needs and opinions of important partners in the
value chain. For a later exploitation this understanding is crucial.
Of course we also did some global online research to find out whether services like "Smart Ski
Goggles" are available somewhere and how the business model there would work.
5.1.2. Results
In general there are two distinct approaches in distributing "Smart Ski Goggles" and the
necessary data ski goggles. The customers can buy the ski goggles or they can rent them. For the
buying option we found out that in general 69,4 % of the participants in the online survey are
basically willing to buy a smart ski goggle, see Figure 32. On the other hand, the average price
these people were willing to pay was between 113,80 Euro (people who thought the goggle
mustn't cost more than a regular ski goggle) and 160,70 Euro (people who thought the goggle
can cost more than a regular ski goggle). In any case, these price expectations are very much less
than the current price of the Oakley Airwave (roughly 650 Euro). Of course, with higher
volumes and less brand-intensive goggle manufacturers the price will go down, but it is not likely
that it falls below 200 Euro in the coming years. So, for the buying models it has to be kept in
mind that only a very small target group can be reached.

Figure 32: Interest in buying a smart ski goggle (segmented in target groups)
5.1.2.1. Scenario 1 "Rental via Retail"
In this scenario the user has to rent the data ski goggles and the software "Smart Ski Googles" is
pre-installed. With this approach more customers can be reached, but on the other hand the

rental logistics are more effort for the operator of the service. An important partner in this
model is the sports retailer with the necessary experience for the rental process. An option would
still be that the rental process is being done by the ski resort operator himself - e.g. with renting a
data ski goggle in the same step as buying the ski ticket.

Figure 33: Business model scenario "Rental via Retail"
5.1.2.2. Scenario 2 "Rental via Special Partner"
Scenario 2 is distinct from scenario 1 especially in the way how the data ski goggles are
distributed to the customer. Here hotels or VIP clubs are renting the goggles to their guests. This
can be either done for a fee (like in scenario 1) or as free premium service or as part of a
package. Of course, this approach makes it necessary that hotels or VIP clubs organise their
rental process by themselves.

Figure 34: Business model scenario "Rental via Special Partner"
5.1.2.3. Scenario 3 "Buying and Branding"
Scenario 3 is based on the approach that a branded version of a data ski google with the app
"Smart Ski Goggles" on it is being offered to customers via regular retail or online shops.
Depending on the marketing value which is being calculated by the ski resort this could even
lead to lower prices as if the customer would by the data ski goggle in an unbranded version. The
advantage here is the simpler logistics process, but on the other hand the ski resort has to invest
in the data ski googles which at the end are potentially not being sold, which is a valid risk.

Figure 35: Business model scenario "Buying and Branding"

5.1.2.4. Scenario 4 "App Only"
The final scenario is based on the approach that only the app "Smart Ski Goggles" is being
distributed online (in the beginning potentially for free). This approach has the lowest risk for
the ski resport but also the potentially lowest marketing value.

Figure 36: Business model scenario "App Only"
5.2. Discussion
For all elaborated business model scenarios it has to be taken into account that probably none of
these will lead to huge turnovers in the first years as the market is not that developed yet. But,
apart from the turnover potential also the marketing potential has to be considered (see chapter
6). "Smart Ski Goggles" is definitely a unique experience for guests and a ski resort can
differentiate itself from competitors through destination-specific information and services like
e.g. real navigation. From a customer point of view for sure some improvements have to be
done to the software, e.g. personalization and implementation of new features. Of course, also
the cost for reaching marketability have to be taken into consideration.


To discuss the elaborated business model scenarios with experts a business model workshop was
conducted in Schladming. With participants from the lift operator, the regions tourism office and
Schladming 2030 the different approaches were evaluated against feasibility and necessary pre-
conditions to implement them in real life. A crucial discussion point was that the cost to further
develop and test the current prototype of "Smart Ski Goggles" to get it market-ready has to kept
low, otherwise the ski resort wouldn't invest in a service with such a small volume of users.
Again, the marketing value as well as the indirect rentability (people reading about the service in
the media might be interest to come to the ski resort, even if they don't buy or rent the data ski
goggle) have to be taken in account. Specific for Schladming was, that the media coverage about
the research project with "Smart Ski Goggles" was huge and so this marketing value was
according to the experts considered to be already exploited to a large extent.
In summary, the business model scenario 3 (Buying and Branding) was the one, which was
evaluated as most interesting of the four presented scenarios. Though, as it can be seen in Figure
37, in the end a combined approached was favoured. So, skiers should be able to buy a branded
data ski google but it should also be possible to only rent it.

Figure 37: Best rated scenario at Business Model Workshop

6. Dissemination
We have issued a press announcement in November 2013 and got some very good press articles
in Austrian print and online media. During the Pilot 1 another press announcement was made by
our project partner Schladming-Dachstein, which resulted in many articles (more than 30!) in the
Austrian and German press and online media, e.g. Figure 38. There was even a radio report in
Austrias largest radio station (3), see Figure 39, and one in TV (Puls 4).

Figure 38: One of the press articles about Smart Ski Goggles


Figure 39: Radio interview with test persons
Due to the fact that we have huge interest by the media we conducted a special test day for
media people. During this day the journalists could test "Smart Ski Goggles" by themselves and
they had the opportunity to ask detailed questions about the service.
The huge media coverage led to an extensive publicity of the project. This was good in the sense
of making the research efforts in this EU funded project public. During Pilot 2 we learned that
this publicity also had a disadvantage. Many of the participants already knew about the project
and the features of "Smart Ski Goggles". Obviously, they had the impression that the software is
already market-ready and some of them even thought the displayed information would be shown
directly on the goggles glasses. For the Pilot 2 this meant that these high expectations for sure
influenced the user's evaluation of the tested prototype.
The project will be submitted to two competitions in Austria: First, it will be submitted to the
"Fast Forward Award" and later on to the "Steirischer Tourismusinnovationspreis".

7. Conclusion
In summary the experiment showed that in general there is a huge potential for using data ski
goggles with apps like "Smart Ski Goggles". People are basically open to use data googles if they
offer an added-value to them. Though, security and privacy aspects have to be taken seriously.
The approach to follow a multi-step co-creation methodology proved to make sense, as the
concept could be developed in a very agile way, always close to the potential customers' needs
and concerns. Another important point was the inclusion of many stakeholders from the
beginning of the experiment to the very end. Only with these experts from different areas like ski
resort operations, retail, tourism and marketing a holistic view of the service could be reached.
Technology-wise the operation of a service like "Smart Ski Goggles" is feasible, all technologies
needed are in place and more or less conventional. The integration of available EXPERIMEDIA
components like e.g. the video analysis module (AVCC) made it during the experimentation
easier to implement certain functionalities. Though, strong support by local stakeholders is
needed (depending on type of data).
From a user's point of view it is crucial to have an easy and clear UI as well as an intuitive
interaction concept. On top of that other user experience factors like the form factor, weight,
battery life, price, etc. have to be taken into account if the service should be developed further to
marketability.
From a business perspective the difficulty is that the market for these devices has yet to be
developed. As long as there are no useful apps available many people won't buy such goggles.
On the other hand as long as not so many devices are used no one wants to invest in developing
such apps. This chicken-egg-problem is typical for the very early stage of new products and can
only be solved by brave entrepreneurs. This experiment have contributed to this in providing
deep insights into technological, organizational, economic and social aspects of such services.
In terms of exploitation the next steps should be to discuss the results of the experiment with
interested stakeholders (HW manufacturers, ski resorts, retailers, etc) and to further develop the
business model scenarios.

D4.10.3 SmartSkiGoggles Experiment Results and Evaluation v1.0

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

D4.10.3 SmartSkiGoggles Experiment Results and Evaluation v1.0

Uploaded by

Copyright:

Available Formats

This document describes the final evaluations of the experiment "Smart Ski Goggles".

You might also like