You are on page 1of 124

Evaluation study of definitions,

gaps and costs of response


capacities for the Union Civil
Protection Mechanism
Implementing Framework Contract
575/PP/2016/FC

Client: DG ECHO

Rotterdam, 9 October 2018


Evaluation study of definitions,
gaps and costs of response
capacities for the Union Civil
Protection Mechanism

Implementing Framework Contract 575/PP/2016/FC

Client: DG ECHO

Dr. Brigitte Slot

Rotterdam, 9 October 2018


Table of contents

1 Introduction 9
1.1 Members of the consortium 9
1.2 Key strengths of the consortium 11
1.3 Background of the study 12
1.3.1 Genesis of the European Union’s civil protection system 12
1.3.2 Functioning of the European Union Civil Protection Mechanism 13
1.3.3 Main features of current response capacities 17
1.3.4 Appropriateness of the existing response capacities in the UCPM 18
1.3.5 Available new response solutions 21
1.4 Purpose and scope of the study 22

2 Methodology 24
2.1 General approach 24
2.1.1 How the tasks are interlinked 24
2.2 Proposed tasks and subtasks 25
2.3 Proposed methodology 26
2.3.1 Data collection 27
2.3.2 Risk and mitigation strategies 41
2.4 Task 0: Project Management 44
2.4.1 Objective 44
2.4.2 Proposed methodology 45
2.4.3 Deliverables 48
2.4.4 Risks and mitigation strategies 49
2.5 Task 1: Review and (re-)define the existing response capacities 50
2.5.1 Objective 50
2.5.2 Proposed methodology 51
2.5.3 Deliverables 56
2.5.4 Risks and mitigation strategies 56
2.6 Task 2: Cost analysis 57
2.6.1 Objective 57
2.6.2 Proposed methodology 58
2.6.3 Deliverables 64
2.6.4 Risks and mitigation strategies 65
2.7 Task 3: Risk based assessment and capacity gap analysis 65
2.7.1 Objective 65
2.7.2 Proposed methodology 66
2.7.3 Deliverables 69
2.7.4 Risks and mitigation strategies 70
2.8 Task 4: Revision of capacity goals 70
2.8.1 Objective 70
2.8.2 Proposed methodology 71
2.8.3 Deliverables 77
2.8.4 Risks and mitigation strategies 77

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 5
2.9 Task 5: Logistical support Error! Bookmark not defined.
2.9.1 Objective Error! Bookmark not defined.
2.9.2 Proposed methodology Error! Bookmark not defined.
2.9.3 Deliverables Error! Bookmark not defined.
2.9.4 Risks and mitigation strategies Error! Bookmark not defined.
2.10 Planning and deliverables 79

3 Project Management 82
3.1 Consortium structure Error! Bookmark not defined.
3.2 Team structure 82
3.3 Roles and responsibilities 83
3.4 Introduction to the team members Error! Bookmark not defined.
3.4.1 Overview 93
3.4.2 Expertise of our team members 95
3.4.3 Profiles of the team members Error! Bookmark not defined.
3.5 Team resources Error! Bookmark not defined.

4 Quality Control 99
4.1 Quality assurance principles and overall approach 99
4.2 Guaranteeing high quality deliverables and services 100
4.2.1 Multi-layered quality control procedures 100
4.2.2 Creating a learning consortium 103
4.2.3 Additional quality control measures 103
4.3 Guaranteeing smooth and efficient implementation 104
4.3.1 Respecting deadlines and timely delivery 104
4.3.2 Ensuring continuity of service 105
4.4 Risk management and mitigation strategies 105

Annex I National contact points UCPM 113

Annex II Interview guidelines 117


Lot 1 117
Lot 2 117
Lot 3 118

Annex III Checkmarket survey 120

6 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 7
1 Introduction

1.1 Members of the consortium

Ecorys has a remarkable 80-year history, which started in 1929 with the establishment of the
Netherlands Economic Institute in Rotterdam by Nobel Prize winner Prof. Jan Tinbergen. With
headquarters in the Netherlands and more than 12 offices around Europe, including Brussels and
the United Kingdom, we provide a full range of services, from research and policy advice, through
institution building and capacity development, to large programme management and
implementation, communication and financing advice, and monitoring and evaluation. Ecorys
specialises in economic and social development and is active at all levels of policymaking and work
worldwide on complex market, policy, competitiveness and management issues. Ecorys has a
strong track-record and excellent in-house expertise in several knowledge areas relevant for the
assignment.

Since 2014, Ecorys has been leading support services for the DG HOME led Community of Users
initiative (ongoing), which aims, among other things, to increase practitioner involvement in EU-
funded security research. In 2015, Ecorys led the first European level capability gap analysis and
corresponding market analysis of First Responders (used as a reference document in the
International Forum for First Responder Innovation). Ecorys is also participating in the FP7
DRIVER+ project as lead partner in the development of a shared understanding of crisis
management in Europe and the sustainability component of the project. In addition to these
projects, Ecorys provides regular evaluations and impact assessments in the policy areas of
migration, border security, and counter-terrorism (including CBNR-E).

The VVA Group was established in 1992 by a team of professors from Bocconi University. Over 20
years it has developed into a full-service consultancy with offices in Milan and Brussels. Our in-
house team of about 100 consultants, academics, economists, lawyers and researchers specialises
in providing high quality advisory services to private and public-sector clients. VVA has built a
reputation for delivering high quality analysis, clearly expressed, to tight deadlines. In addition to
our public sector clients (mostly the European Commission), we have worked for more than 600
companies, both in the Industrial and Mass Market industries. In particular, the VVA team has a
proven track record in providing in-depth analysis, strategic advice and business planning support
to entities and companies involved in digital and technology industries.

Most of our projects are based on market analysis and forecasting, competitiveness assessment,
economic and impact analysis, which are VVA’s core competences but we also deliver much wider
services that incorporate consumer, social, environmental and legal aspects. For instance, we have
recently completed a study for DG CONNECT on the Promotion of European content in audio-
visual media which included a market and sector analysis, an economic and a legal overview, a
detailed content analysis based on large programming datasets from linear and non-linear
providers. The study on non-linear providers in particular was based on an innovative set of data
from a non-linear aggregator platform which allowed us to assess catalogues and also audience
engagement with those catalogues in several countries across Europe.

VVA retains strong links with the most prestigious Universities and Research Centres and
continues to invest in research. For instance, we have recently invested in an artificial intelligence

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 9
start-up (Indigo) and a digital marketing start-up (Rankit) This enables VVA to offer the highest level
of up-to-date methodologies and services that use cutting-edge technology. Our Board of Directors
includes both Professors and PhDs from Bocconi University and the London School of Economics
who take personal responsibility for the academic quality of every piece of work that we do.
We work closely with our clients throughout each project to ensure that we provide a cost-effective,
bespoke service at all stages of the project. Our independent, creative and innovative team of
consultants and researchers from different backgrounds allow us to cover a wide range of strategic
needs. Whilst all client contact is led by our in-house team, we are able to bring in very specific
senior expertise through our wide network of partners and universities.

Risk Society is a consulting company based in The Hague, The Netherlands. Established in 2009
with the purpose of building expertise and capacities to manage risks for sustainable development,
the company has been providing a wide range of consultancy services related to various aspects of
resilience building and disaster/climate risk management. Its consultancy services are regularly
requested by the leading international development and humanitarian organizations across the
globe such as UNDP, UNOPS, UNISDR, UN-Habitat, UNON the World Bank, the European
Commission, Norwegian Refugee Council (NRC), Red Cross organizations, Open Society
Foundation (OSF), Netherlands Helsinki Committee (NHC), Cooperazione Internazionale (COOPI),
and many others.

Risk Society has strong expertise in all-hazard risk assessment and risk governance, national risk
assessments, disaster risk reduction and climate risk management, enterprise risk management
(ERM) but also civil protection, post disaster needs assessment (PDNA) and recovery planning,
resilience building including urban resilience, results-based management (RBM), strategic planning,
monitoring and evaluation. It partners up with leading international institutions to develop risk-
related publication (UCL, UK, for instance) or providing lectures and designing institutional
development strategies for the various crisis management educational institutions (in Armenia and
Iraq, for instance). It has been providing technical support to national civil protection and crisis
management authorities in various countries. Examples of the services provided by Risk Society
are the evaluation of ‘Making Cities Sustainable and Resilient (MCSR)’ implemented by UNISDR
and UN-Habitat (funded by DG DEVCO), co-authoring National Disaster Risk Assessment
Guidance commissioned by UNISDR and designing National Disaster Management Strategies for
Moldova (2012) and Turkmenistan (2015).

The Emergency Services College (Crisis Management Centre, Finland) is a national college
providing education for firefighters, sub-officers, fire officers and emergency response center
operators in Kuopio, Finland. The College also offers a wide variety of specially tailored further
training and in-service training for national and international professionals in the rescue and
emergency field. Furthermore, ESC provides preparedness training for disturbances in normal
conditions and emergency conditions as well as training for international civil crisis management
tasks. ESC is involved in the research and development functions in the field. The College is
supervised by the Finnish Ministry of the Interior.

ESC has been actively involved in the EU Civil Protection Mechanism for several years. It has
conducted and co-organised several large scale exercises within EU, UN, NATO and Barents
Region contexts. In addition to exercises, we organize annually around 50 training events related to
civilian crisis management and civil protection. The College manages EU Civil Protection experts,
an INSARAG classified HUSAR Team and a TAST Team, and is in charge of recruiting, equipping
and training them. ESC is also the home office for Finland's national EU Training Coordinator which

10 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
eases the organisation's cooperation with other Participating Statesꞌ training authorities. ESC has
extensive experience of planning, managing and implementing EC funded projects. Finland has
contributed to the development of EU Civil Protection particularly related to EU’s capacity to
operate in cold conditions.

Sagitta Evolution SPRL is a Brussels-based company providing consulting services to a large


variety of European businesses and institutions since 2014. Specialised in project and programme
management, evaluation, strategic communications, knowledge management and stakeholder
engagement, we propose strategic focus on results and tailored innovative solutions, enabling
efficient design and delivery of programmes and projects, as well as their uptake.

Sagitta Evolution provides professional support to institutional and private operators for study,
research, prospection, management, coordination, implementation, monitoring and evaluation of all
services, at local, regional, national and European level. Sagitta Evolution has been involved, as a
subcontractor, in a number of relevant contracts with the European Commission. For instance, we
designed and implemented the stakeholder consultation strategy in the framework of the Mid-Term
Evaluation of the LIFE programme for the Environment and Climate Action (DG ENV); and for DG
MARE we designed and implemented the stakeholder strategy and knowledge management
process of the Assistance Mechanism for the Atlantic Action Plan to develop project pipeline on
Blue Growth in the Atlantic. We are also playing a leading role in the Secretariat for the Urban
Agenda Air Quality Partnership (DG REGIO) cultivating stakeholders relations of a partnership
gathering 4 Member States, 6 cities, 2 European platforms, and the European Commission (i.e. DG
REGIO, DG ENV, JRC), providing thematic expertise for drafting the Partnership’s Action Plan, for
communication, developing stakeholder consultations, designing and organising events.

1.2 Key strengths of the consortium

The assembled consortium combines a robust understanding of the subject matter with the network
necessary for executing the study outlined in the call for services to – and beyond – specification.
Ecorys and VVA both have access to extensive (pan-European) networks, and are thus well-
equipped to offer services such as deliverable (document) translation and original-language
stakeholder interviews at a high quality level and at a competitive price point. The organisations
which comprise the consortium (Ecorys, VVA) have a proven track record in the activities that they
will take ownership of over the course of the contract implementation phase. As a result, the
consortium can offer outstanding performance when it comes to project management of large,
multi-task EU projects and cost & risk analysis, with intimate knowledge of the UCPM being offered
partially through Ecorys’ project previous project portfolio, and partially through the introduction of
various key experts (which will inform all steps of the data collection process). Combined, this
knowledge base covers not only the theoretical suitability of the UCPM capability goals; it also
incorporates experience that is of relevance when it comes to assessing suitability from an
operational standpoint (against the background of both current-and-future threats).

To reduce the chances of conflict within the consortium and to reduce risk (both on behalf of the
client and of the consortium), parties’ roles have been clearly predefined during the proposal-writing
phase. Ecorys will handle desk research and project management while VVA will handle cost
analysis. The two organisations will coordinate the tag-teaming data collection activities. The inter-
task management structure is designed to ensure all synergies between project task teams are

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 11
seized upon. A wide range of ‘milestone’ deliverables (which feed into the activities of other tasks)
have been conceptualised during the proposal-writing process accordingly.

1.3 Background of the study

The European Union Civil Protection Mechanism (UCPM) was established in order to promote
operational cooperation in regard to the significant increase in the number of natural and man-made
disasters. Its creation aims to promote solidarity, as well as to support, complement and facilitate
the coordination of Member States (MS) action in the field of civil protection with a view to
improving the effectiveness of this system.1

For a proper analysis of the UCPM and possible capacity gaps a good understanding of several
aspects of the civil protection in the EU and the UCPM are needed:
 Understanding of the objective and functioning of the UCPM;
 Understanding of the competences and responsibilities of the EU and MS under UCPM;
 Understanding of the available response capacities under the UCPM;
 Understanding of appropriateness of the currently available resources;
 Understanding of existing gaps in response capacities in the UCPM;
 Understanding of existing cost-effective and appropriate new response solutions.

1.3.1 Genesis of the European Union’s civil protection system


According to Article 4(4) of the Treaty on the Functioning of the European Union (TFEU), in the
development cooperation and humanitarian aid the EU “shall have the competence to carry out
activities and conduct a common policy; however, the exercise of that competence shall not result
in Member States being prevented from exercising theirs”. Thus, the EU role in the area of civil
protection aims to leverage the efforts undertaken by the Member States. In this regard, Article 196
of the TFEU stipulates that the EU “shall encourage cooperation between Member States in order
to improve the effectiveness of systems for preventing and protecting against natural or man-made
disasters”. Furthermore, the Treaty provides the scope of the Union’s action in this field, which
shall aim to:
 support and complement Member States' action at national, regional and local level in risk
prevention, in preparing their civil-protection personnel and in responding to natural or man-
made disasters within the Union;
 promote swift, effective operational cooperation within the Union between national civil-
protection services;
 promote consistency in international civil-protection work.2

Moreover, Article 214 of the TFEU specifies that the humanitarian aid operation, shall be conducted
according to the principles and objectives of the external action of the Union, which should provide
“ad-hoc assistance and relief and protection for people in third countries who are victims of natural
or man-made disasters, in order to meet the humanitarian needs resulting from these situation”3.
To this aim the European Commission may undertake “useful initiative to promote coordination
between action of the Union and those of the Member State, in order to enhance the efficiency and
complementarity of Union a national humanitarian aid measure”.4

1
Recital 1 of the Decision N°1313/2013/EU.
2
Art. 196(1) of the TFEU.
3
Art. 214 of the TFEU.
4
Art. 214(6) of the TFEU.

12 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
The UCPM was instituted by the Council Decision 2001/792/EC and it aimed to “facilitate reinforced
cooperation in civil protection assistance interventions” in the Community. 5 The initial structure of
the mechanism consisted of a Monitoring and Information Centre and a Common Emergency
Communication and Information System.6 Nevertheless, this first embodiment of the Mechanism
based on ad-hoc approach was limited and not sufficiently effective. Its scope was further
expanded with the adoption of the Lisbon Treaty, which called for improving solidarity between the
EU and its Member States in the field of civil protection coordination with the view to improving the
effectiveness of the system for preventing, preparing and responding to natural and man-made
disasters. The implementation of this new system was supported by the Commission Implementing
Decision 2014/762/EU laying down the rule for the implementation of Decision N°1313/2013/EU.
The main innovation of the reviewed UCPM has been the creation of the voluntary pool of available
capacities under European Emergency Response Capacity, which should be registered by the
Member States and maintained in a stand-by with the aim to be deployed if the need for EU action
occurs.

1.3.2 Functioning of the European Union Civil Protection Mechanism

Main objective and structure of the UCPM


The objective of the UCPM is twofold. First of all, it aims to reinforce the cooperation between the
EU and its Member States as well as to facilitate coordination to improve “the effectiveness of
systems for preventing, preparing for and responding to natural and man-made disaster.7 Secondly,
the mechanism aims also at promoting solidarity among Member States through practical
cooperation.8

To achieve the above-mentioned objective the UCPM is composed of the following three structures:

 The Emergency Response Coordination Centre (ERCC) replaced the Monitoring and
Information Centre and plays the key operational role for the UCPM. The EERC is the Brussels
based hub, which monitors emergencies across the world. The ERCC is in charge of collecting
and analysing real-time information on disasters, monitoring hazards, making available pre-
committed response capacities, preparing plans for the deployment of experts, teams and
necessary equipment.9 These tasks enable the ERCC to coordinate actions between Member
States, EU Institutions, dispatched field experts and affected countries. Thus, ERCC offers a
more coherent and consistent European response for emerging disasters. Furthermore, each
Member State designates its own national contact point which will liaise with the ERCC.
 Common Emergency Communication and Information System (CECIS), which is
supporting the ERCC in its activities. CECIS is a web-based platform in charge of sending and
receiving alerts and notifications of required assistance. The system enables real time
communication between ERCC and Member States on alerts, notifications and available
capacity to be deployed.
 European Emergency Response Capacity (EERC) is supporting the ERCC role when it
comes to the deployment of capacities through its ‘voluntary pool’. The EERC represents one of
the largest innovations of the Decision 1313/2013/EU and aspires to improve the effectiveness
of European civil protection response. The creation of the ‘voluntary pool’ aims to group relief

5
Recital 3 of Decision 2001/792/EC.
6
Ibid.
7
Recital 1 of the Implementing Decision 2014/762/EU
8
Recital 3 of the Decision 1313/2013/EU
9
European Commission, Factsheet : Emergency Response Coordination Centre,

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 13
teams, experts, equipment etc., which could be made available for deployment by the
Participating Countries.10 In addition, the participating countries can benefit from the EU
financial support to upgrade their national services, as well as to pay for certification and
training, and cover up to 85% of costs related to the transport of teams and assets to disasters
areas.11 The Commission together with the ERCC defines the key capacities needed to ensure
the required action. Furthermore, in order to guarantee the quality of the available services, a
special certification and registration system was created with the Decision 1313/2013/EU and
Implementing Decision 2014/762/EU for evaluation of response capacities provided by
participating countries.12 The EERC appears to be a successful innovation, as by September
2018, 23 of the participating states have made 100 capacities available for deployment.

These structures contribute to the UCPM’s 3 main axes by supporting EU Member States’ ability to
execute operations in the area of disaster prevention, preparedness, and response.

Criteria for activation of the UCPM


The UCPM is under the responsibility of DG ECHO. However, while deploying the mechanism DG
ECHO works also in cooperation with other relevant Directorates such as DG MOVE, DG DEVCO
(e.g. Copernicus), DG SANTE, DG ENV (e.g. Flood Directive and Seveso Directive), DG CLIMA
(e.g. climate change adaptation) DG HOME (e.g. security and terrorism threats) as well as EEAS.
The European External Action Service (EEAS) provides the UCPM operations with the relevant
information on the situation of third countries, which is crucial for missions’ deployment. In addition,
DG ECHO works also in a close cooperation with the Joint Research Centre, in particular with its
Disasters Risk Management Knowledge Centre. Currently, there are 34 countries participating in
the UCPM; namely: the 28 EU Member States, Norway, Iceland, Serbia, Montenegro, Macedonia,
and Turkey.13

The deployment of the UCPM can cover two types of risks. First of all, according to Article 13 of the
Decision 1313/2013/EU, the UCPM can be activated “in the event of disaster within the Union, or of
an imminent disaster, which causes or is capable of causing trans-boundary effect or affect another
Member State”. Secondly, according to Article 16 of the 1313/2013/EU, the UCPM can be equally
deployed if “disaster occurs outside of the Union or is imminent”14. In addition, the assistance can
be requested trough the United Nations and its agencies as well as through other important
International organisations.15 Moreover, while dealing with outside cataclysms, the Union’s action
can be conducted autonomously or can contribute to an intervention led by a given international
organisation.16

In each of the cases, the request for assistance from the affected country is needed, without delay
and should be notified to the European Commission via the ERCC. The notification enables the
ERCC and the Commission to contact the relevant Member States and services about the eventual
capacity needs.17 Thus, the deployment of the UCPM can only happen on a request and cannot be

10
Article 11 of the Decision 1313/2013/EU
11
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
12
Article 16 of the Implementing Decision 2014/762/EU
13
http://europeanmemoranda.cabinetoffice.gov.uk/files/2017/03/Explanatory_Memorandum_6448_17.pdf
14
Article 16 of the Decision 1313/2013/EU.
15
Ibid.
16
Article 16(1) of the Decision 1313/2013/EU.
17
Article 14 and 16 of the Decision 1313/2013/EU.

14 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
activated unilaterally neither by the European Commission nor by Member States. Nevertheless,
the UCPM can be activated at the request of an EU Member State for support when it comes to
“providing consular assistance to EU citizens affected by disasters outside the Union”.18 In the past
years, the EUPCM has been more frequently activated for disasters taking place outside the EU’s
territory than for disasters within its borders. Out of 224 requests for assistance registered between
2002-2015, over 63% were submitted by non-EU countries.19 The proportion of requests for
assistance through the UCPM is presented in the figure below.

Figure 1: Requests for assistance through the UCPM (inside and outside the EU)

Source: Court of Auditors and European Commission.

Successful application of the UCPM


Since its revival in 2013, the UCPM supported by the ERCC and EERC played a successful role in
the resolution of a wide range of crisis within the EU as well as outside its territory. For the following
notable incidents, support was provided:
 Ebola crisis in West Africa (2014);
 Forest fires in Greece (2015);
 Earthquake in Nepal (2015);
 Forest fires in Cyprus, France and Portugal (2016);
 The Ecuador earthquake (2016);
 The Yellow Fever outbreak in the Democratic Republic of the Congo (2016);
 Hurricane Matthew in Haiti (2016);
 Forest fires in Portugal and Greece (2017);
 Forest fires in Sweden and Greece (2018).

Recent developments
Some shortcomings have been observed during previous missions. The Communication from the
Commission on Solidarity with Responsibility from November 2017, emphasises that the “incentives
for Member States to offer assistance via UCPM are very low” due to the EU budget financing only
a share of transport costs and not the operational ones.20 Thus, the EU called for a more “robust

18
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
19
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
20
European Commission, Communication from the Commission, Strengthening EU Disasters Management : rescEU
Solidarity with Responsibility, November 2017, available at : https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2017:773:FIN%20.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 15
and comprehensive EU disaster capacity”. In this regard, between 2017 and 2018, important works
were undertaken to further improve the effectiveness of the UCPM. In this regard and in the
framework of the Commission’s political priorities for 2015-2017, the Commission put forward an
ambitious new plan ‘rescEU’ to reinforce the EU civil protection response by two actions:

 To emphasise the need of solidarity and reinforce EU and its Member States’ collective ability to
response to disasters (in particular addressing current emerging capacity gaps) by establishing
an EU level special response capacity, the European Civil Protection Pool building on the
already existing EERC (the ‘voluntary pool’) , which could be immediately deployed, command
and controlled by the European Commission.21,22
 To strengthen the effectiveness of prevention as a part of disaster risk management and
reinforcing links with other key EU policies acting in the field, making risk prevention a core
element of disaster risk management supported by the investment in prevention and
preparedness monitoring.

Furthermore, it has been also suggested to reinforce and back up the current incentive structure of
the UCPM, by creating one category of response assets, which would be co-financed by the EU’s
budget and the Members States in the framework of the European Civil Protection Poll.23 The co-
financing from the EU budget should cover the expenses related to adaptation, reparation, transport
and running of operations of the pre-committed response capacities of Member States.24 The figure
below visualises the structure of the new Commission proposal, composed of two main elements:
response and prevention/preparedness.

Figure 2: New plan to strengthen EU's civil protection response

Source: European Commission

In order to implement the ambitious Commission’s plan, in November 2017, the European
Parliament and the Council put forward a proposal for a Decision amending the current legal

21
Proposal for a DECISION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Decision No
1313/2013/EU on a Union Civil Protection Mechanism, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2017:772:FIN.
22
RescEU will be equipped with selected emergency capacities to respond to wildfires, floods, earthquakes and health
emergencies as appropriate.
23
European Commission, Communication from the Commission, Strengthening EU Disasters Management : rescEU
Solidarity with Responsibility, November 2017, available at : https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2017:773:FIN%20.
24
Ibid.

16 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
framework established by the Decision 1313/2013/EU that would be in line with recent
Commission’s ambitions.

Additionally, the Commission adopted the Implementing Decision EU/2018/142, which amends the
list of defined capacity goals (Annex III to the Implementing Decision 2014/762/EU) and the quality
and interoperability requirements as well as registration and certification procedure for EERC
(Annex II to the Implementing Decision 2014/762/EU).25 The main changes are related to the
shortage of emergency medical team and health related interventions.26

1.3.3 Main features of current response capacities


The creation in 2013 of the European Emergency Response Capacity resulted in a predictable
coordination and planning of the available resources in the disaster planning. In order to create an
appropriate pool of available capacities, the Commission, based on the risk identified in the
National Risk Assessments, defines “the types and number of the key response capacities required
for the EERC, which are called the capacity goals.27 Subsequently, these capacity goals should be
reassessed every two years by the Commission and Member States and revised if necessary. In
this regard, the national assessment of risks is a key element that enables the EU to estimate the
needed capacities. The estimation of the capacity goals can be also backed up by the Joint
Research Centre publications relevant in the area of civil protection.

The Annex III of the Implementing Decision 2014/762/EU specified the needed capacity goals and
grouped them into three categories: modules, Technical Assistance and Support Teams (TAST),
and other response capacities.

A “module” is defined as “a self-sufficient and autonomous predefined task, and needs-driven


arrangement of Member States capabilities or a mobile operational team of the Member States
representing a combination of human and material means that can be described in terms of its
capacity for intervention or by the task(s) it is able to undertake”.28 Initially a list of 17 modules was
established in the Annex III to the Implementing Decision 2014/762/EU, which was further extended
to 21 modules with the amendment of the Implementing Decision (EU) 2018/142 in 2018. The new
4 modules were created to cover the needs in the field of emergency medical teams. The table
below presents the list of existing modules within the EERC.

Table 1.1: Existing modules of capacity goals


Capacity goals modules
 High capacity pumping (HCP)
 Medium urban search and rescue — one for cold conditions (MUSAR)
 Water purification (WP)
 Aerial forest fire fighting module using planes (FFFP)
 Advanced medical post (AMP)
 Emergency Temporary Camp (ETC)
 Heavy urban search and rescue (HUSAR)
 CBRN detection and sampling (CBRNDET)

25
Commission Implementing Decision (EU) 2018/142 of 15 January 2018 amending Implementing Decision 2014/762/EU
laying down rules for the implementation of Decision No 1313/2013/EU of the European Parliament and of the Council on
a Union Civil Protection Mechanism (notified under document C(2018) 71), available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?qid=1530262669595&uri=CELEX:32018D0142.
26
Ibid.
27
Article 14 of the Implementing Decision 2014/762/EU
28
Article 4(6) of the Decision 1313/2013/EU.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 17
Capacity goals modules
 Ground forest fire fighting (GFFF)
 Ground forest fire fighting using vehicles (GFFF-V)
 Advanced medical post with surgery (AMP-S)
 Flood containment (FC)
 Flood rescue using boats (FRB)
 Medical aerial evacuation of disaster victims (MEVAC)
 Field hospital (FHOS)
 Aerial forest firefighting module using helicopters (FFTH)
 Emergency medical team type 1: Outpatient Emergency Care — fixed (EMT type 1 fixed)
 Emergency medical team type 1: Outpatient Emergency Care — mobile) (EMPY type 1 mobile)
 Emergency medical team type 2: Inpatient Surgical Emergency Care (EMT type 2)
 USAR in CBRN conditions (CBRNUSAR)

In addition to the modules and TAST, 22 categories of other policy response capacity were
identified, for example: team of mountain search and rescue experts, mobile laboratories for
environmental emergencies, power generators, team of maritime incident response experts and
many others. It has been specified that modules as well as TAST may be composed of resources
provided by one or more Member States.29 Furthermore, capacity responses provided by the
Member States should comply with the requirement of the Annex II to the Implementing decision of
2014, which defines the characteristics of the capacities and main components needed for their
deployment. In this regard, the response capacity should be understood as “assistance that may be
provided through the Union Mechanism upon request”.30

In 2017, it has been noted that the “target for the number of modules the UCPM was surpassed by
274%, by 650% on TAST and 134% on so-called other capacities”.31

1.3.4 Appropriateness of the existing response capacities in the UCPM


In 2016, the Court of Auditors provided its own assessment on the UCPM coordination in disasters
outside the EU and an Interim evaluation of the UCPM was completed in 2017. The analyses show
that the UCPM seems to be a successful tool when it comes to mobilisation and coordination of
assistance from the Member States in the response to crises inside and outside the EU. Some
important synergies were noted between the UCPM and other instruments available at its disposal
such as the Copernicus Emergency Management System (DG GROW), which provided important
geospatial mapping support during floods in Bosnia and Hercegovina.32,33. It should be noted that
some of these tools and capacities have the potential of being further exploited (as shown by the

29
Article 11 of the Implementing Decision 2014/762/EU.
30
Art.4(10) of the Decision 1313/2013/EU.
31
European Commission, Interim evaluation of the Union Civil Protection Mechanism, 2014-2016 : Final Report, August
2017, available at : https://publications.europa.eu/en/publication-detail/-/publication/eb41bfee-78c3-11e7-b2f2-
01aa75ed71a1/language-en/format-PDF
32
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
33
Copernicus Emergency Management Service – Mapping, available at:
http://emergency.copernicus.eu/mapping/#zoom=2&lat=23.42974&lon=64.03963&layers=0BT00.

18 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
report of the Commission to the European Parliament delivered in February 2017, which analysed
the progress made and gaps remaining in the EERC).34

First, a tangible proof for the success is the exceeding number of response capacities registered
between 2014 and 2016, in comparison to the targeted number of modules according to the
Implementing Decision 2014/762/EU.35 However, seven modules such as aerial forest firefighting
using planes, and Emergency Temporary Camp did not meet the capacity goals. Furthermore,
weaknesses were noted in the area of chemical, biological, radiological and nuclear disasters,
medical corps, remotely piloted aircrafts systems and communication teams.36 The figure below
compares the EERC capacity goals with resources registered between 2014 and 2016.

Figure 3: EERC capacity goals compared with resources registered

Source: ICF study, Interim evaluation of the Union Civil Protection Mechanism 2014-2016.

Capacity shortcomings
Regarding the firefighting planes, the events of forest fires in the summer 2016 in Portugal, showed “the
operational necessity and political significance” of having more firefighting planes available in the
framework of EERC.37 When it comes to the shelters and related assistance, the capacity goals has not
been reached in this field under EERC. Additionally, during the peak of the refugee’s crisis, when
numerous Member States where facing the same challenges, shortcomings of the EU civil protection
mechanism became clearly apparent. Thus, the Commission identified shelters and related assistance as
“being a potential significant gap”.38

Secondly, it has been noted that while some resources are sufficiently available, their deployment
possibilities are weak. This is notably the case for the occurrence of chemical, biological,
radiological and nuclear disasters as well as the deployment of big field hospitals, medical
evacuation capacities, remotely piloted aircrafts system and finally communication teams.39

34
European Commission, Report from the Commission to the European Parliament and the Council on progress made and
gaps remaining in the European Emergency Response Capacity, November 2017, available at :
https://ec.europa.eu/transparency/regdoc/rep/1/2017/EN/COM-2017-78-F1-EN-MAIN-PART-1.PDF
35
European Commission, Interim evaluation of the Union Civil Protection Mechanism, 2014-2016 : Final Report, August 2017,
available at : https://publications.europa.eu/en/publication-detail/-/publication/eb41bfee-78c3-11e7-b2f2-
01aa75ed71a1/language-en/format-PDF
36
Ibid.
37
Ibid.
38
Ibid.
39
Ibid.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 19
Deployment shortcomings
In the light of terrorist attacks in Europe, it seems that the EERC does not have enough response
capacities for rescue operations in contaminated environment and treatment of patients exposed to
Chemical, Biological, Radiological and Nuclear (CBRN) agents.40,41. During the Ebola outbreak in West
Africa, shortcomings regarding the evacuation services for medical staff and humanitarian aid workers and
regarding emergency medical teams and other health related intervention surfaced.42 For instance, the
available medical resources were dedicated to general service for aerial evacuation rather than to infected
medical staff.43

Thirdly, it turns out that the Member States lack incentives to deploy the available capacities. The
EU budget is co-financing only the share of transport costs which, in comparison to the operational
costs, are rather low. For instance, the cost of large field hospital transportation constitutes only a
very small part of the costs required to run such hospitals in field, which is approximatively EUR 6
million per month.44 The lack of co-financing for the operating costs of response capacity
deployment is one of the very important gaps of the UCPM.

Lastly, it seems that administrative burden hinders the immediate activation of the UCPM. The
recent report of the Court of Auditors has emphasised that although the Commission transferred the
request for assistance in a timely manner, the mobilisation of the available resources was not
efficient when it comes to activation of the pre-alert phase, availability of the ‘voluntary pool’ or
nomination of the EU coordination team on-the-ground.45 In this regard, in October 2015 a very
slow development of the pool of response capacities was noted due to the “administrative burden
involved in registration and certifying response units/modules”.46 The figure below visualises the
time that has elapsed since the occurrence of the disasters and request for assistance, to the
request in CECIS for nomination of experts.

40
Ibid.
41
The recent Commission Communication from October 2017 Action Plan to enhance preparedness against chemical,
biological, radiological and nuclear security risks emphasised that although the fact that the terrorist organisation have not
used CBRN agents in Europe yet, there “there are credible indications suggesting that terrorist groups might have the
intention of acquiring” such materials or weapons and “develop knowledge capacity to use them”.
42
Op. cit. European Court of Auditors and Implementing Decision (EU) 2018/142.
43
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
44
European Commission, Communication from the Commission, Strengthening EU Disasters Management : rescEU Solidarity
with Responsibility, November 2017, available at : https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2017:773:FIN%20.
45
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
46
Ibid.

20 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Figure: Timeline for selection of EUCP experts in Bosnia and Herzegovina

Source: European Court of Auditors

Some of the above-mentioned shortcomings like the lack of incentives are already being
addressed47, further solutions would need to be based on “the recent economic market-based
information in the field of the development, deployment, operational and maintenance costs of the
capacities used in response to disasters”.48 To this end, innovative solutions could also be based
on the approach undertaken by other international organisations which are dealing with
management of the civil protection mechanism.

1.3.5 Available new response solutions


In a world of increasing variety of risks linked to unpredictable extreme disasters, and terrorist
attacks, the EU response to the emergencies should be further improved and constantly adapted.
The section five of this Chapter has identified some existing shortcomings of the UCPM. It appears
that the Union civil protection response should further adapt its mechanism to more specific needs.
To this aim, the definition and typology of capacities need to be further developed. The first steps in
this area have already been taken with the adoption of the Implementing Decision (EU) 2018/142.
However, the list of the capacities should be revisited regarding the development of additional
capacities related to requirements and in line with potential future disaster risks inside and outside
the EU territory. This review of the capacity definitions should further be backed up with the
upgrading of the national risk assessment and their accuracy.

As has been noted, Member States suffer from a lack of incentives to deploy their response
capacities. Consequently, a horizontal financing support from the EU budget could improve the
availability of registered capacities considerably. To this aim, a detailed evaluation of costs should
be further developed according to different scenarios for each module and category of capacity.

The deployment of the UCPM should not be seen in isolation from other existing initiatives in the
field of civil protection and its efficiency could be improved through the better coordination with
other complementary interventions, actions, tools, and services provided by the EU, Member
States, non-EU countries and international organisations. Furthermore, important lessons can be
derived from the performance of other international organisations specialised in the field of risk
management and civil protection such as the OECD’s Development Assistance Committee
(DAC)49, United Nation Office for Disaster Risk Reduction (UNISDAR)50 or the WHO’s Global

47
See sub-section Recent Developments und section 1.3.2.
48
European Commission, Report from the Commission to the European Parliament and the Council on progress made and
gaps remaining in the European Emergency Response Capacity, November 2017, available at :
https://ec.europa.eu/transparency/regdoc/rep/1/2017/EN/COM-2017-78-F1-EN-MAIN-PART-1.PDF.
49
http://www.oecd.org/development/developmentassistancecommitteedac.htm
50
https://www.unisdr.org/

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 21
Outbreak Alert and Response Network (GOARN) 51 or United Nations High Commissioner for
Refugees (UNHCR).

1.4 Purpose and scope of the study

In order to provide efficient civil protection to natural and man-made disasters, the European
Commission is in charge of monitoring and evaluating the results of the UCPM implementation. To
this end, this study’s purpose is to deliver an evidence-based analysis into response capacity gaps
while taking into account current risks and needs in the area of civil protection. Additionally, the
study allows to refine capacity definitions and formulate options based on capacity costs by
providing a cost analysis of development, deployment, operational and maintenance costs of the
capacities used in response to disasters. Together, these results will feed into DG ECHO’s
monitoring of the UCPM’s progress in achieving its capacity goals.

As stated in the terms of references, the study’s scope covers the UCPM, is functioning and the
capacities provided by its participating states. It shall:
 Review the general requirements of all modules (excluding aerial forest firefighting) and
propose a revised version for outdated definitions and requirements;
 Develop general requirements for response capacities in light of changing or emerging risks
(including quality and interoperability requirements);
 Conduct a cost analysis, based on the (revised) definition of response capacities, and include a
cost breakdown in terms of equipment and human resources;
 Assess the progress that has been made towards achieving the capacity goals. In particular, the
study shall evaluate whether sufficient capacities have been registered in the EERC and
whether capacity goals are set at the appropriate level in light of risks;
 Propose, if needed, a revision of the capacity goals.

Chapter Two outlines how we propose to fulfil the purpose and requirements of this study, starting
with our general approach and then highlighting tools and methodologies to be used per task.

51
http://www.who.int/ihr/alert_and_response/outbreak-network/en/

22 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 23
2 Methodology

2.1 General approach

We propose a holistic approach to this study that enables the best available data and deep insights
from country experts and relevant stakeholders to be extracted. Our approach ensures that
necessary inputs to conduct the analysis under each task are obtained. We see the tasks as being
interlinked with each other and requiring feedbacks and inputs from each other in order to be
accomplished successfully. In the following section, we describe how the tasks are interlinked and
what methods are proposed to complete each task.

2.1.1 How the tasks are interlinked

The interlinkages between the different tasks are visualised in the following figure.

Figure 4 Linkages among Tasks

Source: Ecorys

Task 1 analyses the appropriateness of existing response capacities in light of past experiences
and the most recent risk assessments. This analysis will provide a precise definition of the
response capacities (aka modules) for which a cost analysis will be conducted in Task 2.

24 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
The priority of Task 1 is to identify capacities that can be categorised as category A or B modules
so that they can be subjected to cost analysis of Task 2 immediately. For capacities identified as
category C or D modules, further research needs to be conducted. For category C modules
definitions and quality requirements need to be revised. For category D, general requirements,
including interoperability, need to be developed from scratch. Only then, equipped with new
definitions, categories C and D modules can be analysed under Task 2.

At the same time, cost analysis under Task 2 may have impact on the categorisation of capacities
and development of definitions and requirements under Task 1. Results of the risk assessment in
Task 3 will be considered when examining capacities under Task 1, and review of capability goals
under Task 4 needs to be taken into account by Task 1 as well.

Task 2 involves a cost analysis of capacities that depends on their categorisation under Task 1.
Category A and B capacities do not need to be reviewed, and the cost analysis for these modules
can also be conducted early in the project. For category C and D capacities, the cost analysis can
start only after (revised) definitions and quality requirements are approved by DG ECHO.

As explained above, Task 2 is dependent, and in fact triggered, by Task 1. In addition, the
outcomes of the risk assessment in Task 3 and the review of capability goals in Task 4 may have
implications for cost analysis and need to be taken into account in Task 2. Therefore, Task 3 and
Task 4 feed back into Task 2.

Task 3 is comprised of a risk assessment and a capacity gap analysis. Implementation of this task
can start immediately after the signature of the contract. Based on the risk assessment, existing
response capacities need to be reviewed to asses if they correspond to the EU’s internal and
external risk landscapes. Accordingly, outputs of Task 3 will inform Task 1 and impact Task 2 (cost
analysis) and Task 4 (review of capability goals). Where it is found that identified risks – particularly
new emerging risks – are not adequately addressed by the current available capacities or capacity
goals, the study will formulate recommendations to revise the specification of capacity
requirements. Furthermore, this task will develop definitions of new typologies of capacities when
these are considered a potentially significant capacity gap.

Task 4 is geared towards creating a methodological framework through which (i) the suitability of
existing capability goals can be systematically assessed and (ii) the relative importance of existing
and newly formulated capacity goals can be identified. The activities under Task 4 rely on inputs
from Tasks 1, 2, and 3, and feed into the 2nd and 3rd rounds of interviews which take place at the
end of Task 1. The Task team will conduct extensive desk research and/or analyse Task 3 results
in order to prepare interview material to ensure that collected results provide them with sufficient
input to conceptualise (and apply) the methodological framework. In addition to a methodological
framework, an overview of capacity goal suitability, and a prioritisation of capacity goals, Task 4
may result in the drafting of a request to national contact points and a list of omitted and obsolete
capacity goals.

2.2 Proposed tasks and activities

An overview of the specific activities related to the main tasks is provided below.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 25
Table 2.1 Overview of tasks and subtasks
Task and subtasks Deliverable
Task 0 Project management
Task 0.1 Project management Deliverable 1: Inception Report
Deliverable 7: Final Report
Task 0.2 Quality control
Task 1 Review existing capacities
Task 1.1 Desk research
Task 1.2 Stakeholder consultations via interviews
Task 1.3 Stakeholder consultations via survey (targeted Deliverable 2: survey questionnaire
consultations)
Task 1.4 Categorisation of modules of response capacities
Task 1.5 Recommendations for (revised) definitions and quality Deliverable 3: report of the survey and desk
requirements research
Task 2 Cost analysis
Task 2.1 Understanding data needs
Task 2.2 Data collection
Task 2.3 Defining cost adjustments and scenarios
Task 2.4 Cost calculation Deliverable 4: Cost Analysis
Task 3 Risks and gap analysis
Task 3.1 Risk data collection and preliminary analysis
Task 3.2 Response Capacity Gap Analysis
Task 3.3 Validation workshop
Task 3.4 Develop recommendations Deliverable 5: Risk based capacity gap analysis
Task 4 Capacity goals revision
Task 4.1 Preparation of interview material
Task 4.2 Development of a methodological framework for
assessing suitability of capability goals
Task 4.3 Application of the methodological framework Deliverable 6: Capacity goals revision

2.3 Proposed methodology

We propose to employ a suite of methods to obtain the necessary data and information required for
the four core tasks. Data collection is an important challenge of this project because all project
tasks rely on a large amount of data obtained through different methods and from a wide variety of
sources. In addition, the tasks are interlinked with each other and feed into each other, and they
have to be performed within a limited period. With this in mind, we need to ensure that analysts
have the right data in the right format and in the right time. Therefore, it is crucial that the data and
information from the start are collected and stored with the requirements of all project tasks in mind.

The following methods of data collection are envisaged in the project:


 Desk research
 Interviews
 Targeted survey
 Validation workshop

26 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
We see data collection as a horizontal activity that is informed by data needs of all analytical tasks
(Tasks 1-4) and that feeds the necessary inputs back into them. In the sections below we discuss
every method of data collection separately and explain in detail how it will be used in the context of
this study. The subsequent task descriptions (starting with Section 1.5 forth following) provide task-
related details on the data collection methods applicable for each task.

2.3.1 Data collection


Given the fact that data collection plays a significant role in the project at hand, the project team
anticipates large amounts of data to be processed and stored. To that end, the following working
steps will be conducted to ensure an appropriate data storage and data flow in the project:
1. Assessment of data needs: the initial assessment of data needs is carried out in this proposal.
At the beginning of the project (inception phase), task leaders will review the proposal and
provide an updated information on the necessary data, including data formats and timing.
2. Data collection strategy: On the basis of the data needs, we develop an overall strategy for all
data collection activities. An initial data collection strategy is presented in this chapter of the
proposal. During the inception phase of the project and in cooperation with the Client, we will
review and render this strategy more precise by refining the list of data sources (including
stakeholders and interview candidates), data formats, timing etc. We will also design an internal
database for the analysts that includes a template table for the assessment of collected
literature and an excel structure for assembling the data.
3. Data check: The dedicated data collection team will ensure that the information needed is
delivered and uploaded in time into the database and is available to the other project members
(analysts). The team will also check the uploaded information and compare them with the data
needs and the requirements to data format.

Desk research/literature review

Week Week 1 – 17
Week 25 - 30

Tasks Task 1, 2, 3, 4

The study team and the national correspondents will both be involved in desk research. The goal is
to identify and study relevant documents related to the UCPM. In the case of the country
correspondents, desk research will be geared specifically towards collating the background
information necessary to optimise the interview process. Additionally, national correspondents will
identify and study those documentary sources that are only available in national languages of
UCPM Participating Countries. The types of sources that will be included in the analysis are:
government reports legislation, standards, databases, reports from conferences, -workshops, -
trainings, minutes of meetings of relevant committees and reports on civil protection exercises.
They can be structured in varying degrees according to the scope of the review. An illustration of
the types of data to be collected in relation to the potential documentary sources and databases
that would supply such data is shown in the table below.

Exemplary list of documentary sources to be reviewed


European Commission
 Commission Implementing Decision 2014/762/EU of 16 October 2014 laying down rules for the
implementation of Decision No 1313/2013/EU of the European Parliament and of the Council on a Union
Civil Protection Mechanism

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 27
 Repealing Commission Decisions 2004/277/EC, Euratom
 2007/606/EC, Euratom
 Commission Staff Working Document (2017). Overview of Natural and Man-made Disaster Risks the
European Union may face (SWD(2017) 176 final of 23.5.2017)
 Report from the Commission to the European Parliament and the Council on progress made and gaps
remaining in the European Emergency Response Capacity, COM(2017) 78 final of 17.2.2017
 DG ECHO Annual activity reports (2017)

United Nations
 USAR (INSARAG)
 Directories of USAR (INSARAG)
 UNISDR (2017) National Disaster Risk Assessment: Governance System, Methodologies and Use of
Results

World Health Organisation


 Classification and minimum standards for foreign medical teams in sudden onset disasters of 2013
 Directories of EMT (WHO)

National authorities
 National risk assessments from member states

OECD
 National Risk Assessment: A cross country perspective, OECD, 2018

External evaluations
 Peer reviews of UCPM countries (6 completed in 2015-2016)
 European Court of Auditors (2016). Union Civil Protection Mechanism: the coordination of responses to
disasters outside the EU has been broadly effective. Special report No 33
 Special Eurobarometer 454 (2016-2017). Civil protection
 ICF (2017). Interim evaluation of the Union Civil Protection Mechanism, 2014-2016
 ICF (2018). Comprehensive evaluation of the European Union humanitarian aid, 2012-2016 (included
UCPM in the evaluation of coherence)

Furthermore, a number of databases will be of use in the light of this project:


 EDRIS (European Emergency Disaster Response to identify EU humanitarian aid
contributions)52
 EU Aid Explorer53
 CECIS (Common Emergency Communication and Information System)
 Marine Pollution CECIS
 USAR directory54
 EMT database55
 Eurostat

52
https://webgate.ec.europa.eu/hac/
53
https://euaidexplorer.ec.europa.eu/HumanitarianDonorAtlas.do
54 See for the European region: https://vosocc.unocha.org/USAR_Directory/USARTeamsByregion.asp .

55 See for the European region: https://extranet.who.int/emt/emt-profile/180

28 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Targeted consultations approach
Engaging a wide range of stakeholders into this study is necessary to obtain high-quality evidence,
conduct analysis and revise prevention and disaster response capacities under the UCPM. It will
allow to capture their different specialist views, contribute to greater transparency and
accountability in the present review, and in the overall ongoing EU legislative process. It is also
crucial to ensure successful balance between valuable expert opinion and the undue promotion of
special interests.

For this assignment it is therefore essential to identify and engage all relevant stakeholders in the
consultation process, including not only those involved in the preparation and implementation of
prevention and disaster response actions, but also those affected by them and those directly
interested in them across all relevant sectors at European, national, regional and local level. It is
also of utmost importance to carry out an effective promotion of the stakeholder consultation
activities to achieve a satisfactory response from a representative number of stakeholders from all
sectors and Participating States, and thus provide relevant evidence-based inputs to feed into the
study.

Our approach for the targeted consultations takes into account the specific requirements for this
service request and adheres to the General Principles and Minimum Standards set out in the Better
Regulation Guidelines56, notably:

General principles
 We will adopt an inclusive and transparent approach by consulting as widely as possible.
 We will make the consultation process - and how it is expected to affect policy making -
transparent to those involved.
 We will consult at a time where stakeholder views can still make a difference, respect
proportionality and specific restraints.
 We will ensure the consistency of consultation processes and include quality control
mechanisms.

Minimum standards
 We will produce a clear, concise and complete content for the engagement and consultation
process.
 We will ensure that all relevant parties have an opportunity to express their views.
 Adequate awareness-raising publicity (in cooperation with EC) will be also ensured.
 We will plan sufficient time for gathering responses and written contributions from the
consultation process
 Last, but not least, we will acknowledge the receipt of contributions and analyse the findings of
the consultation activity, drafting a report.

In addition, our approach is based on the specific principles below:


 Thorough stakeholders mapping and identification of respondents thanks to our wide network;
 Tailored questionnaires for specific audiences and tools (surveys and interviews);
 Use of local languages to improve response, thanks to the availability of national
correspondents;
 Systematic cross-validation of findings through data triangulation.

56
https://ec.europa.eu/info/sites/info/files/better-regulation-guidelines.pdf.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 29
Our approach for will be structured around two logical steps, complemented by a rigorous system
for ensuring the highest quality standards of delivery, which is described in Chapter 4. The steps
are the following:
 Step 1: fine-tune stakeholder consultation strategy
 Step 2: consult and gather evidence

Step 1 Fine-tune stakeholder consultation strategy


At project inception, we will fine-tune with the Client the herewith proposed stakeholder consultation
strategy, which sets the scope of the targeted consultations, identifies the relevant stakeholder
categories, outlines tools and methods, and provides an indicative planning for the consultation
activities, and for gathering other relevant inputs.

Stakeholder mapping
Stakeholder analysis identifies the people, groups and organizations that are important to consider
when looking at the impacts of reforms and is of particular relevance for a studied policy area. To
the purpose of the present study, a “stakeholder” is defined as a party that has an interest in the
UCPM and can either affect or be affected by it, or that can help define value propositions for the
UCPM.

Regarding the identification (or mapping) of stakeholders for the targeted consultations, a
combination of the Commission’s contact lists, our own databases and desk research (described
above) will allow to find the most appropriate “mix” from the categories below:

Table 2.2 Stakeholder categories in relation to consultation objectives


Stakeholder relation to Stakeholder category
consultation objective

Implementing the UCPM  DG ECHO, and other EC Services (incl. DG ENV, DG CLIMA, DG NEAR,
DG HOME, DG SANTE, DG ENER, DG JRC, EU-Agencies (ECDC,
EMSA, EASO, EEA))
 National authorities from Participating States, incl. National Contact Points
and national partners (national civil protection agencies)57
 Regional and local authorities from Participating States
 Business professional organizations (e.g. EoE programme Manager
Exercise provider, Transport provider, etc.)
 DG ECHO international partners, including UN Agencies and other
disaster management-focused international organisations (IFRC, IOM,
etc.)
Affected by the UCPM  Host countries having received UCPM support
Interested in the UCPM  EU Member States (authorities other than Civil Protection)58
 Non-EU Member States Participating States (authorities other than Civil
Protection)
 UN agencies (other than Civil Protection)
 World Bank

57
See Annex I.
58
This includes other government counterparts whose actions are overlapping with activities carried out by the UPCM.
Examples include: Ministries of Social Affairs (e.g. protection of people from the impact of disasters), Ministry of National
Security/Interior (e.g. management of civil protection consequences of security threats), Ministry of Public Works (e.g.
protection of assets), and Ministry of Health (e.g. prevention and management of pandemics).

30 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
 Specialised NGOs and other civil society organisations
 Businesses (incl. SMEs) in the domain of health, climate change
adaptation, disaster risk management, environment, research and
innovation, maritime emergencies, security policies
 External evaluators
 Project leads of prevention and preparedness projects and missions
 Academia and research institutes having expertise relevant to the UCPM
(i.e. civil protection and humanitarian aid) , mainly in Participating States
 Other EU Institutions

This non-exhaustive list may be extended at project inception and during the desk research to
include other stakeholder categories, as considered appropriate by the client.

The table on the next page visualises also the level of interest in and influence on the UCPM of the
several typologies of stakeholders.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 31
Table 2.3 Level of interest and influence of stakeholders

Low interest Medium Interest High interest


Low influence  Other EU institutions  Project leads of prevention and preparedness  Business professional organizations (e.g. EoE
projects and missions programme Manager Exercise provider,
Transport provider, etc.)
 Businesses (incl. SMEs) in the domain of health,
climate change adaptation, disaster risk
management, environment, research and
innovation, maritime emergencies, security
policies
 Specialised NGOs and other civil society
organisations
 Academia and research institutes having
expertise relevant to the UCPM (i.e. civil
protection and humanitarian aid) , mainly in
Participating States
 External evaluators
Medium influence  EU Member States (authorities other than Civil  Regional and local authorities from Participating
Protection) States
 Non-EU Member States Participating States  DG ECHO international partners, including UN
(authorities other than Civil Protection)UN Agencies and other disaster management-
agencies (other than Civil Protection) focused international organisations
 World Bank  Host countries having received UCPM support
(EU, Accession, Neighbourhood, and Third
Countries)
High influence  DG ECHO, and other EC Services
 National authorities from Participating States

32 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
This table will allow us to select the right approach to reach out to the diverse stakeholder groups
and tailor the consultation tools accordingly. Not all stakeholders in a particular group will
necessarily share the same concerns or have unified opinions or priorities. It is therefore
noteworthy that the differentiations in the table above are indicative and might not always be clear
cut, as they could depend on several elements, for instance the national context. We will also pay
particular attention to involve groups that risk being excluded through the use of a tailored mix of
consultation tools, seeking at the same time to avoid biased opinions, ensuring balance and
comprehensive coverage to the consultation.

The target stakeholders will be identified through transparent criteria in agreement with the Client.
We will aim at ensuring an overall balanced geographical and thematic representation of the
stakeholders consulted. Likewise, we will aim at avoiding 'regulatory capture’ by ensuring that the
inputs from stakeholders with vested interests are incorporated into the overall findings of the
consultation activities, but do not receive special treatment. Based on the preliminary results of the
desk research, stakeholders selection might be further customised to the local specific contexts and
need for data.

To widen participation, we also propose to ask representative individuals from each category in a
position to act as multipliers to suggest further contacts and thus compile comprehensive lists.

Stakeholder consultation methods and tools


With regards to the choice of consultation methods and tools, according to the terms of reference,
with the target stakeholder groups agreed by the client, we will undertake the activities below:
 Interviews
 Targeted consultations (survey)
 Validation workshop

Our analysis of previous evaluations of the UCPM, confirms the inherent challenges in stakeholder
consultations in this particular domain, characterised by a large number and variety of parties that
are also geographically very dispersed, especially when it comes to ensuring a high response rate
to survey questionnaires or participation in interviews. In light of the above, we will ensure that all
the proposed consultation methods be carefully tailored to the nature of the relevant stakeholder
groups to ensure maximum openness and accessibility, while avoiding stakeholder fatigue.
Particular attention will also be given to the design of consultation tools according to the
stakeholder’s respective level of engagement and experience with the UCPM.

We also assume that the Client can support us in our efforts to promote participation by issuing a
recommendation letter illustrating the assignment that our team will be carrying out with the target
audiences as well as by providing stakeholder contacts and/ or alerting some of the stakeholders to
this study.

Step 2 Consult and gather evidence


Interviews

Week 1st lot: Week 3- 7 (Task 1, 2 and 3)


2nd lot: Week 13-17 (Task 1, 2, and 4)
3rd lot: Week 27-30 (Task 2 and 4)

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 33
Tasks Task 1, 2, 3, 4

Objectives
Stakeholder and expert interviews will be fundamental for collecting valuable data and information
for conducting this study. With the interviews, we pursue three complementary objectives.
1. The first objective is to yield relevant quantitative data and qualitative information to
complement desk research and form the basis of our analysis, and to inform all project tasks.
2. The second objective of the interviews is to validate/ confirm the data and information obtained
from other sources (for example, to validate whether information received from desk research is
still up-to-date).
3. The third objective is to better understand and put in context data and insights gathered through
the targeted surveys as survey answers may sometimes require further elaboration or when
survey results seemingly diverge with findings based on other sources.

General approach
To reach these objectives, we will conduct about 130 (the precise number of interviews is subject to
the availability of the stakeholders) organised in three rounds:

Interview Goal Modus Nr.


Round
1 These interviews will provide initial data and information for Task 1, 2 and 3. Mostly +/-
They will allow to identify the categorisation of the different capacities, to face-to- 30
validate info gathered through desk research, to close information gaps and face
to identify new sources of data. Furthermore, they will allow for the
validation of preliminary findings on risk mapping and capacity goals and the
validation of findings from analysis of CECIS and ERCC
2 These interviews will provide initial data and information for Task 1, 2 and 4. Via +/-
They will allow for the identification and validation of categorisation different phone 60
capacities, to validate info gathered through desk research and the initial
round of interviews, to close information gaps and to create understanding
of capacities and related costs, technologies used. For Task 4, these
interviews will provide inputs for the creation and application of the
methodological framework for establishing capacity goal suitability.
3 Inform and validate Task 2 and Task 4. Provide contingency inputs to Task Via +/-
nd
4, should those derived from the 2 round of interviews proves insufficient phone 40
for executing the Task.

The stakeholder list defined with the client during the inception phase will be further developed,
drawing on combination of sources:
 Web-searches of relevant stakeholders identified to date;
 Suggestions made at the kick-off meeting by the Commission and initial feedback;
 Results of the desk research and initial interviews (e.g. interviewed experts can recommend
other knowledgeable experts and stakeholders).

The table below contains an indicative list of stakeholder categories to be interviewed.

34 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Table 2.4 Potential groups of stakeholders
Potential stakeholders
 DG ECHO officers (especially from Directorate A, ERCC, EERC);
 Disaster Risk Management Knowledge Centre (DRMKC);
 Emergency Response Coordination Centre (ERCC)
 EU agencies, e.g. European Defence Agency, European Maritime Safety Agency
 Representatives of specialised international organisations, such as EMT (WHO), International Civil
Defense Organisation,
 International organizations (UNISDR),
 Other stakeholders - International Organization for Migration; Organisation for the Prohibition of
Chemical Weapon, World Bank, OECD, ANO, United Nations
 Representatives of specialised NGOs, such as Red Cross, Médecins Sans Frontières etc.
 Decision makers (General Directors)
 Evaluators of related programmes such as Prevention, Preparedness and Response to natural and man-
made Disasters (PPRD) in the Southern Neighbourhood (PPRD South II);
 High level representatives (Civil Protection Committee members)
 Expert group on civil protection modules (Group members)
 Member States' contact points
 National civil protection authorities from Participating States;
 Relevant industries
 Universities and think-thanks

Interview process
Develop the interview protocols and interview questions
As different interviewers will conduct interviews it is essential to put in place a mechanism that
allows for a standardised approach towards the interviews. The project team, therefore, proposes to
utilise interview guidelines that outline the questions that should be asked during the interviews.

We will develop Interview Guidelines specifically tailored to the requirements of this assignment. As
a method, we will mainly rely on Standardized Open-Ended Interview. In this format, the
interviewers adhere to a strict script, and there is no flexibility in the wording or order of questions. It
is still considered a qualitative interview rather than a quantitative interview, because the responses
are open-ended. This is the most structured and efficient of the qualitative interviewing techniques
and is useful for reducing bias when several interviewers are involved, and when it is important to
be able to compare the responses of different respondents. We will however ensure that the
interviewer has flexibility to add some questions tapping into the issues that are most relevant to the
interviewees, in order not to lose this level of details.

These guidelines will differ depending on the moment at which the interview is conducted (Lot 1,
Lot 2 or Lot 3). In Annex II, three examples of interview guidelines are presented. At the moment,
these guidelines are initial indications of what the final guideline would look like. The
recommendations of the Client will be taken into account when developing the final guidelines.

Draft questionnaires will be presented to the Commission for comments. Following discussions with
the Commission, the study team will revise the questionnaires. This exchange is considered very
valuable to finalise the questionnaires and ensure the inclusion of all relevant points.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 35
To ensure that national correspondents conducting interviews have a full understanding of the
questions and objectives, a Guidance Note will be produced and shared. The guidance note will
include the following sections:
 Introduction and objectives of the study;
 List of stakeholders;
 Interview questions;
 Standard steps for performing the interview;
 Guidance for writing interview notes and reporting.

The Guidance Note, the Interview Guidelines and overall objectives will be discussed with the
country correspondents to make sure that the methodology and approach are aligned across
interviewers.

All interviews will be conducted by our experts and country correspondents, all of which preside
over relevant experience and/or language knowledge for the country(ies) they will cover (see
Chapter 3 for a complete overview of the proposed country correspondents). The country
correspondents will follow a standardized procedure to ensure the robustness and comparability of
the study results.

The procedure will differ slightly by interview round, as the 2nd and 3rd rounds will be geared
towards collecting different data and information than the 1st. Also, for 2nd and 3rd rounds of
interviews, interview questionnaires will be developed outlining main topics/ questions for the
discussion with the stakeholder; the questionnaires will be presented to the Client for approval prior
to their introduction to country correspondents. We provided exemplary outline of issues for
discussion during interviews in each round in Annex II. In order to collect an abundance of
information from stakeholders, in the 2nd and 3rd round we will employ semi-structured interviews
that allow us or the interviewee to introduce new issues and ideas relevant for this study. This
interview technique is especially necessary when discussing revised definitions and requirements
with interviewees.

Conduct Interviews
The interviews will be scheduled with the relevant stakeholders. Our team will make sure to contact
the relevant stakeholders well in advance in order to ensure the availability of the stakeholders and
to allow enough time for the organisation of the logistics. During the first contact, we will share with
the interviewees the objectives of the assignment and the nature of the interview, and we suggest
supporting the interview request with a recommendation letter from the Commission, which may
motivate the interview candidate to actual participation. We will also inform interview candidates
that notes will be taken during the interview and how the data and information obtained from them
will be used for the study. We will obtain a prior permission from the interviewee if a specific piece
of information has to be mentioned in the report and published. Once the interview questionnaire is
approved by the Commission, the questionnaire will be shared with the interviewees prior to the
interview. We will do so at least one week before the interview takes place to ensure the
interviewees have time to prepare and gather relevant data required to ensure that all relevant
information is collected.

We recognise that some stakeholders will not respond to us, some will be unable or will prefer not
to hold a face-to-face interview. Where stakeholders do not respond to repeated contact through
emails (and telephone), we must recognise their right not to participate in the study. These

36 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
stakeholders will be invited to partake in the online survey to give them another chance to have
their voice heard in the context of this study.

The interviews will be conducted both face-to-face, via skype and by phone. As noted earlier,
especially during the first round of interviews we will aim to conduct as many as possible face-to-
face interviews. Indicative duration of one interview is 1-2 hours.

We aim to conduct as many as possible interviews in the native languages of interview candidates
as this will allow the interviewee to express him/herself freely and more precise. Interviews in native
languages may also encourage participation of experts from different countries. The consortium
covers at least one of the official languages spoken in each UCPM Participating Country, as is
demonstrated in the project management chapter.

Follow up to the interviews


For each interview, we will prepare notes that record a summary of the responses for each
question, as mentioned above. Interview notes for each interviewee will be gathered in a single
database so that they can easily be analysed using appropriate qualitative analysing techniques.

Interviewees will be informed that in our reporting we will not link a single statement with a specific
organisation. All information will be treated according to our high standards of confidentiality. In our
reporting, we will use views at aggregated levels i.e. country / sector / stakeholder group. Finally,
the information gathered during the interviews will be logged into an interview matrix. This is an
internal supporting tool that ensures that the collected evidence is complete, and provides our team
with a convenient overview of the answers by type of question and stakeholder group.

The analysis will take into consideration the type of respondents and/or the country of origin in
order to make a judgment on whether any patterns or diverging views can be observed. The
techniques used are common to all consultation activities; however, each consultation activity will
be analysed and presented separately in order to ensure transparency and facilitate triangulation of
data. The results will be aggregated and documented in a structured manner.

Survey

Week Weeks 9 - 14

Tasks Task 1, 2

In addition to the interviews described above, we will hold a web-based targeted survey as required
by the tender specifications.

The survey will be launched when the first round of interviews has been completed and, thus, when
the first bulk of data has been collected. The survey will be used to validate and cross-check the
information obtained from the interviews and will specifically target Task 1 and 2. The aim of the
survey is to obtain and verify data regarding the categorisation of the modules, to close the existing
gaps in desk research(that will also been addressed during the interviews), to learn about trends
and challenges in the relevant fields, and to reflect on the costs of the various modules.

Therefore, the design of the survey will take place in parallel to the first round of interviews being
conducted. The interview guidelines developed for the first lot of interviews, the findings from desk

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 37
research, and the information gathered during the interviews, will constitute the basis for tailoring
the survey’s questions. The survey will be designed to avoid bias and to ensure a user-friendly
experience. It will be kept at a reasonable length to avoid fatigue in the respondents and thus
maximise the number of responses.

The survey will contain closed and open questions as well as questions aimed at gathering
qualitative data. Questions will be stated in a clear way in order to ensure that stakeholders
understand correctly and provide information covering adequately the point of each question. This
approach will minimise the risk of misunderstanding, guaranteeing the quality of the data gathering
process.

Our approach to the development of the web-based targeted survey on the UCPM consists of the
following steps.
1. We will first develop key profiling questions mainly related to identify the respondents and to
assess their level of expertise and interest in the several aspects covered by the survey. The
profiling section at the beginning of the questionnaire will also intentionally be kept to a
minimum to reduce the likelihood that the respondents come under stereotype threat59, which
would influence the way they respond to the rest of the survey. On the basis of that profiling, we
will develop customised questions to dig into specific issues, in cooperation with the client. The
structure of the survey will be designed to ensure that the different categories of target
stakeholder can only view those questions which are relevant for their level of interest and
expertise in the UCPM, and not the rest.
2. Secondly, the survey will be presented to the client for comments and feedback. Only after the
client’s approval of the questionnaire (as Deliverable 2) will the survey be programmed,
published and disseminated
3. Thirdly, we will program and test the survey. The testing is likely to suggest some changes in
the routing and (form of) the questions/ answers. These changes will be also submitted to the
client and implemented once an approval from the client is received. Likewise, this testing will
give us a precise indication of the time required to complete the exercise and suggests ways to
minimize it, which will be also discussed with the client.
4. Fourthly, the programmed and approved survey will be disseminated to the target stakeholders.
To this end, a dedicated email box will be used for sending out the email invitation and providing
feedback on request. The surveys will remain open for the period of the consultation, as agreed
with the Commission.

In particular, the draft survey will be presented to the Client in week 5 to allow for preliminary
comments and suggestions of the Commission to be already integrated in the version of the survey
that will be subject to the testing in week 7. The final overview of the survey questions (Deliverable
2) will be submitted in week 8. The survey is envisaged to be launched in week 9 and, therefore, it
is essential the Commission responds in a timely fashion to the proposed survey outline.

The survey will be carried out in English. An initial overview of suggested topics that can be asked
to a specific group of stakeholders can be found below.

Table 2.5 Initial overview of topics to be addressed during survey


Example topics to be discussed
 National Civil Protection needs

59
‘Stereotype threat’ refers to situations in which people feel themselves at risk to conform to stereotypes of a social group.

38 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Example topics to be discussed
 EU Civil Protection needs
 UCPM Voluntary pool capacities
 rescEU capacities
 CECIS capacities
 Modules
 Interoperability
 Technical requirements
 Advantages of modules
 Disadvantages of/critique on modules
 Suggestions for improvement of modules
 Development costs of modules
 Transport costs of modules
 Operational costs of modules
 Maintenance costs of modules
 Nr. Of assets used (and associated costs)
 Validation of initial findings from first round of interviews
 Best practices
 Future outlook

We propose to support the survey invitation by a brief background note and by a recommendation
letter written by the Commission that will introduce the stakeholder to the aim of the study, justifying
at the same time the need for the contractor to gather the required information.

The survey will remain online for six weeks (week 9 up until week 14), and the project team will
regularly monitor statistics and metrics of the survey. The previous experience of Ecorys with such
surveys and with these groups of stakeholders shows that six weeks is a sufficient time frame to
collect a substantial amount of data. Two weeks after the launch, a reminder will be sent out to
those that have not yet filled out the survey and also using social media. Another reminder will be
issued one week from the given deadline to respond.

Our team is available to share statistics on the evolution of the survey with the Client and, if
requested by the Client to facilitate the increase in the response ratio, our team could also
accommodate an extension of the original deadline (while carefully considering the consequences
such action might have for other activities in the project).

The survey will be promoted by a tailored mailing campaign to the list of relevant stakeholders
compiled by our team and duly integrated with any inputs received from the Commission. The
survey will also be actively promoted through the Consortium’s network.

We suggest using the Checkmarket survey tool.60 Ecorys has an arrangement with Checkmarket to
use this platform (including support arrangements to develop non-standard question formats). Key
features of the platform include:
 Opportunity to customise the survey,
 Control dashboard to monitor response and take action when required,
 Smart and standard reporting functions for quick result insights,
 Export of raw responses in common formats including Excel, SPSS and pdf.

60
https://www.checkmarket.com/

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 39
An example of a recent survey that Ecorys ran through Checkmarket is depicted in Annex III.

The response rate to the different surveys will be continuously monitored by our team and
measures to increase the response rates will be taken should this become necessary. We will
constantly monitor the tool, perform trouble-shooting to timely fix any bugs, and answer to queries
from the participants and/or from the Commission. After the end date of the survey, once the data
has been cleared of errors and possible biases, we will have an outcome that is valid for the whole
study. This preliminary and descriptive analysis will provide us with a good overview of the survey
outcomes.

The information collected from this survey will be used to contextualise the information gathered in
the desk-based research and first round of interviews, to identify topics or areas that require further
attention, and to identify potential subjects for the second and third round of interviews.

Workshop

Week Week 6

Tasks Task 3

As part of Task 3, a validation workshop will be held to reflect on the preliminary findings of the risk
assessment and capacity gap analysis. The format of a workshop is specifically suitable to validate
these findings as it allows for an open discussion and much interaction between the various
participants as well as with the task team. Given the goal of the workshop, the session will be
structured in such a way that exchange of ideas, visions and experiences is encouraged. At the
same time, task leaders of Tasks 1, 2 and 4 will be invited to the workshop as the information from
and insights of the invited stakeholders may prove valuable to the analysis under those tasks as
well.

To this end, the workshop set-up will be of an open nature in which the participants are informed
about the findings of the analysis and, subsequently, will be invited to discuss these findings in a
structured manner. Therefore, the workshop will commence with a detailed presentation of the
findings and the participants will be provided with the opportunity to ask the task team for
clarifications. In preparation to this, the participants will receive a summary of the project, its
objectives and the preliminary findings of Task 3 prior to the workshop. This allows them to prepare
themselves and enhances the efficiency of the workshop.

Depending on the findings of Task 3, the structure of the workshop will be further defined. Whereas
the main objective of the workshop is to gather information from the free discussions among the
participants, the project team will ensure that these discussions take place in a structured manner.
Structuring the workshop will be done by dividing the workshop into dedicated sessions, each
concentrated on a different aspect of the preliminary findings, during which the participants will be
asked to engage. In order to facilitate the exchange of ideas, the task team will prepare sets of
probing questions.

The workshop will take place in week 6 and is envisaged to take place in Brussels at the Ecorys
premises. The workshop will take one full day. Different types of stakeholders will be invited to join
the workshop. A total of 25 participants will be invited to join the workshop. The key goal is to have

40 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
the attendees of the workshop reflect different types of stakeholders and authorities. Therefore, at
this point the team envisages to approach representatives from UCPM and participating states,
international organizations, NGOs, universities and think-thanks. During the initial stage of Task 3,
the potential participants will be identified and approached.

The findings of the workshop will be summarised and shared with the participants of the workshop.

Consultation timetable
The right timing of the stakeholder consultation activities illustrated above is important to ensure
that stakeholders can express their views during all major steps of the study in a way that allows
our team to capture their views on the several aspects examined under this study. The timetable
below is designed to manage such a stream of information with the target stakeholders.

Table 2.6 Stakeholder consultation activity timeline


Stakeholder Consultation Activity Timing
st
Interviews (1 lot) Week 3- 7
Validation Workshop Week 6
nd
Interviews (2 lot) Week 13-17
rd
Interviews (3 lot) Week 27-30
Survey (publication) Weeks 9 - 14

Stakeholder consultation task force


Since the stakeholder consultation activities will feed into the different tasks, covering a wide
number of countries, we propose the appointment of a Stakeholder consultation task force that will
ensure an efficient and timely execution of the consultations. The task force, composed of an
internal and of external senior expert with both research and management experience, will work in
close coordination with the task leaders and with the national correspondents and will be primarily
responsible for:
 Fine-tuning the stakeholder consultation strategy in cooperation with the client;
 Participating in the design of the questionnaires;
 Overseeing their deployment and the data collection.

The stakeholder consultation taskforce members are included in the data collection team (see
Chapter 3) as they will actively engage with stakeholder both to participate in the survey and in
interviews and therefore support the data collection team (national correspondents).

2.3.2 Risk and mitigation strategies


Previous evaluations, and notably the Interim evaluation of the UCPM carried out in 2016, showed
how challenging stakeholder engagement can be, especially when it comes to ensuring a high
response rate to survey questionnaires across different countries and stakeholders groups. The
project team is well aware of the difficulties pertaining to the collection of data and the engagement
of stakeholders to participate in the study. To that end, a back-up plan for the data collection
strategy has been developed. When it becomes clear that country correspondents cannot get
access to particular information, the project team will engage national UCPM experts to support the
country correspondents in obtaining the required information. A dedicated part of the budget has
been set aside to ensure these national experts can be included in the team should the data
collection process pose significant problems. The entire team will be engaged in the identification of
the relevant national experts that an address the challenges that the national correspondents face.
To this end, the Task leaders will indicate in a clear and concrete manner which data exactly is

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 41
lacking. This will help the national experts to attempt to collect this information in an effective
manner. An initial overview of relevant national experts to be approached can be found in Annex I.

The risk regarding the limited availability of data is one of the major challenges that the project team
expects to encounter. Other risks and their mitigation measures are presented below.

Table 2.7 Risks and mitigation measures for data collection methods
Challenge Impact Probability Mitigation measure
Delay in delivery due to tight H M  Discuss with ECHO clear timeline of
timeline to design, launch and action
analyse results (especially for the  Have a broad team in place to implement
survey) and analyse results
 Make a large use of web-based surveys
with multiple choice answers (mostly
closed)
 Diligent planning of the project timeline
and identification of critical path and
milestones in the project implementation.
Inclusion of safety margins around critical
milestones to reduce the impact of
encountered delays by having regularly
team meetings.
Difficulty in identifying the H M Mobilise national experts who have direct
stakeholders access and can engage the stakeholders, or
reach out to international and local platforms
to facilitate access
Low/participation response rate H H  Pre-alert participants before
survey/interviews are held, explain
context, confidentiality, potential issues
with sensitivity
 Send reminders and use networks above
to facilitate
 Design of questionnaires with a
reasonable number
of questions (mostly closed type of
questions)
 Test of questionnaires prior to publishing
 Support from the client in identifying the
most proper
stakeholders and provision of email
contacts
Use of interviews and to provide further
answers
Follow up e-mails and phone calls
Need to use local language with M M Deploy experts able to cover all languages, or
surveys/interviews/workshops most of them. Survey can be translated easily
with no need of a local expert.
Little availability of data H M As such, for a proper evidence-based
evaluation, we will rely on input from coming

42 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Challenge Impact Probability Mitigation measure
from different sources. We expect that much
of this information requirement can be
covered with input from targeted survey and
interviews.

To address the challenge of “information


access”, we propose to have a
comprehensive assessment of the existing
information and data early on, complemented
by access to stakeholders that allows
knowledge and information gaps to be filled
when required and, equally importantly, to
identify additional information needs early on
and propose alternative sources.

Alternatively, the national UCPM experts can


be approached to help retrieve specific
information.
Limited willingness of private H M In order to ensure that our targets for the
stakeholders to participate in interview programme are met, every effort will
study be made to encourage participation. A
focused client recommendation letter
stressing the importance of stakeholder
participation will be requested and sent out to
relevant organisations.
Bodies governed by public H M In our experience it can be difficult to contact
law may not wish to be the right contact person at public authorities.
interviewed/participate in We have taken the following steps to ensure
stakeholder an adequate response:
engagement activities  We will ask for a letter of
recommendation;
 We will put in a substantial effort to get
the right contacts, with up to 3 contact
attempts being budgeted.
The results of the study do M L The method for obtaining results of the study
not match stakeholders point is reasonable and traceable; stakeholders’
of view due to individual point of view is marked as an opinion.
interests
Non-coherent presentation M M Detailed and rigorous preparation of the
of information across various templates
Member and protocols and guidelines for the
States/interviewees/other interviews,
categories rigorous quality control of the performed
activities by
the task leaders, regular meetings with the
team to
update on such a process and quality of the
intermediate deliverables

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 43
Challenge Impact Probability Mitigation measure
Difficulty to retrieve data by M M In case the national correspondents
national correspondents experience severe difficulties obtaining
information, the project team has set aside
budget to involve national experts who are
better connected and will most likely be able
to retrieve this information more effectively.
The list of national contact points in Annex I
will be consulted and national experts will be
approached based on which national
correspondents experience most difficulties.

2.4 Task 0: Project Management

2.4.1 Objective
We introduce a horizontal Task 0 in order to ensure that the consortium is working efficiently and
effectively towards achieving the goals of this project. This includes the organisation of in person
and virtual meetings, continuous reflection on the state of play and quality of the various tasks as
well as coordination of different teams responsible for different tasks and especially the data
collection, which shall run across all tasks and feed into them. Furthermore, part of Task 0 is to
ensure that outcomes of (sub)tasks and activities are aligned and effectively linked to activities
undertaken in other tasks and, thereby, to exploit synergies within the project as much as possible.
The core team established in the light of this objective will help to oversee and implement cross-
task activities and outputs.

Given the vast amount of interlinkages between the tasks, Task 0 will play a pivotal role in linking
the different outcomes of the activities and ensuring they feed into different tasks at the right
moment in time.

While this section explains our approach to project management and quality control, Chapter 3
contains the details about the project team and their roles and Chapter 4 provides a detailed
explanation of specific quality control measures that will be applied to this project.

This task consists of the following subtasks:


 Subtask 0.1 Project management
 Subtask 0.2 Quality control

Objective To oversee (and ensure) the functioning of the service contract as a whole

Team Ecorys

Task lead Ecorys

Task support Ecorys

Outputs  Deliverable 1: Inception Report


 Deliverable 2: Final Report

Timing Week 0 - 38

44 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
2.4.2 Proposed methodology
The following sections will outline which methodology is foreseen regarding project management
and quality control under Task 0.

Task 0.1 Project management


This subtask is predominantly focused on ensuring smooth and effective communication within the
consortium as well as between the different task leaders, team members and with the Client. The
task team of Task 0 will coordinate the overall work of the project in terms of the meetings with the
Client, submission of deliverables and communication within the project between the various teams
and team members. Furthermore, the task team will maintain oversight and link up relevant
outcomes of (sub)tasks and activities to ensure an optimal use of potential synergies within the
project. Besides the internally-oriented aspects of the project management task, the members
involved in this task will also be responsible for the smooth and effective communication and
collaboration with the Client. This concerns planning and attending meetings and communicating
outputs (deliverables) of the project in a timely manner.

In-person meetings
One of the key tasks of the project management team is to arrange meetings with the Client. As
stated in the Terms of Reference, at least nine meetings with DG ECHO should be scheduled and
held. Four of these meetings are foreseen to take place in person and the other 5 will take place via
phone or via the virtual conference hosting tool GoToMeeting (further elaborated in the following
section). The following meetings are foreseen to be conducted with the Client in person:

Table 2.8 In-person meetings with the Client


Meeting When Attended by
Kick-off meeting Week 1 Core team
Meeting to discuss survey & D4.1 Week 10 Core team
Interim meeting Week 22 Core team
Final meeting Week 38 Core team

Besides the planned meetings, the project management team will be able to update the Client at
any time regarding developments in the project during ad hoc meetings if deemed necessary.

Virtual meetings
In addition to in person meetings with the Client, the project team will schedule five virtual meetings
with the Client. These meetings can be held using the GoToMeeting tool (see below). This tool
allows the project team to speak to the Client while, simultaneously, exchanging files. When there is
no need to exchange documents, the project team proposes to schedule a teleconference with the
relevant parties. Both GoToMeeting as well as teleconferences will be used as part of the internal
communications of the project team. The following overview shows which virtual meetings are
foreseen with the Client.

Table 2.9 Virtual meetings with the Client


Meeting When Attended by
Inception meeting Week 4 Core team
Survey questions validation Week 6 Core team
Meeting to discuss D4.2 Week 18 Task Team 2

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 45
Meeting When Attended by
Meeting to discuss methodology Week 29 Task Team 4
framework
Meeting to discuss D6 Week 34 Task Team 4

In addition to the meetings indicated above, the Client can request an ad hoc virtual meeting when
this is deemed necessary.

Communication tools
Earlier, GoToMeeting and teleconferences have been mentioned as effective tools that will be used
in both internal and external communication. The below outline provides an overview of the different
communication tools that will be used and their respective strength:
 E-mail;
 Teleconference;
 Group calling platform GoToMeeting;
 Asana/Slack.

External and internal coordination is facilitated through email exchange. A centralised project email
address – capable of handling all project-related communication – will be created on the Ecorys
server to ensure a single point of contact.

Teleconferencing can be used when a (large) group of people needs to convene without having to
make use of any visual support. When visual support is required, the project team proposes to use
GoToMeeting as this program allows the team to share documents in real time.

Asana and Slack will help the project team to manage the project in an effective and streamlined
manner. Given the tight deadlines and the relatively large size of the project, it is essential for the
project director and project coordinator to be aware of the activities and progress made by the
different tasks. These platforms allow for tracking progress and creating metrics on the progress.

Division of tasks and responsibilities


The consortium has substantial experience of working on large-scale projects such as the one at
hand. This experience has proved that a flexible, task-oriented approach is most effective.
Therefore, the project proposal at hand is structured along five dedicated tasks that are geared
towards a specific objective and that are led by task leaders. This set-up allows for a clear definition
of roles and responsibilities for each task, each task leader and the respective task team members.
The role of the task leader is to coordinate the activities undertaken for that specific task, to liaise
closely with the team members on the task and to ensure close cooperation between with other
task leaders. The project management team will play a supporting role in exploiting synergies within
the project. The core team (introduced hereafter) will play a pivotal role in the coordination of the
different tasks as well as in the process of linking up different tasks, activities and outcomes.

Core team
The core team established as part of this task is specifically aimed to ensure and safeguard the
different linkages between the various (sub)tasks and outcomes. The project at hand is relatively
complex in the sense that the tasks are strongly interconnected. Therefore, the task leaders of the
different tasks will be seated in the core team where they share the progress and updates from their
respective tasks to ensure an optimal alignment and effective exchange of information and efforts.
The core team will base its efforts on the linkages envisaged between the different tasks as laid

46 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
down in the proposal at hand. This document will serve as a guiding document for the core team.
For each task it is clearly outlined how it is linked to the other tasks, and thereby, the task leader
has a clear overview of the position of its task vis-à-vis the others.

The core team is envisaged to meet monthly during a virtual meeting using GoToMeeting or regular
teleconferences. The project coordinator will host the meetings and is responsible for the logistical
arrangements such as setting up and scheduling the meeting as well as distributing an agenda and
taking minutes. During the meetings, each task leader will provide an update on the state of play of
their respective task. Special attention will be paid to each task’s linkages with the other tasks and
the progress made regarding those linkages. The regular meetings allow the task leaders to identify
potential bottlenecks and/or risks at an early stage and, thereby, mitigation actions can be applied
as early as possible. These meetings will be attended solely by all task leaders and the project
coordinator. However, when the Client wishes, she can attend these meetings, too.

The core team will attend the envisaged meetings with the Client. During these gatherings, the task
leaders will be able to provide an update on the state of play of their respective task as well as to
ask the Client specific questions relevant to the activities in which they are involved.

Task 0.2 Quality control


The quality control is geared towards ensuring the quality of project deliverables (use of proper
English, consistent styling, etc.), as well as to creating synergies between the project’s activities
and/or deliverables. The consortium’s approach for achieving these goals is comprised of the
development of a project roadmap and a guidance tool. The processes surrounding the
development of these documents (as well as their purpose, contents, and role within the quality
control mechanism as a whole) are outlined in further detail below:

Project roadmap
Several measures have been foreseen to mitigate the risks regarding a lack of synergy between
task teams and the consortium’s decentralised structure. In addition to the activities undertaken by
the Project Core team and the use of platforms such as Slack61 and Asana62, the Ecorys
consortium proposes to implement – alongside the quality control procedure already in place for
ensuring the ‘tangible’ quality of project deliverables – a quality control procedure which is geared
specifically towards fostering, identifying, communicating, and operationalising synergies between
task teams.

The activities comprising the aforementioned quality control procedure are centred around the
project roadmap document, which will structure the proactive engagement of individuals within the
project. Each of the project’s task leaders will take ownership of developing an outline of how their
respective task is conducted and how it is related to the other (based on the foundations provided in
the proposal at hand).

In order to ensure synergies between project tasks, task leaders will draw upon these agreed upon
linkages between the tasks. The overviews of these synergies between tasks will be aggregated
into a task overview, which will be disseminated among the task leaders within one week of the
first meeting of the project’s core team. The task leader of Task 0 will subsequently compile the

61
https://slack.com/; project management tool that helps to keep track of the work done and provides useful statistics and
metrics to monitor the team’s performance
62
https://asana.com/; project management tool that allows one to keep track of the tasks assigned to different team
members and share files.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 47
task overviews into a project roadmap, which will be shared with the core team prior to the second
meeting. The project roadmap will detail the following information:

Proposed contents of project roadmap


1. Task-by-task overview of deliverables which are of relevance to the task’s activities;
2. Task-by-task overview of deliverables which are of relevance to other tasks’ activities (sorted by
task);
3. Based on point 1, a task-by-task overview of foreseen actions that the task leader intends to
take to ensure his or her task’s access to the aforementioned deliverables. These will ideally be
presented in long-form text(s) which outline relevance to the project, preferred mode of action,
and expected timing of activities.
4. Based on point 2 a task-by-task overview of foreseen actions that the task leader intends to
take to ensure other tasks’ access to his or her task(s)’ deliverables. These will ideally be
presented in long-form text(s) which outline relevance to the project, preferred mode of action,
and expected timing of activities.
5. A GANTT chart outlining – on a task-by-task basis – the timing of the activities identified as part
of points 4 and 5.

Guidance tool
The Ecorys consortium is committed to ensuring the quality of project deliverables. As a result of
Ecorys’ involvement in various other Commission projects relating to crisis management (for DG
HOME as well as DG ECHO), it is familiar with a.) Commission preferences vis-à-vis reporting
style, and b.) the delicate process of navigating Member State preferences vis-à-vis the disclosure
of politically sensitive information. Based on this (and other) experience, the team members of Task
0 will guide the core team through the process of developing a guidance tool for reviewing the
integrity of the following aspects of submitted deliverables:
 Facts and key developments. Check whether to your knowledge the names, dates and figures
used in the report are correct. Have the most recent and/or relevant developments been
featured? Should the presentation or the balance between them be altered? Should any other
developments or trends be included?
 Consistency. Is the message coming through in the work consistent across sections? Are there
facts or analysis buried in the details of the report that contradict the information presented
elsewhere?
 Evidence. Are statements and analysis supported by sufficient evidence? Are quoted sources
reputable and reliable, or might there be a significant bias?
 Contentiousness/controversy. Are the statements and interpretations widely accepted or are
they controversial? If assertions are controversial, are they appropriately backed by facts and
sources?
 Styling. Does the document styling align with the provided style guide?

The guidance tool will be submitted to representatives of DG ECHO as an annex to the inception
report. The guidance tool will serve to structure the activities of the quality assurance team, which
will be mobilised in accordance with the quality control procedures outlined in Section 4.

2.4.3 Deliverables
The project management team is responsible for the submission of two deliverables, namely the
Inception Report and the Final Report. For each deliverable, the draft outline of the report is
presented below.

48 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Deliverable 1: Inception Report
Based on the discussions held and input received during the kick-off meeting, the project team will
develop the Inception Report. This report will reflect the detailed work programme as well as the
project planning as agreed during the kick-off meeting. Furthermore, the Inception Report will
identify additional needs, potential risks and mitigation strategies. Moreover, the Inception Report
will present the final evaluation methodology.

Draft outline of Inception Report


1. Detailed work programme of the project
2. Detailed planning for the study
3. Description of methodological approaches
4. Identification of additional needs
5. Final presentation of evaluation methodology
6. Preliminary draft of survey questions
7. Detailed work plan for the evaluation
8. Overview of team members and description of their role in the project
9. Intervention logic
10. Planned data sources
11. Risks/challenges and mitigation strategies

Deliverable 7: Final Report


The Final Report will consist of two parts: a Microsoft Word document and a Microsoft Excel
document. The Final Report will present the findings of the study and takes into account the
feedback received by the Client throughout the project’s lifespan. Moreover, the Final Report will
present the conclusions drawn by the project team based on the activities that have been
undertaken in the various tasks.

Draft outline Final Report


1. Abstract
2. Executive summary
3. Introduction
4. Background
5. Methodology
6. Analysis
7. Conclusions
8. Annexes
9. Excel files

2.4.4 Risks and mitigation strategies


Table 2.10 Risks Task 0
Risk I P Mitigation
Lack of coordination H L All task leaders are part of the
between different tasks core team and meet regularly
(virtually) to align the activities
ongoing within their respective
task and to ensure the outcomes
of the tasks feed into each other
efficiently. By ensuring close
connections and communication

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 49
Risk I P Mitigation
between the Task leaders, the risk
of miscommunication and
inefficiencies is mitigated.
Tasks are delayed and, H M The tight planning of the project at
thereby, affect the execution hand requires all involved parties
of other tasks to take their responsibilities in
time. In order to clarify who needs
to deliver what and at which point
in time, the proposal outlines
clearly which deliverables need
input from which tasks as well as
when the Client needs to
deliver/approve. This clear task
division helps to structure the
project and mitigates the risk of
delays.

2.5 Task 1: Review and (re-)define the existing response capacities

2.5.1 Objective
The objective of this task is to collect and analyse evidence on the appropriateness of currently
available response capacities (as per Annex II of the Decision No 1313/2013/EU), in terms of:
 Definitions,
 Quality requirements,
 Past response experiences, and
 Most recent national and European risk assessments.

This objective can be fulfilled through reaching two operational targets:


1. To collect relevant quantitative data and qualitative information from a variety of reliable
sources, which can build a foundation of the necessary evidence; and
2. To analyse the collected data and information with the view to determine whether the current
definitions and quality requirements are up-to-date and appropriate to respond to future risks.
The analysis shall take into account past experiences.

Objective Collect and analyse available evidence regarding the appropriateness of the
currently available response capacities in terms of definitions, quality requirements,
past experience and recent risk assessments

Team Ecorys

Task lead CMC

Task support Ecorys

Outputs  Deliverable 2: Survey questions


 Deliverable 3: Report of the survey and desk research

Timing Week 1 – week 18

50 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Task 1, and especially the stakeholder survey conducted under the aegis of this task, serves as the
basis for all other tasks. The stakeholder survey will collect data and information that can be used
by analysts in Tasks 2-4. The analytical part of Task 1 – categorisation of the different modules of
response capacities based on their need for revision or alteration – serves as the foundation of the
cost analysis in Task 2 and feeds into the revision of capacity goals in Task 4.

At the same time, the feedback from Tasks 3 and Task 4 has to be taken into account in Task 1.
Risk assessment under Task 3 is crucial to understand the appropriateness current response
capacities, and revision of capacity goals under Task 4 may also have impact on the
appropriateness of response capacities and become a basis for further analysis under Task 1.

2.5.2 Proposed methodology


This task will be organised around five activities, of which three activities relate to data collection
and two activities are analytical.

Figure 5 Overview of Task 1 methodology


Step 1 Step 2 Step 3 Step 4 Step 5
Identification of Identify main Develop a survey On the basis of Develop
data needs, stakeholders that for targeted the collected recommendations for
understanding what can cover the consultations to data, determine definitions and quality
data information (remaining) data collect a wide categories of requirements for
already available in needs and consult range of data and modules of categories C and D,
public sources them via information from a response provide justifications
(desk research, interviews. The wide range of capacities (A, for them. Take into
activity 1.1) interviews will be stakeholders and B, C or D), account feedback from
held in several run the survey. taking into Tasks 3 and 4 (activity
rounds and at For the survey account 2.5)
different stages of preparation take feedback from
the analysis into account Task 3 and 4
(activity 1.2) inputs from Tasks (activity 2.4)
1, 3 and 4 (activity
2.3)

The table in Section 2.10 shows a tentative time planning for these activities. Preparation and
conducting of data collection activities will start a bit earlier than analysis, but then will run in parallel
with the analysis most of the time, in order to supply the necessary data for the analysis and be
responsive to changing data needs (e.g. specifically in the case of interviews and desk research,
adjustments can be made very quick). The analysis of capacities’ categories (Task 1.4) has to start
early because it needs to provide input for the first iteration of Task 2. With the income of data and
information from data collection and of finding from other tasks (Tasks 3 and 4), the development of
definitions and quality recommendations can also start relatively early, particularly as the outcomes
of Activity 1.5 inform Task 2.

The sections below provides a more detailed description of each planned activity.

Task 1.1 Desk research


Data collection via desk research is a large and ongoing activity supporting both the analytical
activities and data collection via stakeholder consultations (interviews and survey), namely:

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 51
 For other data collection activities, desk research will identify stakeholders to be contacted,
inform us what data is available in databases and documentary sources and what are the
remaining gaps;
 For analytical activities, desk research will collect a variety of data and information to become a
subject of the subsequent analysis.

The initial review of publicly available data and information sources also helps to formulate data
needs for Task 1. Preliminarily, the following data needs have been identified for Task 1:
 Definitions of existing capacities
 Quality requirements of existing capacities
 Performance indicators (assessments) of the existing response capacities in the past response
operations/ missions
 Performance indicators (assessments) of the existing response capacities in prevention and
preparedness projects
 Gaps identified on the basis of the past experience
 New risks according to national and European risk assessments
 Evolution of existing risks according to national and European risk assessments
 Response capacity requirements to face new risks and/or evolved existing risks
 International standards for response capacities
 Recommendations by national and international experts for response capacities

To compile this and other information, the study team will review documentary sources ranging
from applicable legal documents, to national and international studies and reports (e.g. on risk
assessment, on readiness and preparedness of civil protection/ first responders), to
recommendations by international organisations and specialised NGOs, to reviews and reports on
the missions/ projects/ trainings completed and to other documents and literature. The examples of
documents we are going to look into are outlined in Section 2.3 above.

Based on the desk research by the team, the Module fiches that will be used to calculate and
present the cost analysis of the different modules (Section 2.6.2) will be created. The task team will
draft a fiche in close collaboration with the Task 2 team. This draft will then be shared with the
UCPM experts in the project team via email. Once the feedback from the experts has been
collected, the fiches will be amended to reflect the suggestions. During a virtual meeting with Task
team 1, Task team 2 and the experts, the final template of the fiche will be agreed upon.

Task 1.2 Stakeholder consultations via interviews


Stakeholder interviews for Task 1 shall have a two-fold purpose:
1. They will provide additional data and information for the analytical activities (i.e. close
information gaps);
2. With their help we can verify/ validate data from other sources.

In addition, the interviews conducted at the beginning of the project (first round of interviews) can
help to identify new sources of data and information (e.g. give indication of new or previously
unknown studies, reports or databases, identify a new stakeholder or assist with establishing
contact with a stakeholder).

An indicative list of stakeholders to be interviewed is included in Table 2.2. Of those stakeholders,


the following are of specific relevance for Task 1:
 DG ECHO officers (especially from Directorate A and B, ERCC, EERC);

52 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
 Other relevant DGs and EU-level agencies (e.g. European Maritime Safety Agency)
 Disaster Risk Management Knowledge Centre (DRMKC);
 National civil protection authorities from Participating States;
 Evaluators of related programmes such as Prevention, Preparedness and Response to natural
and man-made Disasters (PPRD) in the Southern Neighbourhood (PPRD South II);
 Representatives of specialised international organisations, such as UNISDR, EMT (WHO),
International Civil Defense Organisation,
 Representatives of specialised NGOs, such as Red Cross, Médecins Sans Frontières etc.

The interviews of the first round will be used primarily to identify modules of category A (i.e. that
comply with international standards and do not require revision) and category B (i.e. modules that
are appropriate and up-to-date and do not require revision). For this information, we will target in
the first line potential interviewers at the EU level (DG ECHO, ERCC etc), international NGOs and
international organisations. We will validate their information not only with the desk research, but
also with representatives of national civil protection authorities in order to check whether the
information is still up to date.

During the second round of interviews, we will target in the first line national civil protection
authorities, programme evaluators, national experts with operation experience etc. These
stakeholders can provide us with information about the modules that are not up-to-date and not
appropriate any more to respond to the new and/or evolved risks; they can explicate the (national)
needs and actual supplies of civil protection in the country and the complementarity and
interoperability of capacities between different EU countries. We will enquire from these
stakeholders what changes, in their opinions, need to be undertaken to render response capacities
up-to-date, how definitions and quality requirements have to be changed/ adjusted, what general
requirements for experts are necessary and how their interoperability has been ensured in practice,
how the interoperability can be enhanced, whether the suggested changes make sense at the
national and European level.

During the third round of interviews, we will be further collecting from national level stakeholders
the missing data that is necessary to complete the analysis of modules under Task 1. At the same
time, we will validate the developed definitions, quality requirements and general requirements. For
the validation task, we will address respective questions not only to national level stakeholders, but
also to EU-level and international stakeholders. The latter stakeholder categories can provide
feedback on whether the suggested revisions are compliant with international standards and can be
used across all UCPM Participating Countries. We will consult thematic experts in civil protection in
order to check whether the suggested definitions and quality requirements are forward-looking
enough.

While the questionnaire with questions related to Task 1 will be developed during the inception
phase and during the initial weeks of desk research, here we provide an indicative list of topics/
issues to be covered by the interviews:
 Issues for the first and second round of interviews:
- National Civil Protection risks
- National Civil Protection needs
- Match between risks and needs on the national level
- Match between needs on national level and UCPM capacities in case of emergency in
participating states;
- Capacities of UCPM Voluntary Pool

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 53
- Capacities of CECIS
- Capacities of European Civil Protection Pool
- Capacities foreseen in rescEU
- National Civil Protection capacities of the participating states that could be used in UCPM
operations (not included in the current UCPM Voluntary Pool or CECIS either in the planned
European Civil Protection Pool or rescEU);
- Risks regarding civil protection outside the UCPM participating states
- What additional Civil Protection assets or support is be needed regarding disasters outside
UCPM participating states
 Additional issues for the second round of interviews:
- Adjustments in modules/ capacities necessary in the view of new/ changed risks
- Adjustments in response capacities necessary based on past experiences (exercises,
operations, training)
- Interoperability issues for experts
- Other requirements to experts
 Additional issues for the third round of interviews:
- What implications a specific definition X (quality requirement Y) has for your national civil
protection response capacities (application of the definition, possibility to implement, cost
factors)?
- Does a specific requirement X improve/ ensure interoperability of the expert?

Task 1.3 Stakeholder consultations via survey (targeted consultations)


For the purposes of Task 1, the stakeholder survey shall serve one main objective, namely
collecting data and information from a large number and large variety of stakeholders in order to
inform the analysis of modules of response capacities and to establish the need for revision of
respective definitions and quality requirements.

As the targeted consultations with stakeholders is a large overarching activity that informs all tasks
of this study and also needs inputs from all other tasks, its detailed description was included in
Section 2.3.1, even though this activity will be coordinated under Task 1. In this section, we only
cover those survey issues that are specific to Task 1 topic, that is review and (re) definition of the
existing response capacities, which mainly refers to topics to be covered by the survey.

Suggestions for a survey questionnaire will be developed during the inception phase, presented at
the kick-off meeting and then elaborated and finalised subsequently, in accordance with the
instructions by the Client. In relation to Task 1, the topics/ issues presented above as topics to be
addressed during the interviews will also be considered to be included in the survey. Based on the
initial round of interviews and the discussions with the Client, this list will be further narrowed down
and specified.

Task 1.4 Categorisation of modules of response capacities


Under this activity, we will analyse the existing modules of response capacities listed in Annex II of
the Decision No 1313/2013 and categorise them into four groups (A, B, C and D). The purpose of
this categorisation is to determine which of the response capacities are to undergo a more rigorous
analysis concerning their definitions and quality requirements. The categorisation and analysis of
response capacities also builds a basis for the cost analysis under Task 2.

54 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
All in all, 15 modules63 plus technical assistance teams (TAST) and experts will be examined under
activity 1.4.

In the first step, on the basis of the data and information collected about the existing response
capacities, we will determine which of them comply with the established international standards
(specifically, USAR and EMT). For these, we will contrast the existing UCPM capacities, registered
by Participating Countries, with the data from international directories of EMT and USAR for
certified modules. We will also compare the legal requirements to those modules with international
standards. Such modules are category A modules and require no further examination under Task 1.
Category A modules will be immediately transferred for cost analysis under Task 2 (first iteration).

In the second step, we will determine which modules are still adapted to the actual risks and
anticipated responses (category B). To this end, for each individual response capacity, we will
compile the findings about potential risks (risk assessment under Task 3), information about the
performance of the modules in past operations, missions, projects and trainings, recommendations
by national and international experts and information about any gaps established in the past. We
will contrast this information with the current definitions and quality requirements applicable to the
specific response capacity. Those capacities will be included in category B, for which none of the
following issues are identified:
 no underperforming in the past operations, missions, projects and trainings;
 no gaps in the performance established;
 no expert recommendations issued.
Category B modules also need to fully cover responses to the eventualities of relevant risks.

Once the category B modules are identified, they will be immediately transferred for cost analysis
under Task 2 (second iteration). At the same time, we will maintain this category under observation
as the progress of the Task 4 analysis may have impact on our assessment.

The remaining response capacities (i.e. not fulfilling one of the conditions above) will be categorised
as category C (in need of revision) and “other response capacities” and expert as category D.
Categories C and D will first undergo an analysis under activity 1.5, then submitted to the client for
approval. Only after the approval, category C will be transferred for cost analysis under Task 2
(third iteration).

Task 1.5 Recommendations for (revised) definitions and quality requirements


The development of recommendations for (revised) definitions and quality requirements will occur
in a slightly different context for category C and category D. Whereas for category C definitions and
quality requirements need to be revised, for category D they need to be developed from scratch.

The recommendations for the revision of definitions and quality requirements for category C
modules will be developed based on the data and information collected in relation to new or
evolved risks (from Task 3), gaps and underperformance established during past operations,
missions, projects and trainings and following recommendations of national and international
experts. We will also take into consideration the definitions and quality requirements of category A
and B modules, which did not need revision, and analyse whether some of the definitions and
quality requirements can be applied to or adjusted to category C modules. Finally, we will take into
account the findings on the revision of capacity goals under Task 4. The suggested changes will be

63
Aerial forest fire fighting (plane and helicopter) is not part of this study, according to the Tender specifications, p. 6.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 55
validated during stakeholder consultations, and their final version will be presented to the client for
approval.

For category D, we will develop general requirements that also include quality and interoperability
requirements. In drafting our recommendations, we will rely on international standards and best
practices (if existing); on past experiences of relevant operations, missions, projects and trainings
where the respective expertise was required or developed; and on the topical risk assessment
(Task 3) which identifies skills and characteristics of (expert) teams necessary for response. To
develop interoperability requirements, we will take into account the quality requirements drafted for
modules. The suggested general requirements will be validated during stakeholder consultations,
checked against the finding of Task 4 and their final version will be presented to the client for
approval.

2.5.3 Deliverables
The deliverables resulting from Task 1 are:
 Deliverable 2: Survey questionnaire – a questionnaire comprising questions and answers
(answer categories) for all stakeholder groups, with routing.
 Deliverable 3: Report on the survey and desk research, including at least the following
elements:

Draft outline report on the survey and desk research


1. Presentation of survey results (with a detailed question-by-question presentation in an annex)
2. Presentation of interview results (with a detailed question-by-question presentation in an annex)
3. Presentation of relevant desk research findings in relation to the analytical part of Task 1
4. Analysis of modules of response capacities
5. Recommended changes and reasoning

2.5.4 Risks and mitigation strategies


Task 1 has data collection via survey included in it, and it relies heavily on interviews with
stakeholders as well as desk research. As a result, all risks and mitigation measures linked to data
collection also apply here. They are presented in Section 1.1.2 above.

Table 2.10 below presents several specific risks and mitigation measures that refer directly to the
analysis to be performed under Task 1.

Table 2.11 Risks Task 1


Risk I P Mitigation
Inability to differentiate between H L From the onset of the project, the study team will
modules of category B and category be strongly supported by experienced civil
C, for instance, due to contradictory/ protection experts. We will rely on their advice to
insufficient data and inconclusive interpret data and resolve inconsistencies.
data/opinions of stakeholders In addition, we will search for additional data via
desk research and interviews and triangulate
them. We will also seek advice from the client
and DG ECHO/ERCC experts to validate data
and our findings.
Inability to develop definitions, quality H L We will draft definitions and requirements based
requirements and/or general on the input from stakeholders, and we will
discuss the suggested definitions and

56 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Risk I P Mitigation
requirements acceptable and suitable requirements with them during the project.
for all Participating Countries. Lastly, we will also validate the suggested
definitions with stakeholders before presenting
them to the client. This approach ensures that all
stakeholder considerations are taking into
account, and they have been included in the
process of the revision.

2.6 Task 2: Cost analysis

2.6.1 Objective
The overall project objective is to respond better and prepare for natural and man-made disasters
including more efficient and effective process with an optimal cost. Task 2 contributes to this
objective by aiming to provide updated data with regard to the development, deployment,
operational and maintenance costs of the capacities used in response to disasters. Moreover, such
update should include the most recent market data such as labour costs and expenditures.

The costs to be analysed for 34 countries (all EU Member States and 6 additional countries
participating in the UCPM) relate to services handling in response to natural, technological, and
radiological disasters or environmental accidents, including accidental marine pollution, wildfires,
floods, earthquakes, hurricanes, industrial accidents. The cost categories to be analysed are:
development, transport, operational and maintenance. The project data collection activities will not
include personal data, therefore no issues are envisaged in relation to General Data Protection
Regulation but overall project includes very sensitive data, therefore the highest confidentiality
requirements will be followed.

Data collection for all tasks, including Task 2, will be conducted in a centralised manner by the
dedicated team of national correspondents. The task team will ensure that relevant questions are
included in the survey, interviews and data needs are made explicit to the national correspondents
doing the desk research. Following the data collection, different capacities scenarios will be
developed and this will be only Task 2 related activity.

Collect detailed data in view of providing a clear, precise, qualitative costs


Objective
breakdown for all the capacities

Team VVA

Task lead VVA

Task support VVA

Outputs Deliverable 4: Cost analysis (3 iterations)

According to ToR execution starts as soon as redefinition of the capacities under


Task 1 have been approved by DG ECHO.
Timing
Start execution after approval of Inception Report as the first deliverable is due in 5
weeks

The link between tasks relates to experts involved, data collection activities and task outputs. The
summary of interaction of Task 2 with other tasks is presented below:

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 57
 Experts involved. Task 1, 3, 4 experts will provide information about different stakeholders and
data sources where cost information could be obtained. Experts will review the capacities cost
scenarios defined and final results of capacities cost calculation.
 Desk research. During the desk research for Task 1, 3, 4 capacities cost information will be
shared with the Task 2 project team.
 Survey. The survey sent out to different stakeholders will include capacity cost related
questions to collect information for Task 2.
 Interviews. The interviews conducted will include questions related to capacities costs.
 Outputs of Task 1, 3, 4 in relation to capacities definition. New capacities definitions will be
used to collect capacities related cost information.

2.6.2 Proposed methodology


The task will be conducted through four activities outlined in the table below.

Table 2.12 Cost analysis project methodology


Step 1 Step 2 Step 3 Step 4

Understand the cost Data collection Define cost Cost calculations by


data to be collected (2.2) categories for which taking into account
and information scenarios to be scenarios defined
already available at applied and to (2.4)
DG ECHO (data identify data source
needs) (2.1) for those
adjustments (2.3)

These activities will be executed in three iterations, but in effective and efficient way by using
synergies between activities. The process iterations are based on the reporting requirements, which
include three reports, and on data availability. The main project iteration blocks are presented
below.

Table 2.13 Cost analysis Task step-by-step


First iteration Second Iteration Third iteration
 Inception Report – Week 5  Week 5 – Week 20  After completion of Task 4
 Based on public data  Based on surveys and  All data sources
 Cost of the capacities not interviews  New capacities identified
requiring a review of their  All capacities
definition

Activity 2.1 Understanding data needs


The data needs for Task 2 are pre-defined by the Terms of Reference of this study, specifically by
the nature of the assignment of Task 2, and by the relevant legal documents. The said documents
build the basis for this initial data needs assessment, which will be further refined and rendered
more precise during the inception phase of the project.

According to Commission Implementation Decision of 16 October 2014 laying down rules for the
implementation of Decision No 1313/2013/EU, Annex II “General requirements for modules and
technical assistance and support teams”, the following elements of emergency operations are
defined:

58 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
 15 modules64 - for each module tasks, capacities and main components are defined. The
examples of modules include high capacity pumping; water purification; medium urban search
and rescue, etc.
 Capacities are defined for each module, more than 100 if different sub-categories are taken
into account. The examples of main capacities include – medium and high capacity pumps;
sufficient personnel to fulfil the task; mobile water storage unit etc.

The capacity costs will be analysed for all UCPM Participating Countries (34 countries). Each
module and capacities short description is included in the Implementation Decision. During the
project the description of selected modules and capacities will be revised, therefore also related
costs to be revised. According to the Terms of Reference, cost analysis to be performed based on
the revised definition of response capacities, therefore the activity will start once the revised
capacity definitions will be available but taking into account very close deadlines the task will start
after Inception Report approval.

In order to understand potential shortcomings of the disaster response capacities under the UCPM,
it is important to define the category of costs that should be analysed while deploying the response
capacities. The following category of cost will be considered:

 Development costs are the costs incurred to establish a given model or a team which would
be ready for immediate deployment if disasters strike. The definition applies to both material
and human resources at the same time and refers to such activities as renting, dry leasing, pre-
positioning at a provider’s location.65
 Transport costs are the costs incurred to transport material and human resources where the
disaster occurs. It covers installation and all necessary process needed to render the module
operational.
 Operational costs are the costs incurred for the running of the model during the entire period
of the mission deployment. The operation costs cover a wide range of items such as material,
staff salaries, accommodation, food, fuel, electricity. These costs are clearly the heaviest
burden for the response capacities.66 However, the operational costs inside and outside the EU
territory can vary considerably.
 Maintenance costs refer to costs which are needed for maintain the modules in standby while
they are not used for the future deployment in the case a disaster strikes. Maintenance costs
cover a wide range of expenses such as trainings, storage, warehousing as well as replacement
of expired medication.67

In addition, costs will be split between equipment and human resources costs. For several cost
categories, EU and non-EU costs to be analysed.

The cost categories for different capacities can have a different meaning, therefore for each module
a Module Fiche will be prepared. The Module Fiche will be prepared with support of Task 1
experts, and draft document will be discussed with some Member States representatives and DG
ECHO. Data collection activities will start only after approval of Module Fiche to ensure a common
understanding. If needed, during the data collection activities Module Fiche will be updated to

64
According to the Terms of Reference, aerial forest fire fighting modules (2 modules – by plane and by helicopter) are
excluded from the scope of the study.
65
Following the indication provided in the TOR.
66
Ibid.
67
Ibid

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 59
reflect additional assumptions made or additional data sources found. Example of information
included in Module Fiche is presented below.

Module Fiche with Capacities sub-fiches


1. Prepared for each module and a separate sub-fiche for each capacity. Module fiche will provide
an overview of different capacities planned
2. Description of each capacity – prepared together with Task 1 technical experts team
3. Assumptions used for each capacity with regards to different cost categories (matrix including
capacity and cost categories)
4. Analysis of capacities for which national level data to be collected, e.g. wages and capacities
where the same cost will be used for all countries analysed, for instance equipment
5. Data sources used in the capacities data collection activities
6. Definition of scenarios in accordance to the number of similar elements in the capacity
7. Definition of scenarios in accordance to different technical equipment used
8. Definition of scenarios in accordance to different wages

As stated above costs will be analysed for 34 Participating Countries, therefore it is important to get
the factual information about all capacities for countries included in the project scope. The basis for
country level information will be Common Emergency Communication and Information System
(CECIS) database where Member States shall register their modules, technical assistance and
support teams, other response capacities and experts. Given the CECIS database will not be fully
available to the project team, the data will be complemented by the survey and stakeholder
interviews.

All data received and obtained during an analysis will be recorded in Excel spreadsheet with a
separate sheet per capacity, where in columns different cost categories will be presented and
each raw will represent a different country. An example of information collection table is presented
below. The table also will include a reference to data resource used and during the project it will be
decided whether reference is needed at country level or it is sufficient to have it at cost category
level. In some cases, the table will also include additional columns where number of similar
elements in the capacity will be defined, for instance, number of tents, and cost per one unit will be
included in the cost table.

Table 2.14 Example of cost sheet


Module Component Development Type of Transport Transport
costs cost, e.g. costs inside costs outside
leasing EU EU
Data source Data Data source Data source
source
High Medium and high
capacity capacity pumps
pumping
Personnel

Activity 2.2 Data collection


General approach of Task 2
The desk research, interviews and surveys that will be carried out as part of the data collection
approach are also closely related to Task 2 since the data gathered through these methods will

60 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
serve as the foundation upon which the cost analysis will be conducted. To this end, Task 2 team
will provide concrete input for both the interviews (to be conducted by the national correspondents)
and survey. This input will take shape of a list of questions that need to be asked during the
interviews and/or survey. An initial outline of the items related to Task 2 that shall be addressed
during the interviews and survey includes:
 Capacities costs
 Development costs
 Transport costs
 Operational costs
 Maintenance costs
 Background and technical information on capacities
 Cost scenarios defined for certain capacities

At this stage, we assume that all topics will be covered by both the interviews and the survey.
However, during the inception phase of the project, a distinction will be made outlining which topics
will be addressed in which data collection method (and why). Furthermore, the Task 2 team will
provide the national correspondents with guidance with respect to the desk research exercise. In
order to collect relevant data of high quality and detail, national correspondents will be trained by
Task 2 team and civil protection experts involved in the project. This will be done in a briefing (on
the basis of the developed guidance) explicating the specifics of the data required and potential
data sources. Throughout the data collection period, the Task 2 team will remain at disposal of the
national correspondents and can provide advice and additional coaching if necessary.

As the stakeholder consultation activities are envisaged to take place throughout the various tasks,
a broad set of stakeholders will be approached. The following stakeholders are likely to hold data
and information necessary for Task 2 and will be interviewed or surveyed:
 DG ECHO and other DGs (e.g. MARE, CLIMA)
 EU-level agencies (e.g. European Defence Agency, European Maritime Safety Agency)
 Emergency Response Coordination Centre (ERCC)
 Member States' UCPM contact points
 Competent national authorities
 Relevant industries
 Specialised non-government organisations (e.g. Samariterbund)
 Other stakeholders - International Organization for Migration; International Federation of Red
Cross and Red Crescent Societies; Organisation for the Prohibition of Chemical Weapon, World
Bank, OECD, ANO, United Nations

Given the priority indicated by the Client to receive the cost analysis, data collection will consist of
three individual iterations. These iterations will receive input from the data collection methods that
have been carried out at the time the deliverable for the specific iteration is due. However, as the
first iteration is due relatively fast after the kick-off of the project, limited data collection methods can
be employed and, therefore, this iteration will be predominantly built on the consultation of publicly
available data (through desk research) and the knowledge of the project team’s experts. The
second iteration will be based, in addition to the above source, on the findings from the survey and
the interviews. The third iteration will add an additional round of interviews to its analysis and
combine all data and analysis obtained at that point, including the third round of interviews.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 61
Public data (first, second and third iteration)
During the first project iteration, the main data source will be public data, therefore during the
proposal stage already several data sources have been identified. These data sources will primarily
be used to gather information regarding the appropriateness of capacities as well as to collect data
on the various kinds of costs that those capacities have. The following sources have been identified
to be of use in this exercise:
for instance the following data to be used in desk research:
 Information from Country card template68 - contact information at Member State level;
 Recent economic data of the European market (EU Member States) - the costs of equipment
and human recourses. The main data source will be EUROSTAT;
 Recent economic data of Iceland, Montenegro, Norway, Serbia, the Former Yugoslav Republic
of Macedonia and Turkey. The main data source will be World Bank;
 OECD statistics (detailed information available about health and education);
 Suppliers publicly available data about different equipment categories;
 Relevant market and manufactures data on modules and capacities;
 Relevant studies commissioned by EU institutions.
 Studies and reports from Participating Countries.

Based on the discussions with DG ECHO and selected project stakeholders, the list will be
elaborated further and the suggested sources will be further explored to provide the Client with a
more detailed list of concrete reports and publications.

The data retrieved by means of this exercise will be gathered and documented in the module fiche.
Based on the publicly available information, the Task team will execute the first iteration of cost
analysis, which will be conducted for modules of category A and B identified under Task 1.

Project team experts’ knowledge (first, second and third iteration)


UCPM content experts involved in this project have good inside knowledge about capacities,
technologies and suppliers. Therefore, they will be very important source of information and based
on the discussions with them Module Fiche and capacities sub-fiches will be completed. UCPM
experts will be asked to first provide their feedback on the draft module fiches (in writing). As a
follow-up to this feedback, Task 1 and Task 2 leads may host a virtual session with UCPM experts
to discuss the suggestions made and to agree on the final module fiche template. Moreover, the
content experts will be asked to share input to the fiches based on their experiences and expertise
with the UCPM and the different modules.

The experts involved in other tasks will also be asked to provide information about public data
sources on capacities, capacities suppliers and contact information, capacities’ technical
descriptions, expert knowledge about capacities costs and functionality, feedback on cost scenarios
defined and feedback on initial cost information.

Targeted consultations / survey (second and third iteration)


Given the early deadline for the completion of the first iteration of the cost analysis, the results of
the survey will only be considered in the second and third iteration. The survey will be launched in
week 9, when the first round of interviews has been conducted. This allows the survey questions to
be develop according to the gaps identified during the interviews and desk research. In other
words, the areas in which little information has been retrieved will be specifically targeted during the
surveys. To this end, the Task 2 team will provide a concrete set of questions that they would want

68
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.320.01.0001.01.ENG

62 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
to integrate into the survey in order to be able to effectively conduct the cost analysis in the second
and third iteration. These suggestions for questions will be discussed with the Commission in week
5 when the draft of the survey is envisaged to be ready.

At this stage, we envisage the following topics to be included in the survey:


 Capacity costs
 Development costs
 Transport costs
 Operational costs
 Maintenance costs
 Technologies
 Suppliers of capacities

As not all stakeholders approached through the survey will have knowledge on the costs related to
the capacities, the project team will ensure that respondents can either choose to skip those
questions or the survey will be programmed in such a way that only a particular type of
stakeholders will be shown the questions related to costs.

Interviews (second and third iteration)


As part of the project at hand, three rounds of interviews will be conducted. Given the high priority
of the first iteration of the cost analysis (category A and B modules), the findings from the interviews
cannot be integrated into this first iteration as the planning will not allow this. However, the second
and third iteration of the cost analysis (mainly category C and D, but category B modules may be
re-examined) will benefit from the interviews conducted. In order to ensure that the right type of
data is collected by the national correspondents, the Task 2 team will provide concrete questions to
be included in the interview guide. This way, the national correspondents are made aware of the
type of information that is requested and, simultaneously, the Task 2 team is able to steer the
national correspondents in the data collection process.

The interviews conducted during the first and second round will feed into the second iteration of the
cost analysis. The third iteration of the cost analysis will also include the findings from the third
round of interviews.

During the interviews, the prepared Module fiches will serve as the basis for data collection as
these highlight exactly which information needs to be collected in order to conduct accurate cost
analysis.

Activity 2.3 Defining cost adjustments and scenarios


Once the data has been gathered, Task 2 team will commence with the cost analyses. These
analyses will take into account the different economic realities of the UCPM Participating Countries
and, therefore, different scenarios will be calculated (including purchasing powers, salary levels,
etc). During the inception phase it will be agreed with the Client which scenarios will be taken into
account.

The main steps in the cost adjustment / scenario definition process will be as follows:
 To understand for which cost categories cost adjustments to be made (e.g. wages)
 To understand capacities which will require scenario definitions, for instance, different
technologies;

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 63
 To identify and agree with the Client the cost data adjustment sources, (for instance
EUROSTAT – purchasing power parity, Harmonised Consumer Price Index, EUROSTAT labour
market statistics)
 To update Module Fiche capacities cost categories descriptions regarding cost adjustment
methodology applied for specific cost category and to update in relation to scenarios defined.

Activity 2.4 Cost calculation


The last step will be to apply the earlier identified and agreed cost adjustment and scenarios.
Subsequently, the final cost information per capacity will be calculated and included in the module
fiche. The excel table used for calculation will include detailed information per country, per capacity,
cost category and three cost scenarios for those capacities where scenarios have been defined.
This document will be shared with the Client.

2.6.3 Deliverables
Deliverable 4: Cost analysis
Task 2 deliverables include three reports on capacities which will be prepared in three iterations.
The reports will be prepared based on the required deadline, data collection principles and data
sources, and capacities to be covered by specific report. The summary of reports to be prepared
are presented in the table below.

Table 2.15 Overview of deliverables Task 2


First iteration Second Iteration Third iteration
5 weeks after contract signature 20 weeks after contract signature After completion of Task 4
Based on public data Based on public data, surveys All data sources
Cost of the capacities not and interviews New capacities identified,
requiring a review of their All capacities adjustments for other capacities is
definition (category A and B) necessary

The reports to be submitted will include the following sub-reports, which could be further tailored
and developed, based on ECHO needs. During the Inception phase the reporting requirements will
be discussed in detail with ECHO, for instance country reports, consolidating reports for several
capacities.

Draft outline Cost analysis deliverables


 Excel table with detailed data at module, country, capacity and cost category level. The
developed spreadsheet tables at the end of project will be provided to DG ECHO and could be
used for other projects and decision-making process.
 Module Fiches including module description, assumptions and summary factual information
about module costs. Capacities sub-fiches including capacity description, assumptions and
summary factual information about capacity costs.
 Different summary cost reports prepared based on the data available in detailed Excel tables.
Where possible, data will be presented in the graphical form. For instance, summary reports per
module, per capacity.
 Cost variation scenarios for wages including minimum, maximum and average cost scenarios
based on the scenarios defined.
 Cost variation scenarios in accordance to the number of similar elements in the capacity.

64 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
 Cost variation scenarios in cases when different technologies were used, including details of the
new technologies, their costs and how such innovation could improve the intervention in case of
disasters.
 Combined cost scenario per capacity will be prepared for each capacity and cover scenarios in
relation to wages, similar elements in capacity and technologies.
 Combines cost scenario report in table format where three cost variation scenarios are linked.

All reports described above will be available to DG ECHO as internal versions. Based on the
internal versions one public version of the report will be prepared including summary information
and information, which has been obtained from public sources.

2.6.4 Risks and mitigation strategies


As the objective of Task 2 is to create reliable cost analysis, it is essential that this Task is being
provided with accurate and rich insights from the data collection process. Therefore, specific
attention will be paid to the following aspects:
 Accurate / Reliable data. The data will be collected in Excel table with different formulas,
adjustments and references, therefore additional control will be performed over Excel table
designed to ensure that formulas are correct. A dedicated independent person will be allocated
to perform the review of Excel table and formulas included in the table.
 Complete data. The data will be collected for all capacities for all Participating Countries and
for each capacity at least 2-3 independent data sources will be used.
 Correct assumptions made. During the cost categories analysis and data collection activities
several assumptions will be made which could relate to costs reported. By assumption made it
is also understood adjustments made to wages, decision on technical equipment where data
will be collected at EU level and no country specific data will be collected, scenario definitions
and other. At least two experts who are involved in other tasks will review assumptions made
and recorded in Module Fiche and final version will be approved by DG ECHO.
 Triangulation of data. For each capacity, a consultation will be held with at least 2-3 persons
and different public data source will be used.

Table 2.16 Risks Task 2

Risk I P Mitigation
Insufficient number of Medium Medium Close cooperation and planning with other project
participants selected for tasks, involvement of industry experts from other
targeted consultation tasks, cooperation with national stakeholders
Insufficient cost data about Medium Medium Close cooperation with experts from other tasks in
capacities order to identify suppliers
Close cooperation with national experts who can
provide an information about capacities and suppliers
Subjective view in the Low Medium Close cooperation with industry suppliers in order to
technology scenario get their views on technical aspects of capacities and
definition data verification from different data sources

2.7 Task 3: Risk based assessment and capacity gap analysis

2.7.1 Objective
The objective of Task 3 is two-fold:

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 65
1. To carry out an assessment of the EU internal and external safety and security risks; and
2. To carry out risk-informed gap analysis of the existing capacities under the EERC and CECIS.

The outputs of the Task 3 will feed into all other tasks. However, before engaging in risk mapping, it
is important to identify the time horizon to be ascertained for risk assessment. This will need to be
discussed with the DG ECHO prior to the start of the risk assessment.

Objective To carry out comprehensive risk-based capacity assessment and capacity gap
analysis

Team Risk Society, Ecorys

Task lead Risk Society

Task support Ecorys

Outputs Deliverable 5: Risk based capacity gap analysis

Timing Week 0 - 8

The findings from the risk assessment and capability-gap analysis will inform all other tasks. Hence,
the overview of the existing and emerging safety and security risks for Europe, which is one of the
outputs of Task 3, will inform Task 1. This overview will be critical to analyse the appropriateness of
the existing response capacities. The findings of Task 3 will be used to define capacity categories
(A, B, C, and D respectively).

The Task 3 team will also work in cooperation with Task 4 team to review the capacity goals in light
of the findings from risk assessment. Based on the findings of risk-based capacity analysis from
Task 3, the categories D and C will be analysed in details: what are the emerging risks? Which
capacities the UCPM is missing? Which capacities needs to be revised and strengthened? Which
are the requirements to the new capacities and those that need revision? Answers to these
questions will be summarised in a list of preliminary recommendations, which will include the
definitions of new capacities, typologies of the UCPM’s capacities, and when appropriate, the
quality requirements of new capacities. The list will be submitted to DG ECHO and only after its
approval the further work on cost analysis for categories C and D will be organised by Task 2.

The findings from the risk assessment and risk-based capacity analysis carried out under Task 3
will further inform (a) the content of the survey 1 and (b) the series of interviews and stakeholders’
consultations. Further, the Task 3 team will be engaged in conceptualising the methodological
framework to be finalised under the Task 4.

2.7.2 Proposed methodology


Task 3 will be carried out following the 4-step method outlined below.

Table 2.17 Steps to accomplish Task 3


Step 1 Step 2 Step 3 Step 4

Risk data Response Validation Develop


collection and Capacity Gap workshop (3.3) recommendations and
preliminary Analysis (3.2) draft final deliverable
analysis (3.1) (3.4)

66 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Activity 3.1 Risk data collection and preliminary analysis
The available national risk assessments from all 34 countries participating in the UCPM will be
analysed. The focus will be on the high likelihood and medium to high impact risks as well as low
likelihood and high impact risks. The risk mapping will be carried out based on specific criteria to be
fine-tuned during the desk research. The criteria would include, but not limited to, the following:
 Geographic distribution of risks per countries: the national risks assessments from each
participating state will be analysed, cross-country comparisons made, and when relevant
general trends revealed in terms of risks geographic distribution across the Europe
 External risks will be analysed (a) based on regional perspective: Eastern neighbourhood and
Southern neighbourhood; and (b) global perspective for humanitarian risks where the UCPM
might be activated.
 Typology of risks: additional analysis will take place to reveal the typology of the risks the
Union is facing. The categorisation for the typology will be further fine-tuned, however, it will be
based on the conceptualisation of ‘crisis’ within the EU and will include (a) man-made crisis
(including armed conflict, terrorism, etc.), (b) natural disasters (based on UNISDR’s hazard
definition - earthquakes, floods, droughts, wild fires, health epidemics, industrial or transport
accidents, fires), and (c) complex emergencies, a combination of natural and man-made crisis
which result in ‘a total or considerable breakdown of authority, which require international
response that goes beyond the mandate or capacity of any single agency, and which has been
assessed to require intensive and extensive political and management coordination’. 69
 Risk severity (i.e. likelihood X impact): the EU-level overview of the risks will be analysed
based on the first-ever EU-wide risk overview carried out by JRC in collaboration with DG
ECHO in 2017. When relevant the information will be combined with the national and external
risks, the compatibility of the findings would require additional elaboration from the project team.

Various sources will be analysed to map the existing and emerging risks outside the EU that can
impact the safety and security of the European Union as well as humanitarian risks where the
UCPM is likely to be activated to provide necessary support for disaster/ emergency response. As
mentioned above, external risks will be analysed from regional (Eastern and Southern) and global
(humanitarian risks) perspective. All three typologies of risks will be explored: caused by natural
hazards, by man-made crisis, and risks of humanitarian crisis. For the risk of natural disaster there
are some reliable early warning mechanisms in place already (for instance, the European Flood
Awareness System (EFAS)) and some global indexes developed (such as INFORM70); however,
the risk mapping for man-made and compound crisis would require additional analysis.

The imperative to understand the contextual risks71 and ensure adequate preparedness has
triggered multiple innovative and useful developments at national, regional and global levels. They
include risk monitoring tools, early warning indicators, various platforms for combined data analysis
and visualization, etc. Given the paramount complexity of this task, the state-of-art developments
by leading international organisations will be analysed. This includes, but is not limited to, the work
carried out by:
 WB on Global Risks Platform72,
 UNDP on Global Risk Platform73,

69
https://interagencystandingcommittee.org/system/files/legacy_files/WG16_4.pdf
70
http://www.inform-index.org/
71
https://irgc.epfl.ch/page-158246.html
72
http://www.europe.undp.org/content/geneva/en/home/partnerships/the-global-risk-platform.html
73
http://www.europe.undp.org/content/geneva/en/home/partnerships/the-global-risk-platform.html

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 67
 OECD on High Level Risk Forums74, and many others.

The Task 3 team will analyse all recent developments to map the external risks (both natural, man-
made, and compound) that can be of relevance for the safety and security to the Union and to the
operational effectiveness of the UCPM. Special attention will be paid to the recent developments
regarding conceptualization of emerging and systemic risks carried out by International Risk
Governance Council from 2018 (IRGC).75 Special attention will be paid to emerging risk areas such
as CBRN, terrorism, (cybersecurity) risks to critical infrastructure, amongst others. And last but not
least, due attention will be paid to cascading and compound risks that may cause a chain of
reactions and multiply disaster impact.

The risk mapping will be carried out through desk research and relying on inputs from interviews
with leading experts/ stakeholders conducted during the first round of interviews.

Activity 3.2 Response Capacity Gap Analysis


When the risk mapping is finalised, the risks will be matched with the existing capacity goals to
identify if the capacity goals and capacity requirements are appropriate in light of the existing
risk(s). This is an analytical exercise that the Task team 3 will carry out in close consultation with
the selected stakeholders including UCPM and national civil protection authorities. It is critical to
define the most ‘plausible’ scenarios for various risks to define which capacities might be required
to address those risks.

When the response capacities and response capacity requirements are analysed vis-à-vis identified
risks, then the existing capacity goals will be compared with the availability of capacities in the
EERC. The purpose is to identify if necessary capacities are easily available for the participating
states and/or when the assistance from the UCPM might be requested. The initial response
capacity gaps will be identified by Task 3 team through exploring which are the missing capacities
within UCPM and which capacities are only partially address the risks. The work will be carried out
via desk research, however, if required additional questions in stakeholders’ interviews may be
included.

When initial response capacity gaps have been identified, the Task 3 team will review the
capacities registered in CECIS as well as any available buffer capacities to determine if they are
sufficient to close the gap. The analysis will be based on the access to the data available from
CECIS to the Team. Given restrictions in accessing the CECIS, the team will submit the overview of
the new capacities to the DG ECHO and request their feedback regarding which capacities should
be considered as priority during the remaining period for the capacity gap analysis. With this, the
team is expecting to receive some additional direction from DG ECHO regarding the capacities
about which there is some assurance at the EU level without actually getting into the specifics of
the quantity and quality of the resources available as buffer capacities for the UCPM.

After that, the Task 3 team will identify (a) those capacity goals and capacity requirements that
would require revision, and (b) those new capacities that are necessary to meet the emerging risks.
On the latter issues, the Task 3 team will cooperate with the Task 1 team because the (preliminary)
findings can already inform the design of the survey and the definitions of categories A, B, and C
modules. The response capacity gap analysis will be carried out in close consultation with the

74
http://www.oecd.org/gov/risk/5th-oecd-high-level-risk-forum.htm
75
https://irgc.epfl.ch/page-158246.html

68 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
UCPM experts that are part of the project team. Also, the engagement of the Task 4 team will be
requested to ensure coherence of deliverables.

Task 3.3 Validation workshop


When the preliminary findings are summarised, one day multi-stakeholder workshop will be
organised. The purpose of the workshop is to validate the findings and proceed to the next step, i.e.
develop recommendations on how to address the response capacity gaps. The workshop is
suggested to be organised in Brussels with the engagement of maximum 25 experts/ stakeholders
from the UCPM participating states, international organisations, NGOs, academia and think-thanks.
The objectives of the workshop will be the following:
 To reflect on and validate the internal and external safety and security risks for Europe,
 To reflect on and validate the capacity gaps identified,
 To reflect, on which capacity requirements are necessary to revise partially relevant capacities
and which capacity requirements necessary to develop for new capacities.

To address all three objectives the workshop will be designed in highly engaging way with
presentations by the Task 3 team and group discussions to ensure the whole variability of options
are duly collected. Representatives of other Task teams will be invited to gain insights from the
workshop and engage with experts/ stakeholders. The findings of the workshop will be summarised
and inform the final recommendations. The detailed organisation and process of workshop is
described in Section 2.3.1.

Activity 3.4 Development of recommendations


Once the capacity gaps identified and insights are collected from the stakeholders through
interviews and the validation workshop, a set of final recommendations will be developed. They will
include:
 New definitions of capacity goals (i.e. a list of new capacity goals will be formulated). When
relevant, the definition of capacity goals will be further supplemented by the set of capacity
requirements tailored to each capacity goal.
 New typologies of capacities based on the findings from the desk-research and stakeholder
consultations (i.e. which capacities are relevant to consider in modules).

The work will be carried out in cooperation with Task 1 by providing input to shape the list of
capacities and definitions/ requirements in categories C and D. The recommendations validated
during the workshop will directly contribute to shaping the methodological framework for the
assessment of the suitability of the existing capacity goals as planned under the Task 4.

2.7.3 Deliverables
Deliverable 5: Risk based capacity gap analysis
There is one deliverable expected from Task 3 and that is Deliverable 5: Risks based Capacity Gap
Analysis report, which will be developed and finalised eight weeks after the approval of the
Inception Report. It will include all findings from Task 3, namely risk mapping, analysis of the
response capacity goals in light of the identified risks, availability of capacities in the EERC vis-à-vis
risk-informed capacity goals, comprehensive overview of the capacity gaps and the availability of
buffer capacities that can be mobilised through CECIS. The report will also include preliminary
recommendations on new capacity typology and new capacity goals.

Draft outline risk based capacity gap analysis report


Risk mapping

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 69
Analysis of the response capacity goals in the light of the identified risks
Availability of capacities in the EERC vis-à-vis risk-informed capacity goals
comprehensive overview of the capacity gaps and the availability of buffer capacities that can be
mobilised through CECIS
Preliminary recommendations on new capacity typology and new capacity goals

2.7.4 Risks and mitigation strategies


Table 2.18 Risks Task 3
Risk I P Mitigation
Not all national risk assessment L H The desk study will be supplemented by stakeholder
data are openly available (some consultations and interviews to collect data on missing
parts of the assessment might be capacities
classified)
The time allocation for Task 3 and H M Well-designed stakeholder consultation strategy, early
the tight deadline (8 weeks after the engagement of stakeholders and timely planning of all
inception report) might not be interviews and consultations and the validation workshop.
sufficient to allocate all the Support letter from the DG ECHO for all stakeholders within
interviews planned and ensure the project to ease the process and stakeholders
necessary representation during engagement.
the validation workshop
Restricted access to CECIS might M H The assistance of DG ECHO will be requested to advise on
not allow detailed analysis of the the list of identified capacities, which of them should be
available at the national level buffer further elaborated within the project and which could be
capacities vis-à-vis identified considered sufficiently backed by the buffer capacities. It is
existing and emerging risks expected that the DG ECHO would be able to provide such
an insight based on their access to CECIS without going
into specifics of the quantity and quality of the buffer
capacities, which might be classified information.

2.8 Task 4: Revision of capacity goals

2.8.1 Objective
The objective of Task 4 is to assess the suitability of existing EERC capacity goals and to prepare
recommendations on their revision. The task involves the development of a methodological
framework assessing EERC capacity goals and a list of potentially significant capacity goals.
Depending on the findings of this task, our team will also provide DG ECHO with a list of capacities
which are considered outdated and/or obsolete and with a draft request to national contact points
which:
 Sets out the details of an assessment of potentially significant capacity gaps, and;
 Invites them to provide information on any capacities available outside of the EERC.

Objective 1. Assess the suitability of existing EERC capability goals;


2. Prepare recommendations pertaining to revision of existing capability goals.

Team CMC, Ecorys

Task lead CMC

Task support Ecorys

70 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Outputs Deliverable 6: Capacity goals revision
Deliverable 7: Final report

Timing Weeks 6 - 38

Task 4 exhibits a high degree of interdependence with Tasks 1, 2, and 3 which provide inputs for
the activities and/or deliverables subsumed under it, while activities completed within Task 4 affect
several activities which take place during the later stages of Task 1. The desk research conducted
during the early stage of Task 4 provides inputs into the design of the 2 nd and 3rd rounds of
interviews. At the same time, Task 4 relies on:
 Data collected from the 2nd and 3rd rounds of interviews;
 Desk research and/or conclusions derived from Task 3, and;
 The 3rd iteration of the cost analysis exercise, to provide it with input data for the suitability
assessment which constitutes the basis for all further analysis within the task.

At the same time, the revision of capacity goals in Task 4 will inform whether new definitions and
quality requirements or typologies of capacities are required under Task 1. If this is the case, an
approval by DG ECHO as specified in Task 1 is needed.

2.8.2 Proposed methodology


The methodology employed in Task 4 is relatively linear and can be divided three activities:

Table 2.19 Overview of Task 4 methodology


Activity 4.1 Activity 4.2 Activity 4.3
Provide national correspondents with Produce a methodological Apply the methodological framework
the information they need to collect framework for assessing developed in Task 4.2 to the data
data in interviews which is of the suitability of existing collected in Tasks 1, 2, and 3. The
relevance to Task 4 as a whole capacity goals and task culminated in a final report, in
(identification of data needs). The formulating and applying addition to an overview of high-and-
Task culminates in the dissemination variable weights during the low priority capacity goals, a list of
of an interview questionnaire, a prioritisation exercise. The revised (and omitted) capability
background information guide, and a Task requires inputs from goals, and – depending on the
list of relevant stakeholders to Tasks 1, 2, and 3. outcome of research – a draft
interview. The Task requires inputs request to national contact points to
from Tasks 1 and 3, in addition to provide additional information.
original desk research.

Activity 4.1 Identification of data needs


The successful execution of Task 4 depends highly on the quality of data derived from 2 nd and 3rd
rounds of interviews. These inform the process of:
1. Evaluating the suitability of existing capacity goals;
2. Formulating technical and/or operation requirements for revised capacity goals;
3. Identifying omitted capability goals, and;
4. Assigning importance (weighting) to the variables which will feed into the prioritisation.

The collection of information which feeds into these processes requires a tailored approach to
conducting interviews. Our previous experience shows that interviewers with extensive knowledge
of subject-matter are able to deliver better results. This is because their subject matter expertise
allows interviewers to evaluate the relevance of responses in real time, and facilitates the process

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 71
of asking thoughtful follow-up questions. Put simply, well-informed interviewers are able to collect
better results because they are able to go beyond a standardized question list.

Task 4.1 is to ensure that the data collected over the course of the 2nd and 3rd rounds of interviews
benefits from well-informed interviewers. In concrete terms, this means that activity 4.1 will produce
(or rather contribute to producing) interview guidance and questions. The guidance will include a
‘digestible’ ‘background’ document that provides interviewers with relevant background information,
and a list of relevant interviewees.

In order to facilitate activity 4.1’s input into the 2nd and 3rd rounds of interviews, activities falling
under activity 4.1 will commence in week 6 and conclude in week 10. The deliverables will be
completed and sent to DG ECHO for review by the beginning of week 11, with the expectation
being that feedback will be provided (and addressed) within three weeks.

Methodology
Activity 4.1 is primarily desk research-oriented and supplements the information collected in Tasks
1 and 3. The guiding research questions could canvas the following issues:
1. How do other countries and/or organisations (if applicable) define the technical and/or
operational requirements for their capacity goals? In the event that these differ from those
outlined in ANNEX II of the UCPM implementing Decision (2014/762/EU), what lessons can
be learnt from the observed discrepancy?
2. What criticism has been regularly leveraged against the UCPM capacity goals (and on what
grounds)?
3. Which UCPM participating states have made the best use of the mechanism?
4. Which UCPM participating states are most likely to need the mechanism in the next 5 years?
Based on this, are there any omissions within the UCPM’s current capacity goals?
5. Who are the key stakeholders to interview when it comes to identifying shortcomings in the
UCPM (technical or otherwise) and/or to identifying its drivers?

The questions will be formulated through a combination of desk research and an analysis of inputs
from stakeholder consultations from previous tasks. Stakeholder analysis will be applied to
inform the results of the literature review. The expected input sources, in addition to an explanation
of the to-be-used data collection techniques, are featured in the table below.

Input sources for activity 4.1


Question Input sources Comment(s)
1 1st round interviews, It is expected that the majority of the information necessary for
literature review addressing question 1 can be derived from Task 1 interviews.
Literature review during Task 4.1 is expected to play a
supplementary role, and will focus on collecting contextual
information to present to interviewers.
st
2 1 round interviews, Task 1 interviews are expected to provide minor (circumstantial)
literature review inputs towards addressing question 2. The majority is expected to be
collected through desk research and/or literature review, including of
ongoing discussions within the public discourse.
3 Literature review Question 3 will be addressed exclusively through the use of the
(CECIS and/or UCPM CECIS and/or UCPM databases.
databases)

72 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Question Input sources Comment(s)
4 Task 3 risk The risk assessment in Task 3 is expected to constitute a key input
assessment, literature into addressing question 4. Additional desk research will be
review conducted to provide contextual information to interviewers.
5 1st round interviews, Question 5 will be addressed by the desk research process, with
Task 3 risk each newly collected data source being collated into a larger
assessment, stakeholder matrix.
stakeholder analysis

The observations derived from the literature review will be tempered through the findings of the
stakeholder analysis. This step is included because it is expected that stakeholder motivations
and/or interests relating to the UCPM are likely to impact the reliability of the data derived from the
literature review (particularly within the public discourse data category), meaning that the team
proposes the stakeholder analysis as a contingency plan to offset the risk of respondent bias. The
stakeholder analysis also feeds directly into the process of compiling a list of stakeholders to
interview.

Activity 4.1 will culminate in four deliverables, including:


1. A preliminary list of ‘omitted’ capacity goals;
2. A ‘guiding questions’ document for use in interviews;
3. A ‘background’ document for use by interviewers, and;
4. A list of suitable interviewees.

The reasoning behind the inclusion of a preliminary list of omitted capability goals is that, in order
for questions relating to the cost analysis and the assessment of these capacity goals to be
addressed through the interview process, questions and context pertaining to this subject should be
provided to interviewers at an early phase. Because the ‘omitted capability goal’ list ties in closely
with the list of ‘potentially significant’ capacity goals (hereafter referred to as the prioritisation
exercise) – and because the expectation is that the 3 rd iteration of the cost analysis will feed into
the aforementioned exercise – the Ecorys consortium will provide the baseline of these
deliverables. With this in mind, activity 4.1 has been designed to ensure all relevant information is
collected in an efficient and systematic manner.

In addition to a general overview (geared towards providing interviewees with context), the
‘background’ document will feature an explanation of the reasoning behind chosen questions,
outline feasible answers, and suggest follow-up questions which (depending on context and/or
interviewee) may improve the quality of results.

Activity 4.2 Development of a methodological framework for assessing suitability of


capability goals
Activity 4.2 will culminate in a methodological framework. This is conceptualised by the Task 4 team
as a systematic approach towards reconciling various variables collected throughout previous
phases of the study (Tasks 1, 2, 3, and activity 4.1) with one another through relative weighting.
These include (but are not limited to) the severity of the threat which the capacity goals are geared
towards addressing (this can be measured through a combination of number of likely affected
individuals, likelihood of occurrence, costs associated therewith, etc.) and the cost(s) of
implementing improvements and/or maintaining the current status quo (this gets, by extension, at
the technical and/or operational adequacy of the underlying capabilities). Relevant variables will

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 73
(wherever possible) be identified during activity 4.1 (on the basis of how other countries conduct
similar analyses, etc.).

Activity 4.2 focuses on analysing data collected through activities of Task 1, the risk assessment
conducted as part of Task 3, and the desk research conducted in activity 4.1. If any research
should fall out of the scope of the aforementioned interviews, the consortium will make use of EU
databases (e.g, Eurostat and CECIS) before resorting to sources such as the World Bank or the
OECD.

Activity 4.2 is expected to be executed from week 23 to week 30. This timing should allow for:
a) the consortium to collect and address DG ECHO’s feedback on the proposed
methodological framework prior to applying it to the collected data, and
b) the required input data (i.e., from the 2rd iteration of the cost analysis, which will include an
exploration of the costs associated with omitted capability goals and with modifying and/or
expanding existing ones) to be collected.

One of the key sub-deliverables for activity 4.2 is a ‘weighting scheme’ which – based on the
information collected throughout Tasks 1, 3, and activity 4.1 – assigns a numerical value denoting
the ‘relative importance’ of the variables on which data has been collected throughout these tasks.
To facilitate this process, the interview questions delivered at the end of activity 4.1 will – where
feasible – be designed to be compatible with ‘Likert’ (a format in which respondents are asked to
score qualitative variables between 1 and 5) and Thematically-based techniques of analysis. Other
techniques which could feasibly be applied are sentiment analysis and word association mining.
The exact methodology for extracting and quantifying qualitative variables from interview data will
vary from variable to variable. As previously outlined, each extraction and/or quantification
methodology will be justified by means of an explanatory text which will be included in the annex of
the methodological report, and which will be prepared on a variable-by-variable basis. These
explanatory texts will (where possible) refer to relevant peer-reviewed (empirical) studies to
maximise the final results’ ability to withstand criticism from within academic circles.

The use of a weighting scheme, along with an accompanying handbook for ‘quantifying’ any
qualitative variables collected throughout the research process is proposed as part of the Ecorys
consortium’s commitment to adhering to a highly robust and replicable methodology.

Methodology
The proposed approach focuses on the quantification of variables which relate to the suitability of
the stated capacity goals. This exercise is done on a gap-by-gap basis and is subsequently applied
to the capacities of the UCPM as a whole. The logic is that an evaluation of the suitability of the
individual capacity goal connects naturally with evaluation of the UCPM as a whole as any
shortcomings in the individual goals will highlight the need for goal amendments and/or goal
additions within the mechanism. Based on our initial desk research, the following high-level
variables are expected to be of relevance for the evaluation of the suitability of the individual
capability goals (as outlined in the request for services):
1. The adequacy of each capability goal’s technical requirements as outlined in Annex II of
Decision No. 2014/762/EU, and;
2. The degree to which the existing capability goals have been met by the ERCC.

These variables will both be made operational through a range of sub-variables, the data for which
will be collected during Tasks 1, 3, and activity 4.1. Regarding the adequacy of each capability

74 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
goal’s technical requirements, the input data will largely be qualitative in nature, and will be derived
from the 2nd and 3rd rounds of interviews and from the risk assessment exercise conducted in Task
3. The overall goal is for the variable to clearly denote to what degree each of the current capacity
goals addresses the risk assessments specified in Task 3. The distinction between variables 1 and
2 is that – while variable 1 is largely theoretical (‘on paper, do the technical requirements outlined in
ANNEX II of Decision No. 2014/762/EU address risks outlined in the risk assessment?’) – variable
2 is far more operational and is geared more towards evaluating whether the current
implementation of these capacity goals is adequately equipped to tackle current (and future) risks.

Given the fact that activity 4.2 focuses on designing a methodological framework for assessing
these variables, activities will largely be comprised of desk research and stakeholder and/or expert
consultation. The consulted stakeholders will be identified under activity 4.1, with the interviews
expected to occur during the 2nd round of interviews. The goal of the respective questions will be to
validate assumptions regarding the relative importance of the previously outlined variables and sub-
variables.

The weighting scheme which will be used to operationalise the prioritisation exercise will be derived
from the 1st round of interviews. In addition, the consortium will seek feedback from DG ECHO to
ensure that the final weighting scheme aligns with your expectation. Because the prioritisation
exercise and the ‘omitted capability goals’ will be offered as baseline services, the methodology for
the prioritisation exercise (i.e. the weighting scheme) will incorporate information regarding the
omitted and/or adjusted capability goals identified as part of Task 3. As a result, the document
outlining the preliminary technical requirements for omitted and/or adjusted capability goals (the first
of the activity 4.2 deliverables) will play a central role in formulating this aspect of the methodology.

In addition to the variables (and sub-variables) outlined in this section, the weighting scheme is
expected to incorporate variables relating to a) the magnitude and/or severity of the risk(s) which a
capability goal is geared towards addressing, and b) the costs associated with implementing,
improving, or maintaining each individual capability goal. This means that activity 4.2 and, by
extension, activity 4.3 are highly dependent on the successful formulation of a set of definitions
and/or typologies of capability goals (both amended and omitted) at the end of Task 3.
Furthermore, activity 4.2 is also dependent on whether we could deliver robust cost estimates of the
resulting (amended) capability goals in Task 2. The final product is a weighting scheme which can
correct for diminishing returns to investments and optimise the process of appropriating public
funds towards capability goals.

Activity 4.3 Application of the methodological framework


The objective of activity 4.3 is applying the methodological framework developed as part of activity
4.2 to the data collected throughout Tasks 1, 2, 3, and activity 4.1. Given the Ecorys consortium’s
commitment to offering the prioritisation exercise and a list of ‘omitted capability goals’ as a
baseline service (and integrating these into the 3 rd round of the cost analysis which takes place in
Task 2), activities 4.1 and 4.2 have been designed to ensure that both the data and the
methodological framework necessary for processing it are ready for use prior to the start of activity
4.3.

Activity 4.3 will produce various deliverables, including:


1. A final overview of the suitability of the UCPM’s capability goals;

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 75
2. A draft version of a request to national contact points which aligns with the requirements
outlined in Article 20(3) of Decision No. 2014/763/EU, should these goals (or the
implementation thereof) be found to be inadequate;
3. A final list of revised and (where omitted) formulated capability goals;
4. An overview of ‘potentially significant’ capability goals (i.e., overview of which goals require
the most immediate attention and/or funding);
5. A list of obsolete capability goals, including explanation, and;
6. A final report.

Methodology
The methodology for executing activity 4.3 will be outlined in the methodological framework
developed in activity 4.2 and approved by DG ECHO prior to implementation. Because the
methodology is expected to have a strong quantitative component, the consortium will make use of
(among others) Microsoft Excel, Tableau, and Atlas.TI to process, catalogue and visualise data in a
systematic manner.

The execution of the methodological framework developed in activity 4.2 is expected to result in a
final overview of the suitability of the UCPM’s capability goals and an overview of ‘potentially
significant’ capacity goals (i.e. deliverables 1 and 4 of activity 4.3). The suitability analysis will be
conducted on a goal-by-goal basis and will incorporate goals which have been reformulated and/or
developed as part of the ‘omitted capacities’ exercise in activity 4.1 and the capacity revision’
exercise which takes place at the end of Task 3. A distinction will be made between theoretical and
operational suitability, with theoretical suitability tests being applied to goals which have been
reformulated and/or developed as part of the ‘omitted capabilities’ exercise in activity 4.1 and the
‘capability revision’ exercise which takes place at the end of Task 3, and operational suitability tests
being applied to capacities which are currently part of the UCPM capacity portfolio.

In the event that the aforementioned reformulated and/or newly introduced capacity goals fail the
‘theoretical suitability’ test, the Task 4 team will employ a combination of desk research, follow up
phone interviews, and (depending on results) a validation stakeholder workshop in Brussels to
identify the shortcomings and to elaborate on operational and/or technical requirements.
Reformulated and/or newly introduced capability goals that pass the ‘theoretical suitability’ test will
be submitted to the UCPM experts involved in the project for final validation, after which a final
version of the will be submitted to DG ECHO for final review. The final version of the capability
goals will feature (if applicable) the newly developed and/or reformulated definitions and typologies
delivered by the Task 3 task team, in addition to an overview of the corresponding technical and/or
operational requirements developed by the project expert panel. The technical and/or operational
requirements will adhere to the format used in Annex II of Decision No. 2014/763/EU. The
reasoning behind amendments and/or additions will be included as an annex.

For all capacity goals which are found to be ‘operationally inadequate’ from an implementation point
of view, a request will be drafted which outlines the methodology used in the to-be-completed study
and the results derived therefrom. The request will further invite Member States to provide
information on any capacities which are available outside of the EERC which may serve to increase
the suitability of the mechanism if integrated. The team will work under the assumption that the
aforementioned request will be sent out to all national contact points, and (as such) will make use of
the national correspondents to translate the drafted document to the relevant languages.

76 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Activity 4.3 deliverables will be developed in tandem, with the methodology for their development
being defined by the methodological framework developed during activity 4.2. It is expected that
those capability goals which are evaluated as of the ‘lowest priority’ will also typically be most likely
candidates for elimination. Information regarding this specific parameter will be worked into the
interview questions developed during activity 4.1. The approach towards identifying ‘potentially
significant’ capability goals will be grounded in the use of variable weights, which will incorporate
data derived over the course of the cost analysis. Because the 2nd iteration of the cost analysis will
cover (among others) the newly developed and/or reformulated definitions and typologies delivered
by the Task 1 team, these capabilities will be included in the prioritisation exercise. For all ‘low
priority’ capability goals whose status is obsolete will be validated on the basis of ad-hoc follow-up
phone interviews and/or desk research. An explanatory document highlighting the reasoning behind
this designation will also be made available.

2.8.3 Deliverables
As outlined in the request for services, Task 4 will culminate in – in addition to the ‘intermediary’
deliverables outlined in the previous sections – a list of revised capacity goals and a Final Report,
The contents of these deliverables are outlined below:

List of revised capability goals


1. Overview of original (existing) capacity goals;
2. Short outline of study results. In concrete terms, this section will outline – on a capacity-by-
capacity basis – why the existing range of capacity goals do or do not succeed at addressing
risks which are likely to impact UCPM participating countries in the near future. This analysis
will incorporate variables derived from the risk assessment in addition to variables pertaining to
the operational and/or theoretical adequacy of the capacity goals in question;
3. Overview of proposed changes to (existing) capacity goals, and;
4. Overview of proposed additions to and/or subtractions from the list of existing capacity goals.

Final report
1. Title page;
2. Table of contents
3. A ±200-word abstract;
4. An executive summary of no more than 6 pages outlining a.) the motivation behind the study,
b.) the methodology applied, and c.) the identified results;
5. An introduction;
6. An explanation of the study background;
7. An in-depth overview of the methodology;
8. An in-depth analysis of the identified results;
9. An overview of the resulting conclusions and recommendations, and;
10. A series of annexes which provide information that supplements points 5-9.

The length of the final report will not exceed 80 pages (before annexes). It will be submitted to DG
ECHO no later than 38 weeks after the contract signature.

2.8.4 Risks and mitigation strategies


Table 2.20 Risks and mitigation strategies in Task 4
Risk Probability Impact Mitigation
Data collected is Medium High Interview data is expected to be of varying quality as a
insufficiently detailed to result of differences in interviewers’ personalities,

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 77
Risk Probability Impact Mitigation
allow for the discrepancies between interviewees’ willingness to
development and/or volunteer information, and unavailability on the part of
application of the key interviewees. To compensate for this risk, the
methodological Task 4 team has planned activity 4.2 to allow for a.)
framework. the use of the 3rd round of interviews to collect relevant
data (in case the 2nd round proves insufficient, and b.)
the conduct of additional (phone-based) interviews by
the Task team itself. In addition, considerable
resources will be invested into ensuring desk research
is as productive as possible.
Desk research does not Medium Medium The Task 4 team will have access to highly
result in sufficient experienced experts, which will provide them with
information to facilitate relevant venues to pursue, and which will validate the
the preparation of quality of all Task deliverables.
interview material.

78 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
2.9 Planning and deliverables

Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

Task 0
Task 0.1
Task 0.2
Task 1
Task 1.1
Task 1.2
Task 1.3
Task 1.4
Task 1.5
Task 2
Task 2.1
Task 2.2
Task 2.3
Task 2.4
Task 2.5
Task 3
Task 3.1
Task 3.2
Task 3.3
Task 3.4
Task 4
Task 4.1
Task 4.2
Task 4.3
Task 5

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 79
Task 5.1

Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

Deliverables
D. 1
D. 2
D. 3
D. 4.1
D. 4.2
D. 4.3
D. 5
D. 6
D. 7
Meetings
M1
M2
M3
Workshop
M4
M5
M6
M7
M8
M9

M1: KO (in-person) M5: meeting to discuss D4.2 (virtual)


M2: Inception meeting (virtual) M6: Interim meeting (in-person)
M3: survey questions (virtual) M7: meeting to discuss method framework (in-person)
Validation workshop (in-person) M8: meeting to discuss D6 (virtual)

80 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
M4: meeting to discuss survey and D4.1 (virtual) M9: Final meeting (in-person)

Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

Task 0
Desk
research
Interviews
Survey
Workshop

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 81
3 Project team

This chapter sets out the general organisation of the Consortium and introduces the project team
that will implement this project. Our approach to project management was in detail outlined in
Section 2.4, as we consider project management a separate horizontal task permeating all other,
content-related tasks of this study.

Building on our experiences and lessons learned from previously-conducted projects on behalf of
DG ECHO and other European Commission DGs and, we propose a clear project team structure
with clearly defined roles and responsibilities This will ensure the accomplishment of both the needs
of the project by means of smooth delivery of all required consultancy, technical support and
assistance services during the running time of the study.

3.1 Team structure

The appropriate attribution of responsibilities, as well as a robust degree of coordination and/or


communication within the consortium, is vital for the successful and efficient functioning and
execution of the tasks outlined within the Terms of Reference. To facilitate this process, individuals
within the consortium have been organised into the following group-level entities:
 Project core team consisting of the Project Director, the Project Coordinator, the leaders of the
project tasks and UCPM experts;
 Quality assurance team consisting of a Quality Manager, an expert in the field of UCPM
evaluations and a Language Editor. The quality assurance team will be responsible for
validating deliverables on the basis of both quality (language, formatting, etc.) and content;
 Task teams consisting of a task leader, task support and junior experts, sometimes
complemented by a dedicated UCPM expert (in-house or external);
 Data collection team consisting of national correspondents for 34 countries participating in the
UCPM and the Project coordinator who is supervising them as well as stakeholder engagement

82 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
experts who will support data collection activities that involve reaching out to stakeholders/
experts;
 Back-up & support team consisting of staffers which provide the safety net in case a member of
the team is unexpectedly unable to fulfil his/her role.

3.2 Introduction of the project team members

In this section we present the short profiles of the project team members and provide descriptions
of the roles assigned to them.

Core team
Laura Birkman, MPA, is Senior Consultant at Ecorys, where she is Deputy
Head of the Security and Justice Unit. She has led a number of studies for the
European Commission at the intersection of security and innovation policy. An
expert in disaster risk and crisis management and evaluation studies, Laura has
led a number of projects in this field for the European Commission. Laura is
currently in charge of the design and implementation of the EU disaster risk
management peer review programme run by DG ECHO. In 2015, she led a study for DG HOME
investigating at European level First Responder capability goals and corresponding technological
requirements. In addition to her thematic expertise, Laura is an active mediator and community
builder in the security research domain. She is currently leading advisory and support services
concerning the building of a Community of Users in Secure, Safe and Resilient societies, which
aims to improve the integration of practitioner requirements into EU funded research. She is also
leading the support facility for DG HOME in its chairmanship of the International Forum to Advance
First Responder Innovation (IFAFRI), a global initiative focused on enhancing and expanding the
development of affordable, innovative technology for first responders worldwide. Laura is a Member
of the Dutch Analyst Network National Security and a Member of the Roster of Experts of the UN
Security Council. In 2012, Laura obtained her MPA from Harvard University with a Fulbright
scholarship. She also holds an Executive Master in International European Relations and
Management from the University of Amsterdam and a Joint M.A. (Honours) in Philosophy and
International Relations from the University of St. Andrews.

Project director
The project director (Laura Birkman) is responsible for the overall implementation of the project and
and is charged with ensuring proper communication between members of the project core team,
ensuring that the various task teams are on-track, referring deliverables to the quality assurance
team, and facilitating the process of forging synergies within the consortium itself.

The main external responsibility of the project manager is to represent the project team to DG
ECHO. A critical element of this role will be to ensure that the decentralised consortium structure
outlined under task 1 is functioning smoothly. The project director will maintain transparent and
regular communication with DG ECHO (e-mail and phone, but also regular and ad-hoc face-to-face
meetings as much as needed but at least during the 9 foreseen meetings) to keep the client up to
date with the progress of the individual task teams. An increase in communications frequency is
anticipated in the lead up to the delivery of key outputs and milestones. This will include an update
on any barriers, challenges or other issues that may impact either the quality or timely delivery of
outputs. The project manager will respond to requests and any other issue raised by DG ECHO.

The project manager has as internal responsibility to chair the project core team and to take
general decisions with respect to the contract. She will orchestrate and supervise the process of
quality assurance, with the objective of guaranteeing that all deliverables are of high quality and are

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 83
submitted in a timely manner. She will additionally monitor budgetary and financial matters, provide
content expertise, and contribute (where necessary) to the execution of the tasks. The project
director has a proven track record in the management of projects relating to crisis management.
Her understanding of the nature and scope of the requested services, as well as the context
underpinning them, will thus inform the project core team’s approach to organising workshops,
delivering on community outreach, and aligning the content of training programs to Commission
preferences.

For the delivery of specific requests for services, the project manager will be responsible for:
1. Discussions with the Commission on the scope of services to be carried out under specific
tasks, including responding to potential requests for clarification on behalf of the task leaders;
2. Assisting in the development of work plans for individual assignments and/or tasks (i.e.:
reviewing the project roadmap prepared by the junior project experts);
3. The monitoring of performance metrics;
4. Maintaining contact and communication with DG ECHO regarding progress and notification of
any issues that may impact either the quality or timely delivery of outputs as they emerge;
5. Proposing mitigation actions in the event of delays or quality issues;
6. Ensuring that outputs are delivered to schedule, and;
7. Dealing with any contractual issues, which may arise between the Client and the Consortium.

Alexandra Schmid is a consultant in the area of security and justice with a


special focus on the policy fields migration and border security, disaster risk
and crisis management. Alexandra has strong analytical skills and ample
experience in data analysis (both quantitative and qualitative), consulting
stakeholders in different jurisdictions through different methods, and project
management. Recent studies include an impact assessment on lowering the
fingerprinting age for children and on storing a scanned copy of travel documents in the Visa
Information System (VIS)for DG Home. In addition, Alexandra was a project team member in an
impact assessment for a potential EU measure on cash payment restrictions (DG ECFIN).
Alexandra is also involved in the multi-year advisory and support services for the DG Home
Community of Users initiative that works to improve EU security research programming (H2020,
ISF, etc.) by reducing the fragmentation and lack of impact of EU funded projects.

Alexandra has a strong academic track-record and has published extensively, mainly focusing on
the migration-security nexus. Her latest publications include two book chapters on EU-Belarus
border management and on the impact of the 9/11 terrorist attacks had on the migration policies of
the European Union. She has also co-authored a conference paper on the parliamentary control of
Frontex. Alexandra studied Political Science & International Relations at the Ludwig-Maximilian-
University in Munich, Germany and at The University of Auckland, New Zealand. She holds a M.A.
in Political Science and is currently finalizing her PhD on the EU’s external migration governance. In
her previous career, Alexandra worked as a research fellow and university lecturer at the Jean-
Monnet Chair for European Politics at the University of Passau, Germany, (4 years) as well as a
journalist for leading German press (5 years).

Project coordinator
The main general responsibility of project coordinator (Alexandra Schmid) is to support the project
director in ensuring the effective implementation of the services outlined in the ToR. In particular,
the project coordinator will be responsible for the daily executive management of the consortium.
This includes maintaining regular contact and communication with task leaders, providing guidance
and feedback on deliverables prior to their submission to the quality assurance team, responding to

84 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
any questions or issues raised, and ensuring that quality protocols and provisions are upheld (in
partnership with the Quality Assurance Group outlined below).

The project coordinator’s role differs from that of the project director in that the project coordinator is
expected to be much more hands-on. Where the project director interacts with the consortium
largely through project core team meetings or when task leaders take up contact proactively, the
project coordinator will work closely with the task leaders and with the junior team members to
ensure the project’s implementation aligns with the goals of the project roadmap. The project
director will thus ensure that the agreed-upon approach, direction and responsibilities set out in the
inception phase are executed as planned (both in terms of quality and work plan), and will be
involved in all parts of project and will be charged with maintaining a clear overview of progress.

In concrete terms, the project coordinator will be responsible for:


 Managing and overseeing the coordination of the consortium’s day-to-day activities;
 Maintaining a clear overview and strategic oversight of progress on individual assignments vis-
à-vis the project roadmap;
 Developing (where necessary) reporting templates, guidance materials and checklists, in
collaboration with the junior team members, task leaders, and the quality assurance team;
 Preparing work plans and setting out internal deadlines for all assignments;
 Maintaining regular contact and communication with the project director, including on overall
progress, feedback and lessons learned and information related to individual assignments;
 Ensuring proper and timely delivery of all outputs and deliverables;
 Leading the process of quality assuring outputs (which will also be achieved through the use of
a multi-layered quality assurance process carried out by our Quality Assurance Group),
including editing outputs to publication standard and consistency checks;
 Organising coordination and management activities (meetings with, and reporting to the EC);
 Providing administrative support to the project director as needed, in terms of contracts
(preparation), templates, minutes, planning of travel and workshop organisation;
 Development and continued maintenance of a tool to track the status and progress of all
deliverables moving through the Quality Review process (i.e., tracking status in relation to the
different QA layers);
 Development of a tool to keep a record of all deliverables submitted and meetings carried out in
an Excel spreadsheet, and;
 Maintaining all administrative records relating to the project implementation.

Mr. Esa Ahlberg has more than 20 years experiences in planning and
conducting national and international civilian crisis management, civil
protection and humanitarian aid trainings and exercises in close collaboration
with International and regional organizations (EU, UN and NATO).
He has been working as a Senior Emergency Planning Officer in the United
Nations Interim Administration Mission for Kosovo (UNMIK) and as a Staff
Officer at NATO Headquarters Euro-Atlantic Disaster Response Coordination
Centre (EADRCC). He is trained EU Civil Protection expert, NATO Civil
Expert, and OPCW’s Qualified Expert. He has EU Civil Protection and NATO
mission experience. He has been involved in various EU projects in the field of civil protection.

Mr. Ahlberg will be involved in Task 1, 2 and 4 and will be predominantly concentrated on
data collection, data analysis and developing recommendations.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 85
Martin Jensen is Managing Director of VVA Brussels SPRL., a public policy
consultancy providing economic, market and regulatory advice to public
sector institutions. Mr Jensen provides direction and leadership on projects
and in business development activities (tenders, proposals, setting up
consortia etc.) in his capacity as Project Director. He steers the work focused
on providing top quality economic, market and regulatory advice. Mr Jensen
has more than 20 years of experience providing research and consultancy
services to the European Commission and the European Investment Bank
(EIB). Mr Jensen has specialised in providing evaluations and impact
assessments of EU policies and programmes, analysing EU governance models, activity based
management and budgeting, and the setting up of executive agencies).

Mr Jensen has participated in the evaluation of regulatory agencies (ECHA-2016, EMSA (2008)
and FRONTEX), has worked on activity based management with DG Budget and assisted the
Commission and EEAS in the preparatory CBA to establish a Shared Service Centre for EU
missions (2014). Mr Jensen has acquired extensive experience in carrying out CBA and
evaluations of EU executive agencies and governance structures (executive agencies and joint
undertakings). He was CBA expert on the initial Cost-Benefit analysis performed to establish the
TENT Executive Agency (2004 and 2005), responsible for modelling and carrying out the actual
cost calculations as well as the updating study part of the proposal to extend the mandate of
TENEA (2005). Since then Mr Jensen was responsible for the retrospective CBA carried out in the
interim evaluation of the EACEA (2009 and TEN-TEA (2007 and 2012). Mr Jensen was leading the
interim evaluation of EAHC (2010) and SESAR JU (2010 and 2014). He is therefore very
acquainted with agency evaluations as well as with the institutional set up and the underlying
regulations (financial regulation, staff regulation and agency regulations). Mr Jensen holds a BA in
Agriculture Economics and a MA in Economics from the University of Copenhagen.

Mr. Jensen will be leading Task 2 on cost analysis.

Magda Stepanyan has over 25 years’ experience in risk governance and


disaster and climate risk management, civil protection, national disaster risk
assessment, early recovery, post disaster needs assessment, preparedness,
resilience building, strategic planning and monitoring & evaluation. She has
worked extensively with multi‐national agencies including but not limited to the
World Bank, UNDP and other UN agencies, and European Commission, the
Norwegian Refugee Council. She has broad geographic experience ranging
across various countries in Eastern Europe, Western Balkans, South Asia,
Central Asia, South Caucasus, Europe, South East Asia, Africa, Caribbean region, and Arab states.

She holds two Master's degrees: MA Sociology (from Yerevan State University, Armenia) and MSc
Public Administration with specialization in Crisis and Safety Management (from the Leiden
University, the Netherlands). Also, she holds several advanced certificates from the world leading
universities: certificate in Leadership in Crisis from the Kennedy School of Government (KSG),
Harvard University, USA; certificate in Blue Ocean Strategy from INSEAD, France; certificate in
International Cooperation and Development from the UN University, Japan, certificate in Risk
Management from the Institute of Risk Management, UK, and many others. She is the CEO &
Founder of the Risk Society organisation (www.risk‐society.com) based in the Netherlands. She is
a member of the Sustainability Board of the DRIVER (Driving Innovation in Crisis Management for
European Resilience) and a visiting lecturer in the Crisis Management State Academy of Armenia.
She is fluent in Dutch, English, Armenian, Russian with good skills in French. She has co-authored
the National Disaster Risk Assessment (NDRA) Guideline (2017) commissioned by UNISDR under

86 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
‘Words into Action’ and authored Risk and Risk Management in Development Aid Practice chapter
in Risk Management Handbook: A practical guide to managing the multiple dimensions of risk as
well as several other publications on risk management and governance for development and
humanitarian fields.

Ms. Stepanyan will lead Task 3 on risk assessments and capability gap analysis.

Mr. Michael Felfernig is involved in disaster response since 20 years. He has


been deployed to several international and national missions. Since 2006 he is
working in the Austrian Ministry of the Interior in the International Civil Protection
Focal Point. Mr. Felfernig has been deployed to DG ECHO as national seconded
expert from 2012 to 2014. Mr. Felfernig has coordinated and supported different
international exercise and training activities as manager, organizer, trainer and role
player. This includes mechanism training courses and modules exercises. He has
been deployed within the Union Civil Protection Mechanism to six missions. Over the last twelve
years, he has progressively gained experience in training and educating staff on different levels in
emergency preparedness, crisis and emergency relief management and humanitarian affairs. This
mainly focused on national and international environments from field to strategic level (including
international natural disaster response missions for the Austrian Forcers Disaster Relief Unit -
AFDRU and the European Civil Protection Mechanism). He has a strong operational and
management background underpinned by practical leadership experience in the field of Civil
Protection and Disaster Management. Michael holds master degrees in political science,
environmental threats and disaster management, as well as in military leadership.

Mr. Felfernig will be involved in Task 1, 2 and 4 and will be predominantly concentrated on
data collection, data analysis and developing recommendations.

Agnes Rajacic is Senior Disaster Management officer at the IFRC. Her tasks consist of monitoring
on-going, developing and potential future disasters in Europe and in the Caucasus regions,
disasters and crisis structures activation, supporting necessary disaster response upon the need
and helping in applying response tools, supporting Europe Region National Societies in responding
to disaster, in planning, implementing operations in response to different types of disasters,
supporting the Red Cross societies to implement Disaster Risk Reduction efforts and actions and
provide technical input, supporting the implementation of Urban Risk Reduction Programmes in
Europe and Asia, and providing technical support to Red Cross National Societies in the Europe
Region to develop disaster management capacity in line with Federation policies and strategies.

Agnes will fulfil the function of subject expert in the core team, and will contribute to Task 4
activities.

Gabriëlle op ‘t Hoog is working as a junior consultant in the Security & Justice


sector of Ecorys. Having obtained Master degrees in international relations
(Linköping University, Sweden) and crisis and security management (Leiden
University, the Netherlands), she now mainly focuses on Crisis Management.
Throughout her academic and professional career, Gabriëlle has gained experience with discourse
analysis, conducting interviews, text analysis and processing large amounts of sources. Gabriëlle is
currently involved in the DG HOME initiative Community of Users on safe, secure and resilient
societies where she plays a coordinating role and develops information briefs on the Community of
Users High Level meetings. Furthermore, she is involved in the FP7 DRIVER+ project (Driving
Innovation in crisis management for European Resilience) where she is part of the team developing
the CMINE (Crisis Management Innovation Network Europe). Having experience in both the private

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 87
and public sector, Gabriëlle is comfortable coordinating projects with different types of stakeholders
and clients and she enjoys working in international environments.

Ms. Op ‘t Hoog will serve as support in Task 0 and 1 and will be involved in the general
project management, data collection and stakeholder consultation.

Task members

Luigi Lo Piparo has over 15 years of professional experience, mainly


achieved at programme or project level in the thematic areas of environment,
regional and urban development, climate change and cooperation. Since the
beginning of his career, he has worked closely with institutional and private
operators at European, national and local level, either to find financial solutions
for investment programmes and innovative projects, or to manage, evaluate,
and boost the impact of those initiatives. Luigi is currently managing director at Sagitta Evolution
and works as free-lance consultant with several European clients, providing consulting services for
project and programme management, evaluation, strategic communications and stakeholder
engagement.

His most recent contracts include the recent thematic management of the Interreg Europe Policy
Learning Platform on Low-carbon economy, the management of the Secretariat for the Urban
Agenda Air Quality Partnership (DG REGIO), the management of the Assistance Mechanism for
the Atlantic Action Plan (DG MARE), the ongoing evaluation of the INTERREG North-West Europe
programme, the mid-term evaluation of the LIFE programme (DG ENV), and the Support to
Copernicus user uptake promotion, communication and dissemination activities (DG GROW). Luigi
is also a certified PRINCE2 project manager with experience in leading teams for EU funded large
projects. He has an MA in Political Science and a Master’s degree in European Politics. He is
Italian native speaker, fluent in English, French and Swedish, with a good knowledge of Spanish.

Mr. Lo Piparo is the team’s stakeholder consultation expert.

Dr. Olga Batura is an associate consultant with Ecorys. Olga is a lawyer specialised in EU and
international law. She holds LL.M. Eur. (University of Bremen), Master of European Studies
(Europa-Kolleg Hamburg) and Dr.jur. from the University of Bremen. Before joining Ecorys in
December 2016, Olga worked as a researcher and lecturer at several universities in Germany, was
an associate professor at the European Humanities University in Vilnius, Lithuania, where she
conducted interdisciplinary research and/or taught courses with a strong interdisciplinary
perspective, and she also worked as a senior fellow at the University of Münster, Germany. At
Ecorys, Olga focuses on EU policy research and analysis, including impact assessments and
evaluations. Olga is experienced in various data collection methods, she has conducted surveys
and interviews herself and also supervised teams of national experts gathering data. Both in her
academic work and for various Ecorys projects, she has carried out qualitative data analysis and
case studies.

Ms. Batura will be involved in the data collection and stakeholder consultations of Task 1 as
well as she will be engaged in the analysis of the capacities.

88 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Agnese Macaluso is a consultant in the Regions and Cities sector of Ecorys.
She specializes in urban innovation and sustainable urban development,
regional and local economic development. Before joining Ecorys, Agnese was
a researcher in the domain of security and conflict prevention (violet extremism
and radicalization, urban security, education in post conflict settings). She has
vast experience with data collection and analysis, primarily with qualitative
methodologies, and has delivered methodology trainings internationally, including in DR. Congo
and Macedonia. Agnese has organized several stakeholder consultations, primarily through the use
of interviews and workshops. Notably she coordinated a large stakeholder engagement project on
violence and conflict prevention in The Hague (which included representatives from the education
sector, police, large and small businesses etc.). She has also run the secretariat of the Knowledge
Platform Security and Rule of Law, a network of NGOs, CSOs, research institutes and businesses
set by the Dutch Government to promote knowledge brokering between research and policy where
stakeholder mobilization was essential. Agnese is proficient in Italian, French, Spanish and English.

Ms. Macaluso is the teams in-house stakeholder consultation expert and will support Mr. Lo
Piparo.

Daina Belicka, Board Member, CSE COE, has in-depth knowledge and
understanding of finance and cost accounting boasting over 20 years of
experience at international audit companies. Daina is experienced Project
Manager for EU wide studies including DG MOVE Core Transport Corridor
Studies (phase 1 and 2), DG REGIO major projects quick appraisals, DG ENV
LIFE project financial monitoring and others. Daina’s cost accounting
experience was gained through different consulting projects, for instance,
developing complex business model for all Post Offices in Latvia, developing
business model for Public Services Delivery Agency in Albania covering all branches and more
than 20 different cost categories (project financed by World Bank), further elaborating Agency
Business model by including new cost items (project financed by Europeaid). Daina has obtained
Certified Accountant qualification (ACCA, UK), Certified Internal Auditor qualification (CIA, US),
Certified Information Systems Auditor qualification (CISA, US), Certified Project Manager (Prince 2)
and has MBA (Riga Business School in cooperation with University of Buffalo, US). Daina has been
lecturing at Riga Business School Financial Accounting and Managerial Accounting course to MBA
and executive MBA programs. Daina speaks Latvian, English and Russian.

Ms. Belicka will support Task 2, cost analysis.

Magdalena Klebba is a Researcher at VVA Economics and Policy. She has


expertise in a wide range of policy areas including adaptation to climate
change and development cooperation aid. At VVA she works on evaluations,
impact assessments, compliance checks of national legislations with
European Directives and other research studies. Her tasks and expertise
include data collection, such as desk research, interviews and surveys, as
well as legal analysis and analysis of gathered data, reporting of analysis and
conclusions. Prior to joining VVA in Brussels, she gained relevant professional experience at
OECD, where she was working as a trainee in Development Assistance Committee. Magdalena
holds a Master’s degree in Environmental Law from Paris-Sud University in France and a Master’s
degree in European Union International Relations and Diplomacy Studies from the College of
Europe in Belgium.

Ms. Klebba will support Task 2, cost analysis.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 89
Thiago Tavares is a civil engineer with a specialisation in transport with over eight years of work
experience in private organisations, public and international institutions. Throughout his career, he
has worked on various transport and infrastructure projects (i.e. road, rail, air) including economic
and policy analysis. He has acquired large experience in the collection and analysis of large
quantities of data as he was responsible for the data collection and analysis in several transport
projects. He has sound experience in the preparation of feasibility studies as well as policy impact
assessments and CBAs. He has developed strong knowledge about the challenges and costs
involved in the development and maintenance of transport and infrastructure projects but also on
different financing instruments such as blending and PPP.

Niels van Wanrooij works at the Security and Justice sector at Ecorys as a
consultant. He studied international political economic issues (MSc, London
School of Economics) and international relations (MSc, Radboud University
Nijmegen). As a Political Scientist, he has been extensively schooled in
qualitative and quantitative research methods (ranging from multiple regression
analysis to process tracing and case studies). He has worked on evaluation and
impact assessment projects for DGs HOME, RTD, ECFIN, and MOVE. Recent studies include the
‘impact assessment lite’ for Explosives Precursors (DG HOME), an evaluation for the SES initiative
(DG MOVE) and research into terrorism and organised crime (Dutch Ministry of Security and
Justice).

Mr. van Wanrooij will be involved in Task 3 and supports the risk assessments and capacity
gap analysis.

Dr Nga Nguyen has more than 10 years of experience as a highly qualified


economist, specialising in the areas of water and environmental economics,
energy and agri-food policy. She has led a broad range of projects providing
economic advisory services for public and private sector clients. Her
technical experience covers economic modelling and analysis, policy impact
assessment, cost effectiveness analysis, and stakeholder consultations.

Prior to joining Ecorys, she was an Economics Manager at Deloitte Australia for 2 years. She held
an Assistant Director and Principal Economist position at the Australian Bureau of Agricultural and
Resource Economics and Sciences (ABARES) for 6 years. Her earlier roles are Economist at
Mekong Economics in Vietnam, a Junior Economist at the Centre for International Economics in
Australia, and a Research and Teaching Assistant at Penn State University in the USA. Her PhD
research focused on the assessment of water quality trading market performance in the US context.

Ms. Nguyen will be involved in Task 4, the development of the assessment methodology and
the development of recommendations.

Hugo van Manen works as a consultant in Ecorys’ Security and Justice sector.
He is currently involved in the International Forum to Advance First Responder
Innovation, Community of Users, and DRIVER+ projects, in addition to DG
ECHO’s peer review programme. Hugo previously worked as an assistant
analyst at the Hague Centre for Strategic Studies, where he conducted
research (both qualitative and quantitative) into security-related issues that
advised policymaking within the Dutch Ministries of Defence and Foreign
Affairs. He holds an MSc in International Public Management and Policy (Public Administration)

90 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
from the Erasmus University in Rotterdam, as well as a BA in International Studies (International
Relations) from Leiden University in The Hague.

Mr. van Manen will be involved in Task 4, the development of the assessment methodology
and the development of recommendations.

Quality control

Brigitte Slot is director at Ecorys, and head of the Security and Justice team.
As such she has led many international projects in the field of migration
(participatory evaluation of European Immigration Liaison officers, study the
fraud and return policies related to the Schengen visa system), illicit financial
flows (money laundering and terrorist financing, financial investigation),
corruption, judicial institutions and organised crime. She has been director of
various justice related studies on behalf of the European Commission (DG Justice, DG Home and
OLAF) and the Dutch Ministry of Justice. Prior to her work as consultant she was senior policy
advisor on anti-money laundering and terrorist financing at the Ministry of Finance in the
Netherlands, where she was project manager of two international studies in money laundering on
behalf of the Financial Action Task Force (FATF): on money laundering in the football sector and on
maritime piracy. She was lead researcher on a study on ill-gotten money in two African countries
(Malawi and Namibia) on behalf of the World Bank. She started her career as international macro-
economic researcher in the financial sector and worked at was associated with the Delft Technical
University for her PhD.

Ms. Slot will be involved in the quality control team in which she will be specifically looking
at the consistency and coherence of the different deliverables.

Paul Baker is an independent economic consultant, an Associate


Consultant to the Ecorys Group, and a Fellow of the World Trade
Institute. He has 30 years of experience in applied economic research
and consultancy. He has worked extensively in the areas of industrial
policy development and the services economy, competitiveness analysis
and productivity issues, and sector strategies. He has contributed to many
policy papers, evaluations and impact assessments, and technical reports
for the European Commission, national and local administrations, and private sector clients. Paul
has also acted as an external expert to the European Economic and Social Committee’s
Consultative Commission on Industrial Change (EESC-CCMI) and the European Committee for
Standardization’s Strategic Advisory Group on Services (CEN-SAGS). In the field of security and
defence, Paul has implemented studies – either as a lead or senior consultant – for the European
Commission that have addressed “Competitiveness of the EU Security Industry” (2009), “Security
Regulation, Conformity Assessment and Certification” (2011), “Civil Military Synergies in the field of
Security” (2012), “Civil Security in Major Third Countries” (2013), “Development of Statistical Data
on the European Security Technological and Industrial Base” (2015) and “First Responders:
identifying Capability Gaps and corresponding Technology Requirements in the EU” (2016). In
addition, he was the economic advisor to a study on “Government-to-Government Sales of Defence
Equipment and Services” (2014) implemented by RAND for the European Commission. Paul also
supported DG Enterprise and Industry for the implementation of a ‘competitiveness proofing’
analysis undertaken as part of the Impact Assessment of the Commission’s Communication on
Security Industrial Policy (2012), and he was a member of the Independent Panel of Experts that
undertook the evaluation of the industry-led proposal for the Horizon 2020 cPPP on Cybersecurity

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 91
(2016). Paul holds a B.Sc. in Economics and Law (University College, Cardiff, UK) and a M.A. in
Economics (Queen’s University, Kingston, Canada). He has previously been employed at Ecorys
(Rotterdam, the Netherlands), BIPE Conseil (Paris, France), the Institute for Fiscal Studies
(London, UK) and University College, London (UK).

Mr. Baker will be involved in the Quality control team and will specifically look quality of the
UCPM and evaluation aspects of the project.

Anita Peeters has over 15 years of professional experience gained in


various international organisations, including focus on economic
development and international law, with roles in project support, tender
coordination & human resources. As a native English speaker and with
sound experience in drafting official correspondence and editorial work, she
is able to provide effective support in relation to proofreading, editing and
grammatical checks. She is able to draw on a variety of professional skills,
including bid management, administrative organisation and human
resources. Her operational experience includes projects across Europe, Africa and the Asia-Pacific
region. She is able to seamlessly coordinate communication across a range of stakeholders,
including government officials, international organisations and donor agencies.

Ms. Peeters is part of the quality control team and will concentrate on English language
checks.

Vibeke Weigelt is an experienced consultant with experience from private sector, public sector,
academia and civil sector. She has comprehensive experience with policy formulation, studies,
evaluations, Impact assessments and analysis. She has been in charge of numerous projects for
the EU including DG EMPL, DG JUST, DG Sanco, DG ENV and DG ECHO. Sectors include
different aspects of society: Civil protection, disaster prevention, employment, health, gender, and
social sector. She has solid experience with European policies. Her skills also include innovation
and process competence. She has solid knowledge of humanitarian aid and of the volunteer sector
both from heading a volunteer programme and from performing assignments for the sector, most
recently for Save the Children. She was in charge of the Preparatory study for an Impact
Assessment on the establishment of a European Voluntary Humanitarian Aid Corps for DG ECHO.
She has worked intensively with disaster risk reduction (DRR) and disaster risk management
(DRM), and was in charge of evaluating Evaluation of European Commission's Actions in the field
of Civil Protection 2007-2009 and involved in an expert capacity in the Ex post evaluation of Civil
Protection Financial Instrument and Community Civil Protection Mechanism (recast) 2007-2013.
Through numerous years of international work she has acquired solid project management skills
and experience with qualitative and quantitative methods. She has some 20 years of experience
with international projects and programmes of which 3 years with the European Commission as a
task manager in the EC Tacis programme at DG RELEX.

Ms. Weigelt is part of the quality control team and will specifically check the quality of the
UCPM-related analysis, evaluations and recommendations.

The data collection team consists of a large group of national correspondents who are presented in
the table below.

Table 3.1 National correspondents


Country VVA Ecorys
Austria Michael Flinckenschild

92 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Country VVA Ecorys
Belgium Elodie Salle
Bulgaria Diliana Mitovska
Croatia Tea Paulovic
Cyprus Alexandros Vigkos
Czech Republic Veronika Brantova
Martin Rune Jensen
Denmark
Estonia Triin Muuk-Adrat
Finland Vilma Kuuliala
France Agnese Macaluso
Germany Alexandra Schmid
Greece Ioannis Giannelos
Hungary Iva Plašilová
Iceland Martin Rune Jensen
Ireland Gabrielle op ‘t Hoog
Italy Agnese Macaluso
Latvia Vesna Sredanovic
Lithuania Milda Butkute
Luxembourg Luc Fischer
Macedonia Vesna Sredanovic
Malta Hugo van Maanen
Montenegro Tea Paulovic
Netherlands Niels van Wanrooij
Norway Neja Bergant
Poland Magdalena Klebba
Iva Plašilová
Portugal
Romania Madalina Nunu
Serbia Vesna Sredanovic
Slovakia Alena Cierna
Slovenia Neja Bergant
Spain Agnese Macaluso
Sweden Alma Agusti Strid
UK Nga Nguyen
Turkey Serdar Demirel

3.2.1 Overview

Core team
Table 3.2 Core team composition
Name Team role Level Organisation
Laura Birkman Project Director Senior Ecorys
Alexandra Schmid Project Coordinator Junior Ecorys
Gabriëlle op ‘t Hoog Project support Junior Ecorys
Esa Ahlberg Lead Task 1 Senior CMC
Martin Jensen Lead Task 2 Senior VVA
Magda Stepanyan Lead Task 3 Senior Risk Society
Michael Felfernig Lead Task 4 Senior CMC
Agnes Rajacic Subject expert Senior IFRC

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 93
Task teams
Table 3.3 Task 0 team composition
Name Team role Level Organisation
Laura Birkman Task lead Senior Ecorys
Alexandra Schmid Task support Junior Ecorys
Gabriëlle op ‘t Hoog Task support Junior Ecorys

Table 3.4 Task 1 team composition


Name Team role Level Organisation
Esa Ahlberg Task lead Senior CMC
Olga Batura Task support Senior Ecorys
Michael Felfernig Task support Senior CMC
Luigi Lo Piparo Stakeholder consultation Senior Sagitta Evolution
expert
Agnese Macaluso Support stakeholder Junior Ecorys
expert
Martin Jensen Task support Senior VVA
Gabriëlle op ‘t Hoog Task support Junior Ecorys

Table 3.5 Task 2 team composition


Name Team role Level Organisation
Martin Jensen Task lead Senior VVA
Diana Belicka Task support Senior VVA
Magdalena Klebba Task support Junior VVA
Thiago Tavares Task Support Junior VVA

Table 3.6 Task 3 team composition


Name Team role Level Organisation
Magda Stepanyan Task lead Senior Risk Society
Niels van Wanrooij Task support Junior Ecorys

Table 3.7 Task 4 team composition


Name Team role Level Organisation
Michael Felfernig Task lead Senior CMC
Agnes Rajacic Task support Senior IFRC
Nga Nguyen Task support Senior Ecorys
Esa Ahlberg Task support Senior CMC
Hugo van Manen Task support Junior Ecorys

Quality assurance team


Table 3.8 Quality assurance team composition
Name Team role Level Organisation
Brigitte Slot Quality control Senior Ecorys
Paul Baker Quality control Senior Ecorys
Anita Peeters Quality control Senior Ecorys
Vibeke Stockholm Quality control Senior VVA
Weigelt

94 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
3.2.2 Expertise of our team members
III = highly experienced, II = experienced, I = Some experience

Laura Birkman

Brigitte Slot

Paul Baker

Anita Peeters

Magda Stepanyan

Alexandra Schmid

Luigi Lo Piparo

Nga Nguyen

Agnese Macaluso

Niels van Wanrooij

Hugo van Manen

Gabriëlle op ‘t Hoog

Olga Batura

Mr. Esa Ahlberg

Mr. Michael Felfernig

Agnes Rajacic

Martin Jensen

Daina Belicka

Magdalena Klebba

Weigelt Vibeke

Thiago Tavares
1 Project management and coordination III III III III I III ||| III I I I II III III III III III I III III
2 Quality control II III III II II III || II I I III III III I III III
3 Logistical support to consortium II II II III I II | I I I I
4 Conducting and analysing interviews III III III III II III ||| II I I III II I III III III III III III
5 Designing and conducting survey III III II II II III | III II I I III I III III I III III
6 Literature analysis III III III III II III ||| III II I I III I I III III III III III III
7 Evaluation of EU programs and initiatives III III III III II III | II II II I III III III III III III
8 Cost analysis II II II II || I I I III III III
9 Quantitative analysis II II III II I III ||| II I I I II I III III III III III
10 Gap analysis II II III III III || II II I III III III II III III
11 Development of assessment methodology II II III III I III || II II I I II I III III II III III
12 Stakeholder engagement and consultation III III III III II III || III I I I III II I III III III III III
13 Organising and conducting workshops II II II III I III || III I I I III II III III III II III III
14 UCPM I II I III III III III III
15 Operational experience with UCPM II III III III II
16 UCPM response capacities I II II III III III
17 Assessment of UCPM capacities I I III III
18 Revision of UCPM capacities I I II III
19 Risk assessments I III II II III III III I III III
20 EERC II III II II III
21 CECIS II III II II III
22 Disaster Risk Reduction II III I
23 Disaster preparedness II III I
24 Post disaster needs assessment and recovery II III
25 Civil Protection II III I
26 International cooperation in disaster management II III
27 Strategic planning II III

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 95
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 97
4 Quality Control

This chapter sets out our bespoke quality control and assurance approach, which we have tailored
specifically in the light of the assignment requirements. The approach incorporates the extensive
experience of the consortium, and in particular, of the lead company Ecorys, with the
implementation of projects in the field of crisis management for the European Commission.

Many of the quality control measures proposed below have been tested in practice and adjusted to
suit the unique needs of the tendered-for project and the assembled consortium. Quality measures
described below concern both content and process quality assurance and are applied at both the
project level as well as at assignment-deliverable level. They follow the work flow from receiving a
request for service to completing the final quality compliance check of all resulting deliverables and
subsequent submission to DG HOME.

In the sections that follow, we first outline a number of principles informing our approach to quality
assurance within the context of this project, and describe our overall QA approach (Section 4.1).
We then elaborate the QA measures that we will put in place for guaranteeing high-quality
deliverables, tailored to the different types of outputs (Section 4.2), followed by a description of how
we address the process-related aspects of quality control (Section 4.3). The chapter then sets out
the risks and constraints as well as the risk mitigation measures (Section 4.4).

ISO certified procedures Ecorys is committed to ensuring its services comply


with the highest professional standards. The quality of our services is the key to
our long-term performance and reputation. Our quality management system for
this contract is built upon existing ISO certification procedures that are present
within the two major constituent entities of Ecorys (UK and NL).

4.1 Quality assurance principles and overall approach

Key principles of our quality assurance activities are as follows:


 Ensuring that there is clear written and verbal guidance between all consortium partners and
subcontractors to ensure that there is no ambiguity regarding assignment requirements, scope
and expectations, timeframe and quality of outputs for each of their assigned tasks. This
guidance will be provided through written documents, telephone calls, skype, and (where
applicable) face-to-face meetings;
 Linked to the above, maintaining regular and on-going communication within the consortium
to ensure that they are aware of up and coming tasks, aware of on-going client feedback and
that they feel part of a wider network of experts throughout the delivery period of the study.
Communications within the consortium will proceed through in-person meetings, systematically
scheduled direct calls, and online tools. The Project Core team will additionally make use of
online the online platforms Slack, Trello, and Asana to facilitate ad-hoc, informal communication
(Slack), and to allow the project and deputy project directors to delegate tasks to junior project
experts (Trello, Asana);
 Quality control which is geared towards ensuring not only content, but also process. The
provided QA activities go well beyond issues such as checking the quality of English, and
instead look at the actual content of each deliverable prior to submission. We have therefore

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 99
established a multi-layered quality assurance system, which ensures that all deliverables
undergo a dedicated quality checks for: (i) content, (ii) methodology, (iii) language, and (iv) form
and format. We have allocated appropriate resources (in persons and man-days) to properly
assess each deliverable, provide written and verbal comments on its content as well as provide
guidance on sources, data and references to information that will lead to its improvement;
 The designated junior project experts will act as ‘first-line’ quality controllers for all deliverables
produced by their respective task team, which will be screened on content, soundness of the
utilized methodology, etc. To ensure that all deliverables are screened by subject experts, we
have consciously included subject-matter experts (Expert Advisors) in the quality
assurance team in order to provide ad-hoc expert advice to our quality manager on difficult,
and/or politically sensitive issues when necessary.

With the above principles in mind, we have ensured that there is a robust, multi-layered review
process from the moment it is received by the quality control team to the moment it is submitted to
DG ECHO. Our procedures ensure that no deliverable will arrive on the Client’s desk without having
been thoroughly reviewed by the quality assurance team.

4.2 Guaranteeing high quality deliverables and services

In this section, we elaborate and tools and approach that will be used to ensure high quality
deliverables and services throughout the project’s 13-month duration. The main components of our
approach are:
 A multi-layered quality control process;
 A tailored approach to the different types of deliverables;
 A dedicated team of experts with clearly defined tasks and roles, and;
 Regular collection and analysis of feedback to enable lessons learned.

We elaborate each of these components below.

4.2.1 Multi-layered quality control procedures


The quality of our deliverables is a key priority. Within the project, our approach to ensuring high
quality deliverables distinguishes three main quality dimensions, namely:
 Content quality;
 Linguistic quality; and
 High editorial quality and consistency across all deliverables provided.
3.
4. These four dimensions comprise our multi-layered system of quality checks, which shall be
carried out by dedicated team members throughout the duration of the contract.

Table Figure 6 Scope of quality control across quality dimensions describes the scope of the
quality control measures covered at each level and who is responsible for carrying out the controls.

Figure 6 Scope of quality control across quality dimensions


Scope of quality control
Content Per deliverable
For each deliverable, experts shall review and validate the content of the information and
analysis provided for:
 Reliability: Information is based on credible and reputable sources; information is current
(up-to-date); data is gathered using appropriate methods; data is representative, etc.;

100 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Scope of quality control
 Accuracy: Analysis is factual and correct; information can be substantiated in more than one
reliable source; opinions can be supported by facts;
 Objectivity: Information and sources are unbiased;
 Relevance: Information and analysis address the assistance needs of the EC; information is
concise, avoiding repetition or superfluous content;
 Completeness: The content addresses the assignment requirements effectively; or in terms
of completeness of data and information presented;
 Coherence and Consistency: Analysis follows a logical flow and avoids logical
inconsistencies in argumentation and contradictory views.
Across-deliverables
Across deliverables within a given assignment, checks cover:
 Consistency in terms of overall quality of analysis (scope, detail, policy utility, etc.);
 Coherence and consistency in terms of terminology and definitions applied, aligned to those
of the Commission.
Responsible partner
Junior project experts (Project Core team), Quality Manager (Quality Assurance Group).
Linguistic Per deliverable
Proofreading by a linguistic editor or native English speaker shall include:
 Checks for errors in the use of the English language, including words, phrasing and
grammar;
 Detection and correction of typos, including incorrect punctuation.
Responsible partner
Linguistic editor (Quality Assurance Group). Project assistant (a native English speaker)
provides general language checks on deliverables not requiring professional proofreading.
Editorial Per deliverable
For each deliverable, editorial quality checks are carried out to ensure consistency in terms of:
 Form and structure of the information presented;
 Visual presentation (layout) of the reports adhere to style and formatting rules, including for
the use of visual graphics, where relevant. Compliance with EC requirements as contracted.
Responsible partner
Project director, supported by deputy project director (Project Core team).

All deliverables will be subject to this layered control process, with some variation in scope and
intensity depending on the nature of the deliverable.

Layers 1 and 2: High quality content and methodology


All deliverables will be reviewed for their content and methodological quality by the responsible
junior project expert. The junior project experts responsible for the submission of deliverables will
receive the comments from the project director and deputy project directors, and will be assisted
with guidance regarding required revisions and further drafting. Once completed, the relevant junior
project expert will review the revised deliverable to assure that all substantive comments have been
adequately addressed before then sending the deliverable to the Quality Manager for review. If a
deliverable remains unsatisfactory following a round of revisions, the junior project expert will notify
the project director, and provide technical advice to the Quality Manager regarding any concerns
over output quality.

Following the ‘first-line quality control’ carried out by the junior project experts, the Quality
Manager will conduct a second, high-level review of all deliverables, focusing on the overall quality
(content and methodology), coherence and consistency of the deliverables. In some cases, an
additional round of revision and review may be necessary, which will be factored into the work plan

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 101
and resourcing accordingly. In the case of complex and/or politically sensitive deliverables, the
Quality Manager will consult our Expert Advisors within the Quality Assurance Group.

The rationale behind our two-stage quality review of the content and methodological quality of the
deliverables is to ensure that all deliverables produced within the consortium’s various task teams
undergo a thorough review by subject matter experts rather than generalists .

Coherence and consistency of the review procedure will be ensured via the provision of both written
and verbal guidance, standardised checklists and reporting templates. These materials will be
prepared by the Project Core team, and will (where applicable) be tailored to the individual task
outputs. More specifically, the guidance will set out the review criteria and how each criterion is to
be understood in the context of the given assignment.

Layer 3: Linguistic quality


All deliverables will be subject to a language check. Deliverables which are to be disseminated
publicly will be professionally proofread by our consortium’s Linguistic Editor (Anita Peeters). The
Linguistic Editor will check and edit the text for linguistic style, errors in the use of the English
language, typos and punctuation. To ensure general readability, it is also important that documents
be concise and to the point, avoiding any repetition of material. For the purpose of the efficient use
of resources, we propose to do this once the final version of a given deliverable has been approved
by the Quality Assurance Group. Our approach to the language quality check is further detailed
below.

For all other deliverables, general language and linguistic style checks will be performed as part of
the final layer of quality control (see Layer 4 below), by the project director and deputy project
director, the latter being an English native speaker. At the onset of the contract, we will additionally
provide ‘style of writing’ guidance materials.

Ensuring native speaker English language quality


Ensuring the quality of our English language usage and consistency across deliverables is a high
priority, particularly for this assignment given the expected large number of deliverables to be
produced.

We will ensure that:


 Written style and presentation are adapted for the various relevant target audiences;
 The quality of English language is native speaker level and sufficient for dissemination (this
aspect is particularly important for those deliverables which will be disseminated or published
and not only for internal use of the Commission);
 Specific terminology is clearly defined, linked to a glossary;
 Tables, graphs, and other presentation tools are used to facilitate understanding; they are well
commented with narrative text;
 Documents fit client style guide and page length guidelines (if relevant).

We have an established quality procedure (see chapter 3.2.1) for ensuring a language quality
check as required for all outputs and deliverables.

For questions of English style, the Inter-institutional Style Guide76 and the English style guide77 will
be used as a reference. Once the final version of a given deliverable has been agreed, the
deliverable will be subject to a final check and detailed proofreading by our Linguistic Editor before

76
Inter-institutional Style Guide: http://publications.europa.eu/code/en/en-000100.htm.
77
http://ec.europa.eu/translation/english/guidelines/documents/styleguide_english_dgt_en.pdf.

102 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
submission to DG ECHO. The final check, editing and proofreading process will be coordinated by
the project’s project director. Furthermore, our QA process for written outputs provides a guide to
common referencing. All staff are required to be rigorous in the ways in which literature references
are stored, and expect that all written outputs will have a consistent structure. Our junior project
experts will check drafts for bibliographic consistency. Anti-plagiarism checks will also be carried
out as outlined below.

Layer 4: Editorial quality


Before submitting the final version of a given deliverable to DG ECHO, a final editorial check will be
performed by the project director, with support from the deputy project director during “high-volume”
periods. This check focuses on ensuring consistency and coherence of outputs in terms of form,
structure, presentation (format and layout) and compliance with EC requirements. In the first
instance, this will be done by members of the Project Core team (supported by the Quality
Assurance Group) at the outset of each individual assignment in the form of guidance material on
the (logical) structure of documents, definitional issues, checklists and templates to inform layout
and presentation.

4.2.2 Creating a learning consortium


A key element of our approach to management and quality control is to establish a system for
gathering and building on lessons learned throughout the project duration, with the aim of
continuously improving the quality of our outputs and services. This ‘continuous learning’ principle
applies to both individual assignments as well as to the overall performance and functioning of the
pilot programme as a whole.

At regular intervals, the project director and the quality manger, in consultation with the
Commission, will take stock of any identified quality issues regarding specific assignments. The
consortium, led by our project directors, additionally proposes to establish a regular feedback
mechanism with the respective project officers at DG ECHO. In particular, we will request
feedback regarding the quality and policy utility of the services offered through the pilot programme
and/or the functionality of tools such as the CMINE and eFIRE-IN. This feedback mechanism could
take the form of teleconferences organized following the completion of a given assignment, which
would be specifically dedicated to evaluating the performance and quality of service delivery on the
previous assignment; or during final report meetings with the Client within a given assignment.

Such discussions and feedback will serve the explicit purpose of helping the Project Core team to
continuously review and improve the quality of our support, as well as to identify any issues
requiring action and a forum for proposing appropriate and timely mitigation actions.

The project director will brief the Project Core team on the outcomes of these sessions, particularly
regarding lessons learned to be adopted in the execution of future activities. This system will
ensure a learning approach within the consortium and foster a strong sense of network coherence
and consistency.

4.2.3 Additional quality control measures


Plagiarism precautions
The Consortium strongly condemns plagiarism – the copying of texts, visualisations or other forms
of information of other authors into own work without any or improper referencing – in any form and
will therefore ensure that it does not occur on any of the proposals or project reports it submits to
DG ECHO.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 103
The Consortium has therefore put a few quality control processes in place to address this issue:
1. The Ecorys-led Consortium has obtained ‘Safe-Assignment’ software to check for plagiarism;
2. In the internal Consortium Agreement as well as in the sub-contracting agreements there is a
penalty provision for plagiarism.

When plagiarism is detected, both the project director and the quality manager will be notified and
pointed to the section or sections where the plagiarism was detected. These will be rewritten
immediately. In the Consortium Agreement between the partners, a specific clause states that the
project director will issue a first warning to the junior project expert whose task team is responsible
for authoring the offending content. If the plagiarism is not taken care of fully by the relevant task
team, sanctions will be executed accordingly.

Graphical requirements
We will adhere to all the graphic requirements with regard to the form of the final deliverable(s). The
consortium will make use of standardised Word templates (with automatic headings and paragraph
styles) during the drafting of each deliverables. For deliverables in which specialised visual work is
of relevance, the consortium shall ensure that formatting and layout conform to the corporate visual
identity of the EC by applying the graphic rules set out in the EC’s Visual Identify Manual, including
its logo. The final layout for all deliverables will be verified by our experienced secretarial staff
based in the Ecorys offices in Brussels and Rotterdam.

4.3 Guaranteeing smooth and efficient implementation

The consortium has embedded various measures to warrant the smooth and efficient functioning of
the project execution. Next to the overall management structures, as explained in detail the
previous chapter, in this section we elaborate the quality mechanisms that have been put in place
to ensure timely execution of assignments and the continuity of service in view of replacements of
staff and dealing with multiple assignments.

4.3.1 Respecting deadlines and timely delivery


The consortium places great importance on ensuring the timely delivery of all deliverables. Given
the thorough quality review procedure which has (as outlined in the preceding sections) been put in
place, it is of the utmost importance to establish a clear roadmap of the working procedures,
milestones and deadlines under each task by means of careful and realistic planning, as well as
continuous monitoring thereof.

The project director and deputy project director will be constantly monitoring progress and respect
of deadlines. Our project director – in close consultation with the junior project experts – will be in
charge of ensuring the operational adherence to the timetables as set out in each contract and the
precise implementation of a detailed production plan for each deliverable, setting deadlines and
responsibilities for each step of the process.

In order to efficiently monitor the progress of the large number of deliverables foreseen by several
of the tasks outlined in the ToR, we will create an excel database in order to track the status of all
deliverables, including their movement through the multi-layered quality assurance procedure. It will
be maintained by the deputy project director, who will monitor the completion of all steps in our
review process, and flag risks of delay to our project director in the event that particular deliverables
require multiple rounds of content and methodology revision.

104 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
4.3.2 Ensuring continuity of service
The consortium views the continuity of the service during a project’s implementation phase as
imperative to its overall success. Because – as is the case with (among others) Tasks 3, 4, and 5
as outlined in the ToR – the interim outputs of several this project’s Tasks feed into the completion
of deliverables in others, a key challenge presents in the simultaneous management of various
processes and assignments. Outside of requiring a robust management structure, this challenge
can be (partially) mitigated through contingency planning. In principle, the team has (through the
FIRE-IN network) access to expert networks – both formal and informal – in the field wildfire
prevention which extend well beyond those which are presented within this proposal. This
guarantees continuous access to sufficient capacity at all times. In the (unlikely) eventuality that
project workloads surpass team capacity and/or that key knowledge is found to be absent, the
consortium will supplement its capacity by subcontracting high level experts from the extensive
network to which it has access.

Subcontracting arrangements
We have built into our subcontracting arrangements (and foresee in the contractual agreement) a
provision to accommodate the replacement of a subject matter expert if required. This will also
provide flexibility in the case of changes in the needs of DG ECHO. This procedure will be
instigated if:
 The project officers at DG ECHO do not feel that the task teams’ approaches and/or outputs are
acceptable or of high enough quality;
 The subject matter expert continually produces sub-standard quality deliverables- this includes
producing badly written, poorly analysed, poorly referenced outputs which are not to a standard
expected of Ecorys and DG ECHO;
 The subject matter expert continually produces late deliverables- including outputs that are sent
to Ecorys beyond the deadline given.

We will inform the client of a proposed change of expert in writing, explaining the reason behind this
decision. We will then seek to source a replacement expert and send their CV and background
details to the client for formal approval. After client agreement, we will then fully brief the new expert
on expectations and tasks before their commencement of the role.

4.4 Risk management and mitigation strategies

It is our aim to be transparent and pro-active about the potential risks that the consortium may face
during the provision of the requested services. In the tables below we present potential risks,
including their probability and impact (L = low, M = medium and H = high), as well as our preventive
and contingency measures. These comprise more generic risks that are valid in the overall
management of the contract as well as specific risks related to the Tasks subsumed under this
project.

Table 4.1 Risks data collection strategies


Challenge I P Mitigation measure
Delay in delivery due to tight H M  Discuss with ECHO clear timeline of action
timeline to design, launch and  Have a broad team in place to implement and analyse
analyse results (especially for results
the survey)  Make a large use of web-based surveys with multiple
choice answers (mostly closed)
 Diligent planning of the project timeline and identification
of critical path and milestones in the project

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 105
Challenge I P Mitigation measure
implementation. Inclusion of safety margins around
critical milestones to reduce the impact of encountered
delays by having regularly team meetings.
Difficulty in identifying the H M Mobilise national experts who have direct access and can
stakeholders engage the stakeholders, or reach out to international and
local platforms to facilitate access
Low/participation response H H  Pre-alert participants before survey/interviews are held,
rate explain context, confidentiality, potential issues with
sensitivity
 Send reminders and use networks above to facilitate
 Design of questionnaires with a reasonable number
of questions (mostly closed type of questions)
 Test of questionnaires prior to publishing
 Support from the client in identifying the most proper
stakeholders and provision of email contacts
Use of interviews and to provide further answers
Follow up e-mails and phone calls
Need to use local language M M Deploy experts able to cover all languages, or most of them.
with Survey can be translated easily with no need of a local
surveys/interviews/workshops expert.
Little availability of data H M As such, for a proper evidence-based evaluation, we will rely
on input from coming from different sources. We expect that
much of this information requirement can be covered with
input from targeted survey and interviews.

To address the challenge of “information access”, we


propose to have a comprehensive assessment of the
existing information and data early on, complemented by
access to stakeholders that allows knowledge and
information gaps to be filled when required and, equally
importantly, to identify additional information needs early on
and propose alternative sources.

Alternatively, the national UCPM experts can be approached


to help retrieve specific information.
Limited willingness of private H M In order to ensure that our targets for the interview
stakeholders to participate in programme are met, every effort will be made to encourage
study participation. A focused client recommendation letter
stressing the importance of stakeholder participation will be
requested and sent out to relevant organisations.
Bodies governed by public H M In our experience it can be difficult to contact the right
law may not wish to be contact person at public authorities. We have taken the
interviewed/participate in following steps to ensure an adequate response:
stakeholder  We will ask for a letter of recommendation;
engagement activities  We will put in a substantial effort to get the right
contacts, with up to 3 contact attempts being budgeted.
The results of the study do M L The method for obtaining results of the study is reasonable
not match stakeholders point and traceable; stakeholders’ point of view is marked as an
of view due to individual opinion.
interests

106 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Challenge I P Mitigation measure
Non-coherent presentation M M Detailed and rigorous preparation of the templates
of information across various and protocols and guidelines for the interviews,
Member rigorous quality control of the performed activities by
States/interviewees/other the task leaders, regular meetings with the team to
categories update on such a process and quality of the
intermediate deliverables
Difficulty to retrieve data by M M In case the national correspondents experience severe
national correspondents difficulties obtaining information, the project team has set
aside budget to involve national experts who are better
connected and will most likely be able to retrieve this
information more effectively. The list of national contact
points in Annex I will be consulted and national experts will
be approached based on which national correspondents
experience most difficulties.

Table 4.2 Risks Task 0


Risk I P Mitigation
Lack of coordination H L All task leaders are part of the core team and meet regularly
between different tasks (virtually) to align the activities ongoing within their
respective task and to ensure the outcomes of the tasks
feed into each other efficiently. By ensuring close
connections and communication between the Task leaders,
the risk of miscommunication and inefficiencies is mitigated.
Tasks are delayed and, H M The tight planning of the project at hand requires all involved
thereby, affect the execution parties to take their responsibilities in time. In order to clarify
of other tasks who needs to deliver what and at which point in time, the
proposal outlines clearly which deliverables need input from
which tasks as well as when the Client needs to
deliver/approve. This clear task division helps to structure
the project and mitigates the risk of delays.

Table 4.3 Risks Task 1


Risk I P Mitigation
Inability to differentiate H L From the onset of the project, the study team will be strongly
between modules of supported by experienced civil protection experts. We will
category B and category C, rely on their advice to interpret data and resolve
for instance, due to inconsistencies.
contradictory/ insufficient In addition, we will search for additional data via desk
data and inconclusive research and interviews and triangulate them. We will also
data/opinions of seek advice from the client and DG ECHO/ERCC experts to
stakeholders validate data and our findings.
Inability to develop H L We will draft definitions and requirements based on the input
definitions, quality from stakeholders, and we will discuss the suggested
requirements and/or general definitions and requirements with them during the project.
requirements acceptable Lastly, we will also validate the suggested definitions with
and suitable for all stakeholders before presenting them to the client. This
Participating Countries. approach ensures that all stakeholder considerations are

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 107
Risk I P Mitigation
taking into account and they have been included in the
process of the revision.

Table 4.4 Risks Task 2


Risk I P Mitigation
Insufficient number of M M Close cooperation and planning with other project tasks,
participants selected for involvement of industry experts from other tasks,
targeted consultation cooperation with national stakeholders
Insufficient cost data about M M Close cooperation with experts from other tasks in order to
capacities identify suppliers
Close cooperation with national experts who can provide an
information about capacities and suppliers
Subjective view in the M M Close cooperation with industry suppliers in order to get their
technology scenario views on technical aspects of capacities and data verification
definition from different data sources

]Table 4.5 Risks Task 3


Risk I P Mitigation
Not all national risk L H The desk study will be supplemented by stakeholder
assessment data are openly consultations and interviews to collect data on missing
available (some parts of the capacities
assessment might be
classified)
The time allocation for Task H M Well-designed stakeholder mapping and timely planning of
3 and the tight deadline (8 all interviews and consultations and the validation workshop.
weeks after the inception Support letter from the DG ECHO for all (validation)
report) might not be workshops within the project to ease the process and
sufficient to allocate all the stakeholders engagement.
interviews planned and
ensure necessary
representation during the
validation workshop
Restricted access to CECIS M H The assistance of DG ECHO will be requested to advice on
might not allow detailed the list of identified capacities, which of them should be
analysis of the available at further elaborated within the project and which could be
the national level buffer considered sufficiently backed by the buffer capacities. It is
capacities vis-à-vis identified expected that the DG ECHO would be able to provide such
existing and emerging risks an insight based on their access to CECIS without going into
specifics of the quantity and quality of the buffer capacities,
which might be classified information.

Table 4.6 Risks Task 4


Risk I P Mitigation
Data collected throughout H M Interview data is expected to be of varying quality as a result
Tasks 1, 2, 3 is insufficiently of differences in interviewers’ personalities, discrepancies
detailed to allow for the between interviewees’ willingness to volunteer information,
development and/or and unavailability on the part of key interviewees. To
application of the compensate for this risk, the Task 4 team has planned Task
methodological framework. 4.2 to allow for a.) the use of the 3rd round of interviews to

108 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Risk I P Mitigation
collect relevant data (in case the 2nd round proves
insufficient, and b.) the conduct of additional (phone-based)
interviews by the Task team itself. In addition, considerable
resources will be invested into ensuring desk research is as
productive as possible.
Desk research does not M M The Task 4 team will have access to highly experienced
result in sufficient experts, which will provide them with relevant venues to
information to facilitate the pursue, and which will validate the quality of all Task
preparation of interview deliverables.
material.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 109
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 111
Annex I National contact points UCPM

National contact points UCPM


Country Contact point Contact person E-mail address
Austria Ministry of Interior TBC bmi-II-13-b@bmi.gv.at
Belgium Ministry of Interior Gaetan Willems gaetan.willems@ibz.fgov.be
Bulgaria DG Fire Safety and Civil Nikolay Nikolov sacp@cp.government.bg
Protection/Permanent
Representation
Croatia National Protection and Dragan Lozančić kabinet@duzs.hr
Rescue Directorate
Cyprus Cyprus Civil Defence Marilena Ayiomamitou mayiomamitou@dgepcd.gov.cy
Czech Republic Fire and Rescue Service Drahoslav Ryba sekretariat.gr@grh.izscr.cz
Denmark Danish Emergency Martin Thomsen brs@brs.dk
Management Agency
Estonia Estonian Rescue Mari Tikan mari.tikan@siseministeerium.ee
Board/Ministry of Interior
Finland Ministry of Interior Janne Koivukoski janne.koivukoski@intermin.fi
France Permanent Pierre Orszag pierre.orszag@interieur.gouv.fr
Representation
Germany Ministry of Interior Claus Höllein claus.hoellein@thw.de
Greece Permanent Samaltanos Andreas andreas@samaltanos.net
Representation
Hungary Permanent István Endrődi pvszovetseg@katved.gov.hu
Representation
Iceland Department of Civil Hjálmar Björgvinsson hjalmar@logreglan.is
Protection and
Emergency
Management
Ireland Ministry of Environment James McConnell james.mcconnell@dublincity.ie
Italy Civil Protection Angelo Borrelli segreteriacd@protezionecivile.it
Department
Lithuania State Fire and Rescue Raimondas Žičkus raimondas.zickus@neringa.lt
Services
Latvia Permanent Sanita Pavļuta- permrep.eu@mfa.gov.lv
Representation Deslandes
Luxemburg Administration des Clément Gorlt gpki@incert.lu
services de secours
Macedonia Pavle Bogoevski p.bogoevski@sobranie.mk

Malta Ministry for Home Affairs Emanuel Psaila emanuel.a.psaila@gov.mt


Montenegro Danilo Miljević mup.emergency.112@t-com.me
Netherlands Ministry of Security and Paul Gelton p.t.gelton@nctv.minvenj.nl
Justice/ National Lodewijk Hekking l.b.hekking@minvenj.nl
Coordinator for Security
and Counterterrorism
Norway Directorate for Civil Karl Kerner karl.kerner@dsb.no
Protection

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 113
Country Contact point Contact person E-mail address
Poland State Fire Service Wiesław Puchalski bwm@kgpsp.gov.pl
Portugal Civil Protection Authority Rute Ricardo Rute.ricardo@prociv.pt
Romania Permanent loan Nitica ioni6I@yahoo.com
Representation/Ministry
of Interior
Serbia Ivan Baras ivan.baras@mup.gov.rs
Slovakia Permanent Ingrid Stanova ingrid.stanova@mzv.sk
Representation
Slovenia Permanent Srečko Šestan srecko.sestan(at)mors.si
Representation/Civil
Protection
Administration
Spain Ministry of Interior Juan Crespo Sanchez juan.crespo@policia.es
Sweden MSB/Ministry of Defence Anders M. Johansson anders.m.johansson@civil.se
United-Kingdom Cabinet Office Simon Lewis simonlewis@redcross.org.uk
Turkey Prime Ministry Disaster Hüseyin Alp Kaya halp.kaya@afad.gov.tr
and Emergency

114 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Country Authority Name
Austria Federal Ministry of the interior Institute for Johann BEZDEKA
science and research Robert STOCKER
National Civil Protection and Disaster Relief Karin ZETTELMANN
Affairs Mirjana JAKOPEC
Kurt KALCHER
Hubert VETTER
Belgium Home Affairs, National Centre of expertise for Jérôme GLORIE
civil protection Marc LOOZE
Sebastiaan DRUYTS
Anne BAETENS
Bulgaria Ministry of State Policy for Disasters and Nikolay NIKOLOV
Accidents DG
National Civil Protection Antoaneta BOYCHEVA
Maria STANIMIROVA
Croatia National Protection and Rescue Directorate Jadran Perinić
Cyprus Ministry of Interior - Cyprus Civil Defence Force Maria PAPA
HQ
Czech Republic General Directorat of Czech Fire and Rescue Jiri CHALUPA
Service - Ministry of Interior Drahoslav RYBA
Miloš SVOBODA
Jiří MUSÍLEK
Denmark Danish Emergency Management Agency Simon Søborg AGGER
(DEMA), Centre for Education and HR Michael Boesgaard BRØNDEL
Development Flemming S. NIELSEN
Ministry of Defence
Estonia Estonian Rescue Board Kuno TAMMEARU
Alo TAMMSALU
Jevgeni Jutkevitś
Finland Ministry of the Interior Esko Koskinen
Eero KYTÖMAA
France Ministry of Interior, Direction of defence and Manuel VALLS
civil security Philippe NARDIN
Germany Federal Ministry of the Interior Franz-Josef HAMMERL
Dr. Klaus-Georg MEYER-
TESCHENDORF
Juergen POLKA
Franz Josef MOLITOR
Greece Ministry of Public Order and Citizen Protection Patroklos GEORGIADIS
General Secretariat for Civil Protection Dr. Kalliopi SAINI
Dr. Olga KAKALIAGOU
Hungary Ministry of the Interior - National Directorate Dr. György BAKONDI
General for Disaster Management (NDGDM) Dr. Attila NYIKOS
Ireland Department of Environment, Community & Kenny BRIAN
Local Government
Italy Presidenza del Consiglio dei Ministri - Dr. Franco GABRIELLI
Dipartimento della Protezione Civile Luigi D’ANGELO
Latvia Latvian State Fire and Rescue Service Oskars ĀBOLIŅŠ
Jānis IVANOVSKIS
Lithuania Fire and Rescue Department, Ministry of General Remigijus BANIULIS
Interior Tatjana MILKAMANOVIC
Raimondas BOGDEVICIUS
Luxemburg Ministry of Interior Jean-Marie HALSDORF
Malta Civil Protection Department John RIZZO

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 115
Albert TABONE
Michel GALEA
Peter-Paul COLEIRO
Netherlands Ministry of Security and Justice Theo LODDER
Martijn Meeder
National Crisis Centre Chris van Duuren
Poland National Headquarters of the State Fire Service Wieslaw LESNIAKIEWICZ
Malgorzata TRZCINSKA
Tomasz KOLODZIEJCZYK
Maciej GODLEWSKI
Adrian BUCALOWSKI
Dariusz KADUKOWSKI
Anita KOŁODYŃSKA
Portugal ANPC - National Authority for Civil Protection Major-General Manuel COUTO
Ana FREITAS
Romania General Inspectorate for Emergency Situations Colonel Nicolae CORNEA
Ministry of Internal Affaires Colonel eng. Marius DOGEANU
Major Vlad PETRE
Slovakia Ministry of Interior of the Slovak Republic Lenka Hmírová
Section of Crisis Management Andrea Geguššová
Jaroslav Valko
Slovenia Ministry of Defence -Administration for Civil Darko But
Protection and Disaster Relief Danica Babič
Spain Ministerio del Interior -Dirección General de Juan Antonio DÍAZ CRUZ
Protección Civil y Emergencias uan Pedro LAHORE LACOSTE-
PEDELABORDE
Sweden Swedish Civil Contingencies Agency (MSB)- Helena Lindberg
Swedish Rescue Services Agency Therese Wikström
Selma Ilijazović
UK Cabinet Office
Civil Contingencies Secretariat
Iceland Icelandic Police – Civil Protection Vidir REYNISSON
Guðrún JÓHANNESDÓTTIR
Norway Ministry of Justice and Public Security Ørjan N. KARLSSON
Geir Arne FREDRIKSEN
Hanne B S ERIKSEN
Turkey Prime Ministry Disaster and Emergency Fuat OKTAY
Management Presidency
FYROM Protection and Rescue Directorate of the Shaban Saliu
former Yugoslav Republic of Macedonia

116 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Annex II Interview guidelines

Lot 1

Interview Guide Lot 1


Introduction
 Introduction of the study
 Introduction of the interviewee

Task 1
 National Civil Protection needs
 Match between needs on national level and UCPM capacities in case of emergency in participating
states;
 Capacities of UCPM Voluntary Pool
 Capacities of CECIS
 Capacities of European Civil Protection Pool
 Capacities foreseen in rescEU
 National Civil Protection capacities of the participating states that could be used in UCPM operations
(not included in the current UCPM Voluntary Pool or CECIS either in the planned European Civil
Protection Pool or rescEU);
 Risks regarding civil protection outside the UCPM participating states
 What additional Civil Protection assets or support is be needed regarding disasters outside UCPM
participating states
 Adjustments in modules/capacities necessary in the view of new/ changed risks
 Adjustments in response capacities necessary based on past experiences (exercises, operations,
training)
 Interoperability issues for experts
 Other requirements to experts

Task 2
 Capacities costs
 Development costs
 Transport costs
 Operational costs
 Maintenance costs
 Background and technical information on capacities
 Cost scenarios defined for certain capacities

Task 3
 National Civil Protection risks
 Match between risks and needs on the national level

Lot 2

Interview Guide Lot 2


Introduction

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 117
Interview Guide Lot 2
 Introduction of the study
 Introduction of the interviewee

Task 1
 National Civil Protection needs
 Match between needs on national level and UCPM capacities in case of emergency in participating
states;
 Capacities of UCPM Voluntary Pool
 Capacities of CECIS
 Capacities of European Civil Protection Pool
 Capacities foreseen in rescEU
 National Civil Protection capacities of the participating states that could be used in UCPM operations
(not included in the current UCPM Voluntary Pool or CECIS either in the planned European Civil
Protection Pool or rescEU);
 Risks regarding civil protection outside the UCPM participating states
 What additional Civil Protection assets or support is be needed regarding disasters outside UCPM
participating states
 Adjustments in modules/ capacities necessary in the view of new/ changed risks
 Adjustments in response capacities necessary based on past experiences (exercises, operations,
training)
 Interoperability issues for experts
 Other requirements to experts

Task 2
 Capacities costs
 Development costs
 Transport costs
 Operational costs
 Maintenance costs
 Background and technical information on capacities
 Cost scenarios defined for certain capacities

Task 4
 How do other countries and/or organisations (if applicable) define the technical and/or operational
requirements for their capacity goals? In the event that these differ from those outlined in ANNEX II of
the UCPM implementing decision (2014/762/EU), what lessons can be learnt from the observed
discrepancy?
 What criticism has been regularly leveraged against the UCPM capacity goals (and on what grounds)?
 Which UCPM participating states have made the best use of the mechanism?
 Which UCPM participating states are most likely to need the mechanism in the next 5 years? Based on
this, are there any omissions within the UCPM’s current capacity goals?
 Who are the key stakeholders to interview when it comes to identifying shortcomings in the UCPM
(technical or otherwise) and/or to identifying its drivers?

Lot 3

Interview Guide Lot 3


Introduction

118 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Interview Guide Lot 3
 Introduction of the study
 Introduction of the interviewee

Task 1
 What implications a specific definition X (quality requirement Y) has for your national civil protection
response capacities (application of the definition, possibility to implement, cost factors)?
 Does a spoecific requirement X improve/ ensure interoperability of the expert?

Task 2
 Capacities costs
 Development costs
 Transport costs
 Operational costs
 Maintenance costs
 Background and technical information on capacities
 Cost scenarios defined for certain capacities

Task 4
 How do other countries and/or organisations (if applicable) define the technical and/or operational
requirements for their capacity goals? In the event that these differ from those outlined in ANNEX II of
the UCPM implementing decision (2014/762/EU), what lessons can be learnt from the observed
discrepancy?
 What criticism has been regularly leveraged against the UCPM capacity goals (and on what grounds)?
 Which UCPM participating states have made the best use of the mechanism?
Which UCPM participating states are most likely to need the mechanism in the next 5 years? Based on this,
are there any omissions within the UCPM’s current capacity goals?
Who are the key stakeholders to interview when it comes to identifying shortcomings in the UCPM (technical
or otherwise) and/or to identifying its drivers?

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 119
Annex III Checkmarket survey

Example of a Checkmarket survey conducted by Ecorys:

120 FJK97864
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 121
122 FJK97864
About Ecorys

Ecorys is a leading international research and consultancy company, addressing society's key
challenges. With world-class research-based consultancy, we help public and private clients make
and implement informed decisions leading to positive impact on society. We support our clients with
sound analysis and inspiring ideas, practical solutions and delivery of projects for complex market,
policy and management issues.

In 1929, businessmen from what is now Erasmus University Rotterdam founded the Netherlands
Economic Institute (NEI). Its goal was to bridge the opposing worlds of economic research and
business – in 2000, this much respected Institute became Ecorys.

Throughout the years, Ecorys expanded across the globe, with offices in Europe, Africa, the Middle
East and Asia. Our staff originates from many different cultural backgrounds and areas of expertise
because we believe in the power that different perspectives bring to our organisation and our
clients.

Ecorys excels in seven areas of expertise:


- Economic growth;
- Social policy;
- Natural resources;
- Regions & Cities;
- Transport & Infrastructure;
- Public sector reform;
- Security & Justice.

Ecorys offers a clear set of products and services:


- preparation and formulation of policies;
- programme management;
- communications;
- capacity building;
- monitoring and evaluation.

We value our independence, our integrity and our partners. We care about the environment in
which we work and live. We have an active Corporate Social Responsibility policy, which aims to
create shared value that benefits society and business. We are ISO 14001 certified, supported by
all our staff.

Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 123
P.O. Box 4175
3006 AD Rotterdam
The Netherlands

Watermanweg 44
3067 GG Rotterdam
The Netherlands

T +31 (0)10 453 88 00


F +31 (0)10 453 07 68
E netherlands@ecorys.com
Registration no. 24316726

W www.ecorys.nl

Sound analysis, inspiring ideas

BELGIUM – BULGARIA – CROATIA – INDIA – THE NETHERLANDS – POLAND – SPAIN – TURKEY – UNITED KINGDOM

You might also like