Professional Documents
Culture Documents
Client: DG ECHO
Client: DG ECHO
1 Introduction 9
1.1 Members of the consortium 9
1.2 Key strengths of the consortium 11
1.3 Background of the study 12
1.3.1 Genesis of the European Union’s civil protection system 12
1.3.2 Functioning of the European Union Civil Protection Mechanism 13
1.3.3 Main features of current response capacities 17
1.3.4 Appropriateness of the existing response capacities in the UCPM 18
1.3.5 Available new response solutions 21
1.4 Purpose and scope of the study 22
2 Methodology 24
2.1 General approach 24
2.1.1 How the tasks are interlinked 24
2.2 Proposed tasks and subtasks 25
2.3 Proposed methodology 26
2.3.1 Data collection 27
2.3.2 Risk and mitigation strategies 41
2.4 Task 0: Project Management 44
2.4.1 Objective 44
2.4.2 Proposed methodology 45
2.4.3 Deliverables 48
2.4.4 Risks and mitigation strategies 49
2.5 Task 1: Review and (re-)define the existing response capacities 50
2.5.1 Objective 50
2.5.2 Proposed methodology 51
2.5.3 Deliverables 56
2.5.4 Risks and mitigation strategies 56
2.6 Task 2: Cost analysis 57
2.6.1 Objective 57
2.6.2 Proposed methodology 58
2.6.3 Deliverables 64
2.6.4 Risks and mitigation strategies 65
2.7 Task 3: Risk based assessment and capacity gap analysis 65
2.7.1 Objective 65
2.7.2 Proposed methodology 66
2.7.3 Deliverables 69
2.7.4 Risks and mitigation strategies 70
2.8 Task 4: Revision of capacity goals 70
2.8.1 Objective 70
2.8.2 Proposed methodology 71
2.8.3 Deliverables 77
2.8.4 Risks and mitigation strategies 77
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 5
2.9 Task 5: Logistical support Error! Bookmark not defined.
2.9.1 Objective Error! Bookmark not defined.
2.9.2 Proposed methodology Error! Bookmark not defined.
2.9.3 Deliverables Error! Bookmark not defined.
2.9.4 Risks and mitigation strategies Error! Bookmark not defined.
2.10 Planning and deliverables 79
3 Project Management 82
3.1 Consortium structure Error! Bookmark not defined.
3.2 Team structure 82
3.3 Roles and responsibilities 83
3.4 Introduction to the team members Error! Bookmark not defined.
3.4.1 Overview 93
3.4.2 Expertise of our team members 95
3.4.3 Profiles of the team members Error! Bookmark not defined.
3.5 Team resources Error! Bookmark not defined.
4 Quality Control 99
4.1 Quality assurance principles and overall approach 99
4.2 Guaranteeing high quality deliverables and services 100
4.2.1 Multi-layered quality control procedures 100
4.2.2 Creating a learning consortium 103
4.2.3 Additional quality control measures 103
4.3 Guaranteeing smooth and efficient implementation 104
4.3.1 Respecting deadlines and timely delivery 104
4.3.2 Ensuring continuity of service 105
4.4 Risk management and mitigation strategies 105
6 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 7
1 Introduction
Ecorys has a remarkable 80-year history, which started in 1929 with the establishment of the
Netherlands Economic Institute in Rotterdam by Nobel Prize winner Prof. Jan Tinbergen. With
headquarters in the Netherlands and more than 12 offices around Europe, including Brussels and
the United Kingdom, we provide a full range of services, from research and policy advice, through
institution building and capacity development, to large programme management and
implementation, communication and financing advice, and monitoring and evaluation. Ecorys
specialises in economic and social development and is active at all levels of policymaking and work
worldwide on complex market, policy, competitiveness and management issues. Ecorys has a
strong track-record and excellent in-house expertise in several knowledge areas relevant for the
assignment.
Since 2014, Ecorys has been leading support services for the DG HOME led Community of Users
initiative (ongoing), which aims, among other things, to increase practitioner involvement in EU-
funded security research. In 2015, Ecorys led the first European level capability gap analysis and
corresponding market analysis of First Responders (used as a reference document in the
International Forum for First Responder Innovation). Ecorys is also participating in the FP7
DRIVER+ project as lead partner in the development of a shared understanding of crisis
management in Europe and the sustainability component of the project. In addition to these
projects, Ecorys provides regular evaluations and impact assessments in the policy areas of
migration, border security, and counter-terrorism (including CBNR-E).
The VVA Group was established in 1992 by a team of professors from Bocconi University. Over 20
years it has developed into a full-service consultancy with offices in Milan and Brussels. Our in-
house team of about 100 consultants, academics, economists, lawyers and researchers specialises
in providing high quality advisory services to private and public-sector clients. VVA has built a
reputation for delivering high quality analysis, clearly expressed, to tight deadlines. In addition to
our public sector clients (mostly the European Commission), we have worked for more than 600
companies, both in the Industrial and Mass Market industries. In particular, the VVA team has a
proven track record in providing in-depth analysis, strategic advice and business planning support
to entities and companies involved in digital and technology industries.
Most of our projects are based on market analysis and forecasting, competitiveness assessment,
economic and impact analysis, which are VVA’s core competences but we also deliver much wider
services that incorporate consumer, social, environmental and legal aspects. For instance, we have
recently completed a study for DG CONNECT on the Promotion of European content in audio-
visual media which included a market and sector analysis, an economic and a legal overview, a
detailed content analysis based on large programming datasets from linear and non-linear
providers. The study on non-linear providers in particular was based on an innovative set of data
from a non-linear aggregator platform which allowed us to assess catalogues and also audience
engagement with those catalogues in several countries across Europe.
VVA retains strong links with the most prestigious Universities and Research Centres and
continues to invest in research. For instance, we have recently invested in an artificial intelligence
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 9
start-up (Indigo) and a digital marketing start-up (Rankit) This enables VVA to offer the highest level
of up-to-date methodologies and services that use cutting-edge technology. Our Board of Directors
includes both Professors and PhDs from Bocconi University and the London School of Economics
who take personal responsibility for the academic quality of every piece of work that we do.
We work closely with our clients throughout each project to ensure that we provide a cost-effective,
bespoke service at all stages of the project. Our independent, creative and innovative team of
consultants and researchers from different backgrounds allow us to cover a wide range of strategic
needs. Whilst all client contact is led by our in-house team, we are able to bring in very specific
senior expertise through our wide network of partners and universities.
Risk Society is a consulting company based in The Hague, The Netherlands. Established in 2009
with the purpose of building expertise and capacities to manage risks for sustainable development,
the company has been providing a wide range of consultancy services related to various aspects of
resilience building and disaster/climate risk management. Its consultancy services are regularly
requested by the leading international development and humanitarian organizations across the
globe such as UNDP, UNOPS, UNISDR, UN-Habitat, UNON the World Bank, the European
Commission, Norwegian Refugee Council (NRC), Red Cross organizations, Open Society
Foundation (OSF), Netherlands Helsinki Committee (NHC), Cooperazione Internazionale (COOPI),
and many others.
Risk Society has strong expertise in all-hazard risk assessment and risk governance, national risk
assessments, disaster risk reduction and climate risk management, enterprise risk management
(ERM) but also civil protection, post disaster needs assessment (PDNA) and recovery planning,
resilience building including urban resilience, results-based management (RBM), strategic planning,
monitoring and evaluation. It partners up with leading international institutions to develop risk-
related publication (UCL, UK, for instance) or providing lectures and designing institutional
development strategies for the various crisis management educational institutions (in Armenia and
Iraq, for instance). It has been providing technical support to national civil protection and crisis
management authorities in various countries. Examples of the services provided by Risk Society
are the evaluation of ‘Making Cities Sustainable and Resilient (MCSR)’ implemented by UNISDR
and UN-Habitat (funded by DG DEVCO), co-authoring National Disaster Risk Assessment
Guidance commissioned by UNISDR and designing National Disaster Management Strategies for
Moldova (2012) and Turkmenistan (2015).
The Emergency Services College (Crisis Management Centre, Finland) is a national college
providing education for firefighters, sub-officers, fire officers and emergency response center
operators in Kuopio, Finland. The College also offers a wide variety of specially tailored further
training and in-service training for national and international professionals in the rescue and
emergency field. Furthermore, ESC provides preparedness training for disturbances in normal
conditions and emergency conditions as well as training for international civil crisis management
tasks. ESC is involved in the research and development functions in the field. The College is
supervised by the Finnish Ministry of the Interior.
ESC has been actively involved in the EU Civil Protection Mechanism for several years. It has
conducted and co-organised several large scale exercises within EU, UN, NATO and Barents
Region contexts. In addition to exercises, we organize annually around 50 training events related to
civilian crisis management and civil protection. The College manages EU Civil Protection experts,
an INSARAG classified HUSAR Team and a TAST Team, and is in charge of recruiting, equipping
and training them. ESC is also the home office for Finland's national EU Training Coordinator which
10 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
eases the organisation's cooperation with other Participating Statesꞌ training authorities. ESC has
extensive experience of planning, managing and implementing EC funded projects. Finland has
contributed to the development of EU Civil Protection particularly related to EU’s capacity to
operate in cold conditions.
Sagitta Evolution provides professional support to institutional and private operators for study,
research, prospection, management, coordination, implementation, monitoring and evaluation of all
services, at local, regional, national and European level. Sagitta Evolution has been involved, as a
subcontractor, in a number of relevant contracts with the European Commission. For instance, we
designed and implemented the stakeholder consultation strategy in the framework of the Mid-Term
Evaluation of the LIFE programme for the Environment and Climate Action (DG ENV); and for DG
MARE we designed and implemented the stakeholder strategy and knowledge management
process of the Assistance Mechanism for the Atlantic Action Plan to develop project pipeline on
Blue Growth in the Atlantic. We are also playing a leading role in the Secretariat for the Urban
Agenda Air Quality Partnership (DG REGIO) cultivating stakeholders relations of a partnership
gathering 4 Member States, 6 cities, 2 European platforms, and the European Commission (i.e. DG
REGIO, DG ENV, JRC), providing thematic expertise for drafting the Partnership’s Action Plan, for
communication, developing stakeholder consultations, designing and organising events.
The assembled consortium combines a robust understanding of the subject matter with the network
necessary for executing the study outlined in the call for services to – and beyond – specification.
Ecorys and VVA both have access to extensive (pan-European) networks, and are thus well-
equipped to offer services such as deliverable (document) translation and original-language
stakeholder interviews at a high quality level and at a competitive price point. The organisations
which comprise the consortium (Ecorys, VVA) have a proven track record in the activities that they
will take ownership of over the course of the contract implementation phase. As a result, the
consortium can offer outstanding performance when it comes to project management of large,
multi-task EU projects and cost & risk analysis, with intimate knowledge of the UCPM being offered
partially through Ecorys’ project previous project portfolio, and partially through the introduction of
various key experts (which will inform all steps of the data collection process). Combined, this
knowledge base covers not only the theoretical suitability of the UCPM capability goals; it also
incorporates experience that is of relevance when it comes to assessing suitability from an
operational standpoint (against the background of both current-and-future threats).
To reduce the chances of conflict within the consortium and to reduce risk (both on behalf of the
client and of the consortium), parties’ roles have been clearly predefined during the proposal-writing
phase. Ecorys will handle desk research and project management while VVA will handle cost
analysis. The two organisations will coordinate the tag-teaming data collection activities. The inter-
task management structure is designed to ensure all synergies between project task teams are
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 11
seized upon. A wide range of ‘milestone’ deliverables (which feed into the activities of other tasks)
have been conceptualised during the proposal-writing process accordingly.
The European Union Civil Protection Mechanism (UCPM) was established in order to promote
operational cooperation in regard to the significant increase in the number of natural and man-made
disasters. Its creation aims to promote solidarity, as well as to support, complement and facilitate
the coordination of Member States (MS) action in the field of civil protection with a view to
improving the effectiveness of this system.1
For a proper analysis of the UCPM and possible capacity gaps a good understanding of several
aspects of the civil protection in the EU and the UCPM are needed:
Understanding of the objective and functioning of the UCPM;
Understanding of the competences and responsibilities of the EU and MS under UCPM;
Understanding of the available response capacities under the UCPM;
Understanding of appropriateness of the currently available resources;
Understanding of existing gaps in response capacities in the UCPM;
Understanding of existing cost-effective and appropriate new response solutions.
Moreover, Article 214 of the TFEU specifies that the humanitarian aid operation, shall be conducted
according to the principles and objectives of the external action of the Union, which should provide
“ad-hoc assistance and relief and protection for people in third countries who are victims of natural
or man-made disasters, in order to meet the humanitarian needs resulting from these situation”3.
To this aim the European Commission may undertake “useful initiative to promote coordination
between action of the Union and those of the Member State, in order to enhance the efficiency and
complementarity of Union a national humanitarian aid measure”.4
1
Recital 1 of the Decision N°1313/2013/EU.
2
Art. 196(1) of the TFEU.
3
Art. 214 of the TFEU.
4
Art. 214(6) of the TFEU.
12 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
The UCPM was instituted by the Council Decision 2001/792/EC and it aimed to “facilitate reinforced
cooperation in civil protection assistance interventions” in the Community. 5 The initial structure of
the mechanism consisted of a Monitoring and Information Centre and a Common Emergency
Communication and Information System.6 Nevertheless, this first embodiment of the Mechanism
based on ad-hoc approach was limited and not sufficiently effective. Its scope was further
expanded with the adoption of the Lisbon Treaty, which called for improving solidarity between the
EU and its Member States in the field of civil protection coordination with the view to improving the
effectiveness of the system for preventing, preparing and responding to natural and man-made
disasters. The implementation of this new system was supported by the Commission Implementing
Decision 2014/762/EU laying down the rule for the implementation of Decision N°1313/2013/EU.
The main innovation of the reviewed UCPM has been the creation of the voluntary pool of available
capacities under European Emergency Response Capacity, which should be registered by the
Member States and maintained in a stand-by with the aim to be deployed if the need for EU action
occurs.
To achieve the above-mentioned objective the UCPM is composed of the following three structures:
The Emergency Response Coordination Centre (ERCC) replaced the Monitoring and
Information Centre and plays the key operational role for the UCPM. The EERC is the Brussels
based hub, which monitors emergencies across the world. The ERCC is in charge of collecting
and analysing real-time information on disasters, monitoring hazards, making available pre-
committed response capacities, preparing plans for the deployment of experts, teams and
necessary equipment.9 These tasks enable the ERCC to coordinate actions between Member
States, EU Institutions, dispatched field experts and affected countries. Thus, ERCC offers a
more coherent and consistent European response for emerging disasters. Furthermore, each
Member State designates its own national contact point which will liaise with the ERCC.
Common Emergency Communication and Information System (CECIS), which is
supporting the ERCC in its activities. CECIS is a web-based platform in charge of sending and
receiving alerts and notifications of required assistance. The system enables real time
communication between ERCC and Member States on alerts, notifications and available
capacity to be deployed.
European Emergency Response Capacity (EERC) is supporting the ERCC role when it
comes to the deployment of capacities through its ‘voluntary pool’. The EERC represents one of
the largest innovations of the Decision 1313/2013/EU and aspires to improve the effectiveness
of European civil protection response. The creation of the ‘voluntary pool’ aims to group relief
5
Recital 3 of Decision 2001/792/EC.
6
Ibid.
7
Recital 1 of the Implementing Decision 2014/762/EU
8
Recital 3 of the Decision 1313/2013/EU
9
European Commission, Factsheet : Emergency Response Coordination Centre,
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 13
teams, experts, equipment etc., which could be made available for deployment by the
Participating Countries.10 In addition, the participating countries can benefit from the EU
financial support to upgrade their national services, as well as to pay for certification and
training, and cover up to 85% of costs related to the transport of teams and assets to disasters
areas.11 The Commission together with the ERCC defines the key capacities needed to ensure
the required action. Furthermore, in order to guarantee the quality of the available services, a
special certification and registration system was created with the Decision 1313/2013/EU and
Implementing Decision 2014/762/EU for evaluation of response capacities provided by
participating countries.12 The EERC appears to be a successful innovation, as by September
2018, 23 of the participating states have made 100 capacities available for deployment.
These structures contribute to the UCPM’s 3 main axes by supporting EU Member States’ ability to
execute operations in the area of disaster prevention, preparedness, and response.
The deployment of the UCPM can cover two types of risks. First of all, according to Article 13 of the
Decision 1313/2013/EU, the UCPM can be activated “in the event of disaster within the Union, or of
an imminent disaster, which causes or is capable of causing trans-boundary effect or affect another
Member State”. Secondly, according to Article 16 of the 1313/2013/EU, the UCPM can be equally
deployed if “disaster occurs outside of the Union or is imminent”14. In addition, the assistance can
be requested trough the United Nations and its agencies as well as through other important
International organisations.15 Moreover, while dealing with outside cataclysms, the Union’s action
can be conducted autonomously or can contribute to an intervention led by a given international
organisation.16
In each of the cases, the request for assistance from the affected country is needed, without delay
and should be notified to the European Commission via the ERCC. The notification enables the
ERCC and the Commission to contact the relevant Member States and services about the eventual
capacity needs.17 Thus, the deployment of the UCPM can only happen on a request and cannot be
10
Article 11 of the Decision 1313/2013/EU
11
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
12
Article 16 of the Implementing Decision 2014/762/EU
13
http://europeanmemoranda.cabinetoffice.gov.uk/files/2017/03/Explanatory_Memorandum_6448_17.pdf
14
Article 16 of the Decision 1313/2013/EU.
15
Ibid.
16
Article 16(1) of the Decision 1313/2013/EU.
17
Article 14 and 16 of the Decision 1313/2013/EU.
14 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
activated unilaterally neither by the European Commission nor by Member States. Nevertheless,
the UCPM can be activated at the request of an EU Member State for support when it comes to
“providing consular assistance to EU citizens affected by disasters outside the Union”.18 In the past
years, the EUPCM has been more frequently activated for disasters taking place outside the EU’s
territory than for disasters within its borders. Out of 224 requests for assistance registered between
2002-2015, over 63% were submitted by non-EU countries.19 The proportion of requests for
assistance through the UCPM is presented in the figure below.
Figure 1: Requests for assistance through the UCPM (inside and outside the EU)
Recent developments
Some shortcomings have been observed during previous missions. The Communication from the
Commission on Solidarity with Responsibility from November 2017, emphasises that the “incentives
for Member States to offer assistance via UCPM are very low” due to the EU budget financing only
a share of transport costs and not the operational ones.20 Thus, the EU called for a more “robust
18
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
19
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
20
European Commission, Communication from the Commission, Strengthening EU Disasters Management : rescEU
Solidarity with Responsibility, November 2017, available at : https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2017:773:FIN%20.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 15
and comprehensive EU disaster capacity”. In this regard, between 2017 and 2018, important works
were undertaken to further improve the effectiveness of the UCPM. In this regard and in the
framework of the Commission’s political priorities for 2015-2017, the Commission put forward an
ambitious new plan ‘rescEU’ to reinforce the EU civil protection response by two actions:
To emphasise the need of solidarity and reinforce EU and its Member States’ collective ability to
response to disasters (in particular addressing current emerging capacity gaps) by establishing
an EU level special response capacity, the European Civil Protection Pool building on the
already existing EERC (the ‘voluntary pool’) , which could be immediately deployed, command
and controlled by the European Commission.21,22
To strengthen the effectiveness of prevention as a part of disaster risk management and
reinforcing links with other key EU policies acting in the field, making risk prevention a core
element of disaster risk management supported by the investment in prevention and
preparedness monitoring.
Furthermore, it has been also suggested to reinforce and back up the current incentive structure of
the UCPM, by creating one category of response assets, which would be co-financed by the EU’s
budget and the Members States in the framework of the European Civil Protection Poll.23 The co-
financing from the EU budget should cover the expenses related to adaptation, reparation, transport
and running of operations of the pre-committed response capacities of Member States.24 The figure
below visualises the structure of the new Commission proposal, composed of two main elements:
response and prevention/preparedness.
In order to implement the ambitious Commission’s plan, in November 2017, the European
Parliament and the Council put forward a proposal for a Decision amending the current legal
21
Proposal for a DECISION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Decision No
1313/2013/EU on a Union Civil Protection Mechanism, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2017:772:FIN.
22
RescEU will be equipped with selected emergency capacities to respond to wildfires, floods, earthquakes and health
emergencies as appropriate.
23
European Commission, Communication from the Commission, Strengthening EU Disasters Management : rescEU
Solidarity with Responsibility, November 2017, available at : https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2017:773:FIN%20.
24
Ibid.
16 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
framework established by the Decision 1313/2013/EU that would be in line with recent
Commission’s ambitions.
Additionally, the Commission adopted the Implementing Decision EU/2018/142, which amends the
list of defined capacity goals (Annex III to the Implementing Decision 2014/762/EU) and the quality
and interoperability requirements as well as registration and certification procedure for EERC
(Annex II to the Implementing Decision 2014/762/EU).25 The main changes are related to the
shortage of emergency medical team and health related interventions.26
The Annex III of the Implementing Decision 2014/762/EU specified the needed capacity goals and
grouped them into three categories: modules, Technical Assistance and Support Teams (TAST),
and other response capacities.
25
Commission Implementing Decision (EU) 2018/142 of 15 January 2018 amending Implementing Decision 2014/762/EU
laying down rules for the implementation of Decision No 1313/2013/EU of the European Parliament and of the Council on
a Union Civil Protection Mechanism (notified under document C(2018) 71), available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?qid=1530262669595&uri=CELEX:32018D0142.
26
Ibid.
27
Article 14 of the Implementing Decision 2014/762/EU
28
Article 4(6) of the Decision 1313/2013/EU.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 17
Capacity goals modules
Ground forest fire fighting (GFFF)
Ground forest fire fighting using vehicles (GFFF-V)
Advanced medical post with surgery (AMP-S)
Flood containment (FC)
Flood rescue using boats (FRB)
Medical aerial evacuation of disaster victims (MEVAC)
Field hospital (FHOS)
Aerial forest firefighting module using helicopters (FFTH)
Emergency medical team type 1: Outpatient Emergency Care — fixed (EMT type 1 fixed)
Emergency medical team type 1: Outpatient Emergency Care — mobile) (EMPY type 1 mobile)
Emergency medical team type 2: Inpatient Surgical Emergency Care (EMT type 2)
USAR in CBRN conditions (CBRNUSAR)
In addition to the modules and TAST, 22 categories of other policy response capacity were
identified, for example: team of mountain search and rescue experts, mobile laboratories for
environmental emergencies, power generators, team of maritime incident response experts and
many others. It has been specified that modules as well as TAST may be composed of resources
provided by one or more Member States.29 Furthermore, capacity responses provided by the
Member States should comply with the requirement of the Annex II to the Implementing decision of
2014, which defines the characteristics of the capacities and main components needed for their
deployment. In this regard, the response capacity should be understood as “assistance that may be
provided through the Union Mechanism upon request”.30
In 2017, it has been noted that the “target for the number of modules the UCPM was surpassed by
274%, by 650% on TAST and 134% on so-called other capacities”.31
29
Article 11 of the Implementing Decision 2014/762/EU.
30
Art.4(10) of the Decision 1313/2013/EU.
31
European Commission, Interim evaluation of the Union Civil Protection Mechanism, 2014-2016 : Final Report, August
2017, available at : https://publications.europa.eu/en/publication-detail/-/publication/eb41bfee-78c3-11e7-b2f2-
01aa75ed71a1/language-en/format-PDF
32
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
33
Copernicus Emergency Management Service – Mapping, available at:
http://emergency.copernicus.eu/mapping/#zoom=2&lat=23.42974&lon=64.03963&layers=0BT00.
18 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
report of the Commission to the European Parliament delivered in February 2017, which analysed
the progress made and gaps remaining in the EERC).34
First, a tangible proof for the success is the exceeding number of response capacities registered
between 2014 and 2016, in comparison to the targeted number of modules according to the
Implementing Decision 2014/762/EU.35 However, seven modules such as aerial forest firefighting
using planes, and Emergency Temporary Camp did not meet the capacity goals. Furthermore,
weaknesses were noted in the area of chemical, biological, radiological and nuclear disasters,
medical corps, remotely piloted aircrafts systems and communication teams.36 The figure below
compares the EERC capacity goals with resources registered between 2014 and 2016.
Source: ICF study, Interim evaluation of the Union Civil Protection Mechanism 2014-2016.
Capacity shortcomings
Regarding the firefighting planes, the events of forest fires in the summer 2016 in Portugal, showed “the
operational necessity and political significance” of having more firefighting planes available in the
framework of EERC.37 When it comes to the shelters and related assistance, the capacity goals has not
been reached in this field under EERC. Additionally, during the peak of the refugee’s crisis, when
numerous Member States where facing the same challenges, shortcomings of the EU civil protection
mechanism became clearly apparent. Thus, the Commission identified shelters and related assistance as
“being a potential significant gap”.38
Secondly, it has been noted that while some resources are sufficiently available, their deployment
possibilities are weak. This is notably the case for the occurrence of chemical, biological,
radiological and nuclear disasters as well as the deployment of big field hospitals, medical
evacuation capacities, remotely piloted aircrafts system and finally communication teams.39
34
European Commission, Report from the Commission to the European Parliament and the Council on progress made and
gaps remaining in the European Emergency Response Capacity, November 2017, available at :
https://ec.europa.eu/transparency/regdoc/rep/1/2017/EN/COM-2017-78-F1-EN-MAIN-PART-1.PDF
35
European Commission, Interim evaluation of the Union Civil Protection Mechanism, 2014-2016 : Final Report, August 2017,
available at : https://publications.europa.eu/en/publication-detail/-/publication/eb41bfee-78c3-11e7-b2f2-
01aa75ed71a1/language-en/format-PDF
36
Ibid.
37
Ibid.
38
Ibid.
39
Ibid.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 19
Deployment shortcomings
In the light of terrorist attacks in Europe, it seems that the EERC does not have enough response
capacities for rescue operations in contaminated environment and treatment of patients exposed to
Chemical, Biological, Radiological and Nuclear (CBRN) agents.40,41. During the Ebola outbreak in West
Africa, shortcomings regarding the evacuation services for medical staff and humanitarian aid workers and
regarding emergency medical teams and other health related intervention surfaced.42 For instance, the
available medical resources were dedicated to general service for aerial evacuation rather than to infected
medical staff.43
Thirdly, it turns out that the Member States lack incentives to deploy the available capacities. The
EU budget is co-financing only the share of transport costs which, in comparison to the operational
costs, are rather low. For instance, the cost of large field hospital transportation constitutes only a
very small part of the costs required to run such hospitals in field, which is approximatively EUR 6
million per month.44 The lack of co-financing for the operating costs of response capacity
deployment is one of the very important gaps of the UCPM.
Lastly, it seems that administrative burden hinders the immediate activation of the UCPM. The
recent report of the Court of Auditors has emphasised that although the Commission transferred the
request for assistance in a timely manner, the mobilisation of the available resources was not
efficient when it comes to activation of the pre-alert phase, availability of the ‘voluntary pool’ or
nomination of the EU coordination team on-the-ground.45 In this regard, in October 2015 a very
slow development of the pool of response capacities was noted due to the “administrative burden
involved in registration and certifying response units/modules”.46 The figure below visualises the
time that has elapsed since the occurrence of the disasters and request for assistance, to the
request in CECIS for nomination of experts.
40
Ibid.
41
The recent Commission Communication from October 2017 Action Plan to enhance preparedness against chemical,
biological, radiological and nuclear security risks emphasised that although the fact that the terrorist organisation have not
used CBRN agents in Europe yet, there “there are credible indications suggesting that terrorist groups might have the
intention of acquiring” such materials or weapons and “develop knowledge capacity to use them”.
42
Op. cit. European Court of Auditors and Implementing Decision (EU) 2018/142.
43
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
44
European Commission, Communication from the Commission, Strengthening EU Disasters Management : rescEU Solidarity
with Responsibility, November 2017, available at : https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2017:773:FIN%20.
45
Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been
broadly effective, N°33, 2016, available at:
https://www.eca.europa.eu/Lists/ECADocuments/SR16_33/SR_DISASTER_RESPONSE_EN.pdf.
46
Ibid.
20 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Figure: Timeline for selection of EUCP experts in Bosnia and Herzegovina
Some of the above-mentioned shortcomings like the lack of incentives are already being
addressed47, further solutions would need to be based on “the recent economic market-based
information in the field of the development, deployment, operational and maintenance costs of the
capacities used in response to disasters”.48 To this end, innovative solutions could also be based
on the approach undertaken by other international organisations which are dealing with
management of the civil protection mechanism.
As has been noted, Member States suffer from a lack of incentives to deploy their response
capacities. Consequently, a horizontal financing support from the EU budget could improve the
availability of registered capacities considerably. To this aim, a detailed evaluation of costs should
be further developed according to different scenarios for each module and category of capacity.
The deployment of the UCPM should not be seen in isolation from other existing initiatives in the
field of civil protection and its efficiency could be improved through the better coordination with
other complementary interventions, actions, tools, and services provided by the EU, Member
States, non-EU countries and international organisations. Furthermore, important lessons can be
derived from the performance of other international organisations specialised in the field of risk
management and civil protection such as the OECD’s Development Assistance Committee
(DAC)49, United Nation Office for Disaster Risk Reduction (UNISDAR)50 or the WHO’s Global
47
See sub-section Recent Developments und section 1.3.2.
48
European Commission, Report from the Commission to the European Parliament and the Council on progress made and
gaps remaining in the European Emergency Response Capacity, November 2017, available at :
https://ec.europa.eu/transparency/regdoc/rep/1/2017/EN/COM-2017-78-F1-EN-MAIN-PART-1.PDF.
49
http://www.oecd.org/development/developmentassistancecommitteedac.htm
50
https://www.unisdr.org/
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 21
Outbreak Alert and Response Network (GOARN) 51 or United Nations High Commissioner for
Refugees (UNHCR).
In order to provide efficient civil protection to natural and man-made disasters, the European
Commission is in charge of monitoring and evaluating the results of the UCPM implementation. To
this end, this study’s purpose is to deliver an evidence-based analysis into response capacity gaps
while taking into account current risks and needs in the area of civil protection. Additionally, the
study allows to refine capacity definitions and formulate options based on capacity costs by
providing a cost analysis of development, deployment, operational and maintenance costs of the
capacities used in response to disasters. Together, these results will feed into DG ECHO’s
monitoring of the UCPM’s progress in achieving its capacity goals.
As stated in the terms of references, the study’s scope covers the UCPM, is functioning and the
capacities provided by its participating states. It shall:
Review the general requirements of all modules (excluding aerial forest firefighting) and
propose a revised version for outdated definitions and requirements;
Develop general requirements for response capacities in light of changing or emerging risks
(including quality and interoperability requirements);
Conduct a cost analysis, based on the (revised) definition of response capacities, and include a
cost breakdown in terms of equipment and human resources;
Assess the progress that has been made towards achieving the capacity goals. In particular, the
study shall evaluate whether sufficient capacities have been registered in the EERC and
whether capacity goals are set at the appropriate level in light of risks;
Propose, if needed, a revision of the capacity goals.
Chapter Two outlines how we propose to fulfil the purpose and requirements of this study, starting
with our general approach and then highlighting tools and methodologies to be used per task.
51
http://www.who.int/ihr/alert_and_response/outbreak-network/en/
22 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 23
2 Methodology
We propose a holistic approach to this study that enables the best available data and deep insights
from country experts and relevant stakeholders to be extracted. Our approach ensures that
necessary inputs to conduct the analysis under each task are obtained. We see the tasks as being
interlinked with each other and requiring feedbacks and inputs from each other in order to be
accomplished successfully. In the following section, we describe how the tasks are interlinked and
what methods are proposed to complete each task.
The interlinkages between the different tasks are visualised in the following figure.
Source: Ecorys
Task 1 analyses the appropriateness of existing response capacities in light of past experiences
and the most recent risk assessments. This analysis will provide a precise definition of the
response capacities (aka modules) for which a cost analysis will be conducted in Task 2.
24 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
The priority of Task 1 is to identify capacities that can be categorised as category A or B modules
so that they can be subjected to cost analysis of Task 2 immediately. For capacities identified as
category C or D modules, further research needs to be conducted. For category C modules
definitions and quality requirements need to be revised. For category D, general requirements,
including interoperability, need to be developed from scratch. Only then, equipped with new
definitions, categories C and D modules can be analysed under Task 2.
At the same time, cost analysis under Task 2 may have impact on the categorisation of capacities
and development of definitions and requirements under Task 1. Results of the risk assessment in
Task 3 will be considered when examining capacities under Task 1, and review of capability goals
under Task 4 needs to be taken into account by Task 1 as well.
Task 2 involves a cost analysis of capacities that depends on their categorisation under Task 1.
Category A and B capacities do not need to be reviewed, and the cost analysis for these modules
can also be conducted early in the project. For category C and D capacities, the cost analysis can
start only after (revised) definitions and quality requirements are approved by DG ECHO.
As explained above, Task 2 is dependent, and in fact triggered, by Task 1. In addition, the
outcomes of the risk assessment in Task 3 and the review of capability goals in Task 4 may have
implications for cost analysis and need to be taken into account in Task 2. Therefore, Task 3 and
Task 4 feed back into Task 2.
Task 3 is comprised of a risk assessment and a capacity gap analysis. Implementation of this task
can start immediately after the signature of the contract. Based on the risk assessment, existing
response capacities need to be reviewed to asses if they correspond to the EU’s internal and
external risk landscapes. Accordingly, outputs of Task 3 will inform Task 1 and impact Task 2 (cost
analysis) and Task 4 (review of capability goals). Where it is found that identified risks – particularly
new emerging risks – are not adequately addressed by the current available capacities or capacity
goals, the study will formulate recommendations to revise the specification of capacity
requirements. Furthermore, this task will develop definitions of new typologies of capacities when
these are considered a potentially significant capacity gap.
Task 4 is geared towards creating a methodological framework through which (i) the suitability of
existing capability goals can be systematically assessed and (ii) the relative importance of existing
and newly formulated capacity goals can be identified. The activities under Task 4 rely on inputs
from Tasks 1, 2, and 3, and feed into the 2nd and 3rd rounds of interviews which take place at the
end of Task 1. The Task team will conduct extensive desk research and/or analyse Task 3 results
in order to prepare interview material to ensure that collected results provide them with sufficient
input to conceptualise (and apply) the methodological framework. In addition to a methodological
framework, an overview of capacity goal suitability, and a prioritisation of capacity goals, Task 4
may result in the drafting of a request to national contact points and a list of omitted and obsolete
capacity goals.
An overview of the specific activities related to the main tasks is provided below.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 25
Table 2.1 Overview of tasks and subtasks
Task and subtasks Deliverable
Task 0 Project management
Task 0.1 Project management Deliverable 1: Inception Report
Deliverable 7: Final Report
Task 0.2 Quality control
Task 1 Review existing capacities
Task 1.1 Desk research
Task 1.2 Stakeholder consultations via interviews
Task 1.3 Stakeholder consultations via survey (targeted Deliverable 2: survey questionnaire
consultations)
Task 1.4 Categorisation of modules of response capacities
Task 1.5 Recommendations for (revised) definitions and quality Deliverable 3: report of the survey and desk
requirements research
Task 2 Cost analysis
Task 2.1 Understanding data needs
Task 2.2 Data collection
Task 2.3 Defining cost adjustments and scenarios
Task 2.4 Cost calculation Deliverable 4: Cost Analysis
Task 3 Risks and gap analysis
Task 3.1 Risk data collection and preliminary analysis
Task 3.2 Response Capacity Gap Analysis
Task 3.3 Validation workshop
Task 3.4 Develop recommendations Deliverable 5: Risk based capacity gap analysis
Task 4 Capacity goals revision
Task 4.1 Preparation of interview material
Task 4.2 Development of a methodological framework for
assessing suitability of capability goals
Task 4.3 Application of the methodological framework Deliverable 6: Capacity goals revision
We propose to employ a suite of methods to obtain the necessary data and information required for
the four core tasks. Data collection is an important challenge of this project because all project
tasks rely on a large amount of data obtained through different methods and from a wide variety of
sources. In addition, the tasks are interlinked with each other and feed into each other, and they
have to be performed within a limited period. With this in mind, we need to ensure that analysts
have the right data in the right format and in the right time. Therefore, it is crucial that the data and
information from the start are collected and stored with the requirements of all project tasks in mind.
26 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
We see data collection as a horizontal activity that is informed by data needs of all analytical tasks
(Tasks 1-4) and that feeds the necessary inputs back into them. In the sections below we discuss
every method of data collection separately and explain in detail how it will be used in the context of
this study. The subsequent task descriptions (starting with Section 1.5 forth following) provide task-
related details on the data collection methods applicable for each task.
Week Week 1 – 17
Week 25 - 30
Tasks Task 1, 2, 3, 4
The study team and the national correspondents will both be involved in desk research. The goal is
to identify and study relevant documents related to the UCPM. In the case of the country
correspondents, desk research will be geared specifically towards collating the background
information necessary to optimise the interview process. Additionally, national correspondents will
identify and study those documentary sources that are only available in national languages of
UCPM Participating Countries. The types of sources that will be included in the analysis are:
government reports legislation, standards, databases, reports from conferences, -workshops, -
trainings, minutes of meetings of relevant committees and reports on civil protection exercises.
They can be structured in varying degrees according to the scope of the review. An illustration of
the types of data to be collected in relation to the potential documentary sources and databases
that would supply such data is shown in the table below.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 27
Repealing Commission Decisions 2004/277/EC, Euratom
2007/606/EC, Euratom
Commission Staff Working Document (2017). Overview of Natural and Man-made Disaster Risks the
European Union may face (SWD(2017) 176 final of 23.5.2017)
Report from the Commission to the European Parliament and the Council on progress made and gaps
remaining in the European Emergency Response Capacity, COM(2017) 78 final of 17.2.2017
DG ECHO Annual activity reports (2017)
United Nations
USAR (INSARAG)
Directories of USAR (INSARAG)
UNISDR (2017) National Disaster Risk Assessment: Governance System, Methodologies and Use of
Results
National authorities
National risk assessments from member states
OECD
National Risk Assessment: A cross country perspective, OECD, 2018
External evaluations
Peer reviews of UCPM countries (6 completed in 2015-2016)
European Court of Auditors (2016). Union Civil Protection Mechanism: the coordination of responses to
disasters outside the EU has been broadly effective. Special report No 33
Special Eurobarometer 454 (2016-2017). Civil protection
ICF (2017). Interim evaluation of the Union Civil Protection Mechanism, 2014-2016
ICF (2018). Comprehensive evaluation of the European Union humanitarian aid, 2012-2016 (included
UCPM in the evaluation of coherence)
52
https://webgate.ec.europa.eu/hac/
53
https://euaidexplorer.ec.europa.eu/HumanitarianDonorAtlas.do
54 See for the European region: https://vosocc.unocha.org/USAR_Directory/USARTeamsByregion.asp .
28 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Targeted consultations approach
Engaging a wide range of stakeholders into this study is necessary to obtain high-quality evidence,
conduct analysis and revise prevention and disaster response capacities under the UCPM. It will
allow to capture their different specialist views, contribute to greater transparency and
accountability in the present review, and in the overall ongoing EU legislative process. It is also
crucial to ensure successful balance between valuable expert opinion and the undue promotion of
special interests.
For this assignment it is therefore essential to identify and engage all relevant stakeholders in the
consultation process, including not only those involved in the preparation and implementation of
prevention and disaster response actions, but also those affected by them and those directly
interested in them across all relevant sectors at European, national, regional and local level. It is
also of utmost importance to carry out an effective promotion of the stakeholder consultation
activities to achieve a satisfactory response from a representative number of stakeholders from all
sectors and Participating States, and thus provide relevant evidence-based inputs to feed into the
study.
Our approach for the targeted consultations takes into account the specific requirements for this
service request and adheres to the General Principles and Minimum Standards set out in the Better
Regulation Guidelines56, notably:
General principles
We will adopt an inclusive and transparent approach by consulting as widely as possible.
We will make the consultation process - and how it is expected to affect policy making -
transparent to those involved.
We will consult at a time where stakeholder views can still make a difference, respect
proportionality and specific restraints.
We will ensure the consistency of consultation processes and include quality control
mechanisms.
Minimum standards
We will produce a clear, concise and complete content for the engagement and consultation
process.
We will ensure that all relevant parties have an opportunity to express their views.
Adequate awareness-raising publicity (in cooperation with EC) will be also ensured.
We will plan sufficient time for gathering responses and written contributions from the
consultation process
Last, but not least, we will acknowledge the receipt of contributions and analyse the findings of
the consultation activity, drafting a report.
56
https://ec.europa.eu/info/sites/info/files/better-regulation-guidelines.pdf.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 29
Our approach for will be structured around two logical steps, complemented by a rigorous system
for ensuring the highest quality standards of delivery, which is described in Chapter 4. The steps
are the following:
Step 1: fine-tune stakeholder consultation strategy
Step 2: consult and gather evidence
Stakeholder mapping
Stakeholder analysis identifies the people, groups and organizations that are important to consider
when looking at the impacts of reforms and is of particular relevance for a studied policy area. To
the purpose of the present study, a “stakeholder” is defined as a party that has an interest in the
UCPM and can either affect or be affected by it, or that can help define value propositions for the
UCPM.
Regarding the identification (or mapping) of stakeholders for the targeted consultations, a
combination of the Commission’s contact lists, our own databases and desk research (described
above) will allow to find the most appropriate “mix” from the categories below:
Implementing the UCPM DG ECHO, and other EC Services (incl. DG ENV, DG CLIMA, DG NEAR,
DG HOME, DG SANTE, DG ENER, DG JRC, EU-Agencies (ECDC,
EMSA, EASO, EEA))
National authorities from Participating States, incl. National Contact Points
and national partners (national civil protection agencies)57
Regional and local authorities from Participating States
Business professional organizations (e.g. EoE programme Manager
Exercise provider, Transport provider, etc.)
DG ECHO international partners, including UN Agencies and other
disaster management-focused international organisations (IFRC, IOM,
etc.)
Affected by the UCPM Host countries having received UCPM support
Interested in the UCPM EU Member States (authorities other than Civil Protection)58
Non-EU Member States Participating States (authorities other than Civil
Protection)
UN agencies (other than Civil Protection)
World Bank
57
See Annex I.
58
This includes other government counterparts whose actions are overlapping with activities carried out by the UPCM.
Examples include: Ministries of Social Affairs (e.g. protection of people from the impact of disasters), Ministry of National
Security/Interior (e.g. management of civil protection consequences of security threats), Ministry of Public Works (e.g.
protection of assets), and Ministry of Health (e.g. prevention and management of pandemics).
30 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Specialised NGOs and other civil society organisations
Businesses (incl. SMEs) in the domain of health, climate change
adaptation, disaster risk management, environment, research and
innovation, maritime emergencies, security policies
External evaluators
Project leads of prevention and preparedness projects and missions
Academia and research institutes having expertise relevant to the UCPM
(i.e. civil protection and humanitarian aid) , mainly in Participating States
Other EU Institutions
This non-exhaustive list may be extended at project inception and during the desk research to
include other stakeholder categories, as considered appropriate by the client.
The table on the next page visualises also the level of interest in and influence on the UCPM of the
several typologies of stakeholders.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 31
Table 2.3 Level of interest and influence of stakeholders
32 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
This table will allow us to select the right approach to reach out to the diverse stakeholder groups
and tailor the consultation tools accordingly. Not all stakeholders in a particular group will
necessarily share the same concerns or have unified opinions or priorities. It is therefore
noteworthy that the differentiations in the table above are indicative and might not always be clear
cut, as they could depend on several elements, for instance the national context. We will also pay
particular attention to involve groups that risk being excluded through the use of a tailored mix of
consultation tools, seeking at the same time to avoid biased opinions, ensuring balance and
comprehensive coverage to the consultation.
The target stakeholders will be identified through transparent criteria in agreement with the Client.
We will aim at ensuring an overall balanced geographical and thematic representation of the
stakeholders consulted. Likewise, we will aim at avoiding 'regulatory capture’ by ensuring that the
inputs from stakeholders with vested interests are incorporated into the overall findings of the
consultation activities, but do not receive special treatment. Based on the preliminary results of the
desk research, stakeholders selection might be further customised to the local specific contexts and
need for data.
To widen participation, we also propose to ask representative individuals from each category in a
position to act as multipliers to suggest further contacts and thus compile comprehensive lists.
Our analysis of previous evaluations of the UCPM, confirms the inherent challenges in stakeholder
consultations in this particular domain, characterised by a large number and variety of parties that
are also geographically very dispersed, especially when it comes to ensuring a high response rate
to survey questionnaires or participation in interviews. In light of the above, we will ensure that all
the proposed consultation methods be carefully tailored to the nature of the relevant stakeholder
groups to ensure maximum openness and accessibility, while avoiding stakeholder fatigue.
Particular attention will also be given to the design of consultation tools according to the
stakeholder’s respective level of engagement and experience with the UCPM.
We also assume that the Client can support us in our efforts to promote participation by issuing a
recommendation letter illustrating the assignment that our team will be carrying out with the target
audiences as well as by providing stakeholder contacts and/ or alerting some of the stakeholders to
this study.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 33
Tasks Task 1, 2, 3, 4
Objectives
Stakeholder and expert interviews will be fundamental for collecting valuable data and information
for conducting this study. With the interviews, we pursue three complementary objectives.
1. The first objective is to yield relevant quantitative data and qualitative information to
complement desk research and form the basis of our analysis, and to inform all project tasks.
2. The second objective of the interviews is to validate/ confirm the data and information obtained
from other sources (for example, to validate whether information received from desk research is
still up-to-date).
3. The third objective is to better understand and put in context data and insights gathered through
the targeted surveys as survey answers may sometimes require further elaboration or when
survey results seemingly diverge with findings based on other sources.
General approach
To reach these objectives, we will conduct about 130 (the precise number of interviews is subject to
the availability of the stakeholders) organised in three rounds:
The stakeholder list defined with the client during the inception phase will be further developed,
drawing on combination of sources:
Web-searches of relevant stakeholders identified to date;
Suggestions made at the kick-off meeting by the Commission and initial feedback;
Results of the desk research and initial interviews (e.g. interviewed experts can recommend
other knowledgeable experts and stakeholders).
34 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Table 2.4 Potential groups of stakeholders
Potential stakeholders
DG ECHO officers (especially from Directorate A, ERCC, EERC);
Disaster Risk Management Knowledge Centre (DRMKC);
Emergency Response Coordination Centre (ERCC)
EU agencies, e.g. European Defence Agency, European Maritime Safety Agency
Representatives of specialised international organisations, such as EMT (WHO), International Civil
Defense Organisation,
International organizations (UNISDR),
Other stakeholders - International Organization for Migration; Organisation for the Prohibition of
Chemical Weapon, World Bank, OECD, ANO, United Nations
Representatives of specialised NGOs, such as Red Cross, Médecins Sans Frontières etc.
Decision makers (General Directors)
Evaluators of related programmes such as Prevention, Preparedness and Response to natural and man-
made Disasters (PPRD) in the Southern Neighbourhood (PPRD South II);
High level representatives (Civil Protection Committee members)
Expert group on civil protection modules (Group members)
Member States' contact points
National civil protection authorities from Participating States;
Relevant industries
Universities and think-thanks
Interview process
Develop the interview protocols and interview questions
As different interviewers will conduct interviews it is essential to put in place a mechanism that
allows for a standardised approach towards the interviews. The project team, therefore, proposes to
utilise interview guidelines that outline the questions that should be asked during the interviews.
We will develop Interview Guidelines specifically tailored to the requirements of this assignment. As
a method, we will mainly rely on Standardized Open-Ended Interview. In this format, the
interviewers adhere to a strict script, and there is no flexibility in the wording or order of questions. It
is still considered a qualitative interview rather than a quantitative interview, because the responses
are open-ended. This is the most structured and efficient of the qualitative interviewing techniques
and is useful for reducing bias when several interviewers are involved, and when it is important to
be able to compare the responses of different respondents. We will however ensure that the
interviewer has flexibility to add some questions tapping into the issues that are most relevant to the
interviewees, in order not to lose this level of details.
These guidelines will differ depending on the moment at which the interview is conducted (Lot 1,
Lot 2 or Lot 3). In Annex II, three examples of interview guidelines are presented. At the moment,
these guidelines are initial indications of what the final guideline would look like. The
recommendations of the Client will be taken into account when developing the final guidelines.
Draft questionnaires will be presented to the Commission for comments. Following discussions with
the Commission, the study team will revise the questionnaires. This exchange is considered very
valuable to finalise the questionnaires and ensure the inclusion of all relevant points.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 35
To ensure that national correspondents conducting interviews have a full understanding of the
questions and objectives, a Guidance Note will be produced and shared. The guidance note will
include the following sections:
Introduction and objectives of the study;
List of stakeholders;
Interview questions;
Standard steps for performing the interview;
Guidance for writing interview notes and reporting.
The Guidance Note, the Interview Guidelines and overall objectives will be discussed with the
country correspondents to make sure that the methodology and approach are aligned across
interviewers.
All interviews will be conducted by our experts and country correspondents, all of which preside
over relevant experience and/or language knowledge for the country(ies) they will cover (see
Chapter 3 for a complete overview of the proposed country correspondents). The country
correspondents will follow a standardized procedure to ensure the robustness and comparability of
the study results.
The procedure will differ slightly by interview round, as the 2nd and 3rd rounds will be geared
towards collecting different data and information than the 1st. Also, for 2nd and 3rd rounds of
interviews, interview questionnaires will be developed outlining main topics/ questions for the
discussion with the stakeholder; the questionnaires will be presented to the Client for approval prior
to their introduction to country correspondents. We provided exemplary outline of issues for
discussion during interviews in each round in Annex II. In order to collect an abundance of
information from stakeholders, in the 2nd and 3rd round we will employ semi-structured interviews
that allow us or the interviewee to introduce new issues and ideas relevant for this study. This
interview technique is especially necessary when discussing revised definitions and requirements
with interviewees.
Conduct Interviews
The interviews will be scheduled with the relevant stakeholders. Our team will make sure to contact
the relevant stakeholders well in advance in order to ensure the availability of the stakeholders and
to allow enough time for the organisation of the logistics. During the first contact, we will share with
the interviewees the objectives of the assignment and the nature of the interview, and we suggest
supporting the interview request with a recommendation letter from the Commission, which may
motivate the interview candidate to actual participation. We will also inform interview candidates
that notes will be taken during the interview and how the data and information obtained from them
will be used for the study. We will obtain a prior permission from the interviewee if a specific piece
of information has to be mentioned in the report and published. Once the interview questionnaire is
approved by the Commission, the questionnaire will be shared with the interviewees prior to the
interview. We will do so at least one week before the interview takes place to ensure the
interviewees have time to prepare and gather relevant data required to ensure that all relevant
information is collected.
We recognise that some stakeholders will not respond to us, some will be unable or will prefer not
to hold a face-to-face interview. Where stakeholders do not respond to repeated contact through
emails (and telephone), we must recognise their right not to participate in the study. These
36 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
stakeholders will be invited to partake in the online survey to give them another chance to have
their voice heard in the context of this study.
The interviews will be conducted both face-to-face, via skype and by phone. As noted earlier,
especially during the first round of interviews we will aim to conduct as many as possible face-to-
face interviews. Indicative duration of one interview is 1-2 hours.
We aim to conduct as many as possible interviews in the native languages of interview candidates
as this will allow the interviewee to express him/herself freely and more precise. Interviews in native
languages may also encourage participation of experts from different countries. The consortium
covers at least one of the official languages spoken in each UCPM Participating Country, as is
demonstrated in the project management chapter.
Interviewees will be informed that in our reporting we will not link a single statement with a specific
organisation. All information will be treated according to our high standards of confidentiality. In our
reporting, we will use views at aggregated levels i.e. country / sector / stakeholder group. Finally,
the information gathered during the interviews will be logged into an interview matrix. This is an
internal supporting tool that ensures that the collected evidence is complete, and provides our team
with a convenient overview of the answers by type of question and stakeholder group.
The analysis will take into consideration the type of respondents and/or the country of origin in
order to make a judgment on whether any patterns or diverging views can be observed. The
techniques used are common to all consultation activities; however, each consultation activity will
be analysed and presented separately in order to ensure transparency and facilitate triangulation of
data. The results will be aggregated and documented in a structured manner.
Survey
Week Weeks 9 - 14
Tasks Task 1, 2
In addition to the interviews described above, we will hold a web-based targeted survey as required
by the tender specifications.
The survey will be launched when the first round of interviews has been completed and, thus, when
the first bulk of data has been collected. The survey will be used to validate and cross-check the
information obtained from the interviews and will specifically target Task 1 and 2. The aim of the
survey is to obtain and verify data regarding the categorisation of the modules, to close the existing
gaps in desk research(that will also been addressed during the interviews), to learn about trends
and challenges in the relevant fields, and to reflect on the costs of the various modules.
Therefore, the design of the survey will take place in parallel to the first round of interviews being
conducted. The interview guidelines developed for the first lot of interviews, the findings from desk
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 37
research, and the information gathered during the interviews, will constitute the basis for tailoring
the survey’s questions. The survey will be designed to avoid bias and to ensure a user-friendly
experience. It will be kept at a reasonable length to avoid fatigue in the respondents and thus
maximise the number of responses.
The survey will contain closed and open questions as well as questions aimed at gathering
qualitative data. Questions will be stated in a clear way in order to ensure that stakeholders
understand correctly and provide information covering adequately the point of each question. This
approach will minimise the risk of misunderstanding, guaranteeing the quality of the data gathering
process.
Our approach to the development of the web-based targeted survey on the UCPM consists of the
following steps.
1. We will first develop key profiling questions mainly related to identify the respondents and to
assess their level of expertise and interest in the several aspects covered by the survey. The
profiling section at the beginning of the questionnaire will also intentionally be kept to a
minimum to reduce the likelihood that the respondents come under stereotype threat59, which
would influence the way they respond to the rest of the survey. On the basis of that profiling, we
will develop customised questions to dig into specific issues, in cooperation with the client. The
structure of the survey will be designed to ensure that the different categories of target
stakeholder can only view those questions which are relevant for their level of interest and
expertise in the UCPM, and not the rest.
2. Secondly, the survey will be presented to the client for comments and feedback. Only after the
client’s approval of the questionnaire (as Deliverable 2) will the survey be programmed,
published and disseminated
3. Thirdly, we will program and test the survey. The testing is likely to suggest some changes in
the routing and (form of) the questions/ answers. These changes will be also submitted to the
client and implemented once an approval from the client is received. Likewise, this testing will
give us a precise indication of the time required to complete the exercise and suggests ways to
minimize it, which will be also discussed with the client.
4. Fourthly, the programmed and approved survey will be disseminated to the target stakeholders.
To this end, a dedicated email box will be used for sending out the email invitation and providing
feedback on request. The surveys will remain open for the period of the consultation, as agreed
with the Commission.
In particular, the draft survey will be presented to the Client in week 5 to allow for preliminary
comments and suggestions of the Commission to be already integrated in the version of the survey
that will be subject to the testing in week 7. The final overview of the survey questions (Deliverable
2) will be submitted in week 8. The survey is envisaged to be launched in week 9 and, therefore, it
is essential the Commission responds in a timely fashion to the proposed survey outline.
The survey will be carried out in English. An initial overview of suggested topics that can be asked
to a specific group of stakeholders can be found below.
59
‘Stereotype threat’ refers to situations in which people feel themselves at risk to conform to stereotypes of a social group.
38 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Example topics to be discussed
EU Civil Protection needs
UCPM Voluntary pool capacities
rescEU capacities
CECIS capacities
Modules
Interoperability
Technical requirements
Advantages of modules
Disadvantages of/critique on modules
Suggestions for improvement of modules
Development costs of modules
Transport costs of modules
Operational costs of modules
Maintenance costs of modules
Nr. Of assets used (and associated costs)
Validation of initial findings from first round of interviews
Best practices
Future outlook
We propose to support the survey invitation by a brief background note and by a recommendation
letter written by the Commission that will introduce the stakeholder to the aim of the study, justifying
at the same time the need for the contractor to gather the required information.
The survey will remain online for six weeks (week 9 up until week 14), and the project team will
regularly monitor statistics and metrics of the survey. The previous experience of Ecorys with such
surveys and with these groups of stakeholders shows that six weeks is a sufficient time frame to
collect a substantial amount of data. Two weeks after the launch, a reminder will be sent out to
those that have not yet filled out the survey and also using social media. Another reminder will be
issued one week from the given deadline to respond.
Our team is available to share statistics on the evolution of the survey with the Client and, if
requested by the Client to facilitate the increase in the response ratio, our team could also
accommodate an extension of the original deadline (while carefully considering the consequences
such action might have for other activities in the project).
The survey will be promoted by a tailored mailing campaign to the list of relevant stakeholders
compiled by our team and duly integrated with any inputs received from the Commission. The
survey will also be actively promoted through the Consortium’s network.
We suggest using the Checkmarket survey tool.60 Ecorys has an arrangement with Checkmarket to
use this platform (including support arrangements to develop non-standard question formats). Key
features of the platform include:
Opportunity to customise the survey,
Control dashboard to monitor response and take action when required,
Smart and standard reporting functions for quick result insights,
Export of raw responses in common formats including Excel, SPSS and pdf.
60
https://www.checkmarket.com/
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 39
An example of a recent survey that Ecorys ran through Checkmarket is depicted in Annex III.
The response rate to the different surveys will be continuously monitored by our team and
measures to increase the response rates will be taken should this become necessary. We will
constantly monitor the tool, perform trouble-shooting to timely fix any bugs, and answer to queries
from the participants and/or from the Commission. After the end date of the survey, once the data
has been cleared of errors and possible biases, we will have an outcome that is valid for the whole
study. This preliminary and descriptive analysis will provide us with a good overview of the survey
outcomes.
The information collected from this survey will be used to contextualise the information gathered in
the desk-based research and first round of interviews, to identify topics or areas that require further
attention, and to identify potential subjects for the second and third round of interviews.
Workshop
Week Week 6
Tasks Task 3
As part of Task 3, a validation workshop will be held to reflect on the preliminary findings of the risk
assessment and capacity gap analysis. The format of a workshop is specifically suitable to validate
these findings as it allows for an open discussion and much interaction between the various
participants as well as with the task team. Given the goal of the workshop, the session will be
structured in such a way that exchange of ideas, visions and experiences is encouraged. At the
same time, task leaders of Tasks 1, 2 and 4 will be invited to the workshop as the information from
and insights of the invited stakeholders may prove valuable to the analysis under those tasks as
well.
To this end, the workshop set-up will be of an open nature in which the participants are informed
about the findings of the analysis and, subsequently, will be invited to discuss these findings in a
structured manner. Therefore, the workshop will commence with a detailed presentation of the
findings and the participants will be provided with the opportunity to ask the task team for
clarifications. In preparation to this, the participants will receive a summary of the project, its
objectives and the preliminary findings of Task 3 prior to the workshop. This allows them to prepare
themselves and enhances the efficiency of the workshop.
Depending on the findings of Task 3, the structure of the workshop will be further defined. Whereas
the main objective of the workshop is to gather information from the free discussions among the
participants, the project team will ensure that these discussions take place in a structured manner.
Structuring the workshop will be done by dividing the workshop into dedicated sessions, each
concentrated on a different aspect of the preliminary findings, during which the participants will be
asked to engage. In order to facilitate the exchange of ideas, the task team will prepare sets of
probing questions.
The workshop will take place in week 6 and is envisaged to take place in Brussels at the Ecorys
premises. The workshop will take one full day. Different types of stakeholders will be invited to join
the workshop. A total of 25 participants will be invited to join the workshop. The key goal is to have
40 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
the attendees of the workshop reflect different types of stakeholders and authorities. Therefore, at
this point the team envisages to approach representatives from UCPM and participating states,
international organizations, NGOs, universities and think-thanks. During the initial stage of Task 3,
the potential participants will be identified and approached.
The findings of the workshop will be summarised and shared with the participants of the workshop.
Consultation timetable
The right timing of the stakeholder consultation activities illustrated above is important to ensure
that stakeholders can express their views during all major steps of the study in a way that allows
our team to capture their views on the several aspects examined under this study. The timetable
below is designed to manage such a stream of information with the target stakeholders.
The stakeholder consultation taskforce members are included in the data collection team (see
Chapter 3) as they will actively engage with stakeholder both to participate in the survey and in
interviews and therefore support the data collection team (national correspondents).
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 41
lacking. This will help the national experts to attempt to collect this information in an effective
manner. An initial overview of relevant national experts to be approached can be found in Annex I.
The risk regarding the limited availability of data is one of the major challenges that the project team
expects to encounter. Other risks and their mitigation measures are presented below.
Table 2.7 Risks and mitigation measures for data collection methods
Challenge Impact Probability Mitigation measure
Delay in delivery due to tight H M Discuss with ECHO clear timeline of
timeline to design, launch and action
analyse results (especially for the Have a broad team in place to implement
survey) and analyse results
Make a large use of web-based surveys
with multiple choice answers (mostly
closed)
Diligent planning of the project timeline
and identification of critical path and
milestones in the project implementation.
Inclusion of safety margins around critical
milestones to reduce the impact of
encountered delays by having regularly
team meetings.
Difficulty in identifying the H M Mobilise national experts who have direct
stakeholders access and can engage the stakeholders, or
reach out to international and local platforms
to facilitate access
Low/participation response rate H H Pre-alert participants before
survey/interviews are held, explain
context, confidentiality, potential issues
with sensitivity
Send reminders and use networks above
to facilitate
Design of questionnaires with a
reasonable number
of questions (mostly closed type of
questions)
Test of questionnaires prior to publishing
Support from the client in identifying the
most proper
stakeholders and provision of email
contacts
Use of interviews and to provide further
answers
Follow up e-mails and phone calls
Need to use local language with M M Deploy experts able to cover all languages, or
surveys/interviews/workshops most of them. Survey can be translated easily
with no need of a local expert.
Little availability of data H M As such, for a proper evidence-based
evaluation, we will rely on input from coming
42 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Challenge Impact Probability Mitigation measure
from different sources. We expect that much
of this information requirement can be
covered with input from targeted survey and
interviews.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 43
Challenge Impact Probability Mitigation measure
Difficulty to retrieve data by M M In case the national correspondents
national correspondents experience severe difficulties obtaining
information, the project team has set aside
budget to involve national experts who are
better connected and will most likely be able
to retrieve this information more effectively.
The list of national contact points in Annex I
will be consulted and national experts will be
approached based on which national
correspondents experience most difficulties.
2.4.1 Objective
We introduce a horizontal Task 0 in order to ensure that the consortium is working efficiently and
effectively towards achieving the goals of this project. This includes the organisation of in person
and virtual meetings, continuous reflection on the state of play and quality of the various tasks as
well as coordination of different teams responsible for different tasks and especially the data
collection, which shall run across all tasks and feed into them. Furthermore, part of Task 0 is to
ensure that outcomes of (sub)tasks and activities are aligned and effectively linked to activities
undertaken in other tasks and, thereby, to exploit synergies within the project as much as possible.
The core team established in the light of this objective will help to oversee and implement cross-
task activities and outputs.
Given the vast amount of interlinkages between the tasks, Task 0 will play a pivotal role in linking
the different outcomes of the activities and ensuring they feed into different tasks at the right
moment in time.
While this section explains our approach to project management and quality control, Chapter 3
contains the details about the project team and their roles and Chapter 4 provides a detailed
explanation of specific quality control measures that will be applied to this project.
Objective To oversee (and ensure) the functioning of the service contract as a whole
Team Ecorys
Timing Week 0 - 38
44 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
2.4.2 Proposed methodology
The following sections will outline which methodology is foreseen regarding project management
and quality control under Task 0.
In-person meetings
One of the key tasks of the project management team is to arrange meetings with the Client. As
stated in the Terms of Reference, at least nine meetings with DG ECHO should be scheduled and
held. Four of these meetings are foreseen to take place in person and the other 5 will take place via
phone or via the virtual conference hosting tool GoToMeeting (further elaborated in the following
section). The following meetings are foreseen to be conducted with the Client in person:
Besides the planned meetings, the project management team will be able to update the Client at
any time regarding developments in the project during ad hoc meetings if deemed necessary.
Virtual meetings
In addition to in person meetings with the Client, the project team will schedule five virtual meetings
with the Client. These meetings can be held using the GoToMeeting tool (see below). This tool
allows the project team to speak to the Client while, simultaneously, exchanging files. When there is
no need to exchange documents, the project team proposes to schedule a teleconference with the
relevant parties. Both GoToMeeting as well as teleconferences will be used as part of the internal
communications of the project team. The following overview shows which virtual meetings are
foreseen with the Client.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 45
Meeting When Attended by
Meeting to discuss methodology Week 29 Task Team 4
framework
Meeting to discuss D6 Week 34 Task Team 4
In addition to the meetings indicated above, the Client can request an ad hoc virtual meeting when
this is deemed necessary.
Communication tools
Earlier, GoToMeeting and teleconferences have been mentioned as effective tools that will be used
in both internal and external communication. The below outline provides an overview of the different
communication tools that will be used and their respective strength:
E-mail;
Teleconference;
Group calling platform GoToMeeting;
Asana/Slack.
External and internal coordination is facilitated through email exchange. A centralised project email
address – capable of handling all project-related communication – will be created on the Ecorys
server to ensure a single point of contact.
Teleconferencing can be used when a (large) group of people needs to convene without having to
make use of any visual support. When visual support is required, the project team proposes to use
GoToMeeting as this program allows the team to share documents in real time.
Asana and Slack will help the project team to manage the project in an effective and streamlined
manner. Given the tight deadlines and the relatively large size of the project, it is essential for the
project director and project coordinator to be aware of the activities and progress made by the
different tasks. These platforms allow for tracking progress and creating metrics on the progress.
Core team
The core team established as part of this task is specifically aimed to ensure and safeguard the
different linkages between the various (sub)tasks and outcomes. The project at hand is relatively
complex in the sense that the tasks are strongly interconnected. Therefore, the task leaders of the
different tasks will be seated in the core team where they share the progress and updates from their
respective tasks to ensure an optimal alignment and effective exchange of information and efforts.
The core team will base its efforts on the linkages envisaged between the different tasks as laid
46 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
down in the proposal at hand. This document will serve as a guiding document for the core team.
For each task it is clearly outlined how it is linked to the other tasks, and thereby, the task leader
has a clear overview of the position of its task vis-à-vis the others.
The core team is envisaged to meet monthly during a virtual meeting using GoToMeeting or regular
teleconferences. The project coordinator will host the meetings and is responsible for the logistical
arrangements such as setting up and scheduling the meeting as well as distributing an agenda and
taking minutes. During the meetings, each task leader will provide an update on the state of play of
their respective task. Special attention will be paid to each task’s linkages with the other tasks and
the progress made regarding those linkages. The regular meetings allow the task leaders to identify
potential bottlenecks and/or risks at an early stage and, thereby, mitigation actions can be applied
as early as possible. These meetings will be attended solely by all task leaders and the project
coordinator. However, when the Client wishes, she can attend these meetings, too.
The core team will attend the envisaged meetings with the Client. During these gatherings, the task
leaders will be able to provide an update on the state of play of their respective task as well as to
ask the Client specific questions relevant to the activities in which they are involved.
Project roadmap
Several measures have been foreseen to mitigate the risks regarding a lack of synergy between
task teams and the consortium’s decentralised structure. In addition to the activities undertaken by
the Project Core team and the use of platforms such as Slack61 and Asana62, the Ecorys
consortium proposes to implement – alongside the quality control procedure already in place for
ensuring the ‘tangible’ quality of project deliverables – a quality control procedure which is geared
specifically towards fostering, identifying, communicating, and operationalising synergies between
task teams.
The activities comprising the aforementioned quality control procedure are centred around the
project roadmap document, which will structure the proactive engagement of individuals within the
project. Each of the project’s task leaders will take ownership of developing an outline of how their
respective task is conducted and how it is related to the other (based on the foundations provided in
the proposal at hand).
In order to ensure synergies between project tasks, task leaders will draw upon these agreed upon
linkages between the tasks. The overviews of these synergies between tasks will be aggregated
into a task overview, which will be disseminated among the task leaders within one week of the
first meeting of the project’s core team. The task leader of Task 0 will subsequently compile the
61
https://slack.com/; project management tool that helps to keep track of the work done and provides useful statistics and
metrics to monitor the team’s performance
62
https://asana.com/; project management tool that allows one to keep track of the tasks assigned to different team
members and share files.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 47
task overviews into a project roadmap, which will be shared with the core team prior to the second
meeting. The project roadmap will detail the following information:
Guidance tool
The Ecorys consortium is committed to ensuring the quality of project deliverables. As a result of
Ecorys’ involvement in various other Commission projects relating to crisis management (for DG
HOME as well as DG ECHO), it is familiar with a.) Commission preferences vis-à-vis reporting
style, and b.) the delicate process of navigating Member State preferences vis-à-vis the disclosure
of politically sensitive information. Based on this (and other) experience, the team members of Task
0 will guide the core team through the process of developing a guidance tool for reviewing the
integrity of the following aspects of submitted deliverables:
Facts and key developments. Check whether to your knowledge the names, dates and figures
used in the report are correct. Have the most recent and/or relevant developments been
featured? Should the presentation or the balance between them be altered? Should any other
developments or trends be included?
Consistency. Is the message coming through in the work consistent across sections? Are there
facts or analysis buried in the details of the report that contradict the information presented
elsewhere?
Evidence. Are statements and analysis supported by sufficient evidence? Are quoted sources
reputable and reliable, or might there be a significant bias?
Contentiousness/controversy. Are the statements and interpretations widely accepted or are
they controversial? If assertions are controversial, are they appropriately backed by facts and
sources?
Styling. Does the document styling align with the provided style guide?
The guidance tool will be submitted to representatives of DG ECHO as an annex to the inception
report. The guidance tool will serve to structure the activities of the quality assurance team, which
will be mobilised in accordance with the quality control procedures outlined in Section 4.
2.4.3 Deliverables
The project management team is responsible for the submission of two deliverables, namely the
Inception Report and the Final Report. For each deliverable, the draft outline of the report is
presented below.
48 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Deliverable 1: Inception Report
Based on the discussions held and input received during the kick-off meeting, the project team will
develop the Inception Report. This report will reflect the detailed work programme as well as the
project planning as agreed during the kick-off meeting. Furthermore, the Inception Report will
identify additional needs, potential risks and mitigation strategies. Moreover, the Inception Report
will present the final evaluation methodology.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 49
Risk I P Mitigation
between the Task leaders, the risk
of miscommunication and
inefficiencies is mitigated.
Tasks are delayed and, H M The tight planning of the project at
thereby, affect the execution hand requires all involved parties
of other tasks to take their responsibilities in
time. In order to clarify who needs
to deliver what and at which point
in time, the proposal outlines
clearly which deliverables need
input from which tasks as well as
when the Client needs to
deliver/approve. This clear task
division helps to structure the
project and mitigates the risk of
delays.
2.5.1 Objective
The objective of this task is to collect and analyse evidence on the appropriateness of currently
available response capacities (as per Annex II of the Decision No 1313/2013/EU), in terms of:
Definitions,
Quality requirements,
Past response experiences, and
Most recent national and European risk assessments.
Objective Collect and analyse available evidence regarding the appropriateness of the
currently available response capacities in terms of definitions, quality requirements,
past experience and recent risk assessments
Team Ecorys
50 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Task 1, and especially the stakeholder survey conducted under the aegis of this task, serves as the
basis for all other tasks. The stakeholder survey will collect data and information that can be used
by analysts in Tasks 2-4. The analytical part of Task 1 – categorisation of the different modules of
response capacities based on their need for revision or alteration – serves as the foundation of the
cost analysis in Task 2 and feeds into the revision of capacity goals in Task 4.
At the same time, the feedback from Tasks 3 and Task 4 has to be taken into account in Task 1.
Risk assessment under Task 3 is crucial to understand the appropriateness current response
capacities, and revision of capacity goals under Task 4 may also have impact on the
appropriateness of response capacities and become a basis for further analysis under Task 1.
The table in Section 2.10 shows a tentative time planning for these activities. Preparation and
conducting of data collection activities will start a bit earlier than analysis, but then will run in parallel
with the analysis most of the time, in order to supply the necessary data for the analysis and be
responsive to changing data needs (e.g. specifically in the case of interviews and desk research,
adjustments can be made very quick). The analysis of capacities’ categories (Task 1.4) has to start
early because it needs to provide input for the first iteration of Task 2. With the income of data and
information from data collection and of finding from other tasks (Tasks 3 and 4), the development of
definitions and quality recommendations can also start relatively early, particularly as the outcomes
of Activity 1.5 inform Task 2.
The sections below provides a more detailed description of each planned activity.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 51
For other data collection activities, desk research will identify stakeholders to be contacted,
inform us what data is available in databases and documentary sources and what are the
remaining gaps;
For analytical activities, desk research will collect a variety of data and information to become a
subject of the subsequent analysis.
The initial review of publicly available data and information sources also helps to formulate data
needs for Task 1. Preliminarily, the following data needs have been identified for Task 1:
Definitions of existing capacities
Quality requirements of existing capacities
Performance indicators (assessments) of the existing response capacities in the past response
operations/ missions
Performance indicators (assessments) of the existing response capacities in prevention and
preparedness projects
Gaps identified on the basis of the past experience
New risks according to national and European risk assessments
Evolution of existing risks according to national and European risk assessments
Response capacity requirements to face new risks and/or evolved existing risks
International standards for response capacities
Recommendations by national and international experts for response capacities
To compile this and other information, the study team will review documentary sources ranging
from applicable legal documents, to national and international studies and reports (e.g. on risk
assessment, on readiness and preparedness of civil protection/ first responders), to
recommendations by international organisations and specialised NGOs, to reviews and reports on
the missions/ projects/ trainings completed and to other documents and literature. The examples of
documents we are going to look into are outlined in Section 2.3 above.
Based on the desk research by the team, the Module fiches that will be used to calculate and
present the cost analysis of the different modules (Section 2.6.2) will be created. The task team will
draft a fiche in close collaboration with the Task 2 team. This draft will then be shared with the
UCPM experts in the project team via email. Once the feedback from the experts has been
collected, the fiches will be amended to reflect the suggestions. During a virtual meeting with Task
team 1, Task team 2 and the experts, the final template of the fiche will be agreed upon.
In addition, the interviews conducted at the beginning of the project (first round of interviews) can
help to identify new sources of data and information (e.g. give indication of new or previously
unknown studies, reports or databases, identify a new stakeholder or assist with establishing
contact with a stakeholder).
52 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Other relevant DGs and EU-level agencies (e.g. European Maritime Safety Agency)
Disaster Risk Management Knowledge Centre (DRMKC);
National civil protection authorities from Participating States;
Evaluators of related programmes such as Prevention, Preparedness and Response to natural
and man-made Disasters (PPRD) in the Southern Neighbourhood (PPRD South II);
Representatives of specialised international organisations, such as UNISDR, EMT (WHO),
International Civil Defense Organisation,
Representatives of specialised NGOs, such as Red Cross, Médecins Sans Frontières etc.
The interviews of the first round will be used primarily to identify modules of category A (i.e. that
comply with international standards and do not require revision) and category B (i.e. modules that
are appropriate and up-to-date and do not require revision). For this information, we will target in
the first line potential interviewers at the EU level (DG ECHO, ERCC etc), international NGOs and
international organisations. We will validate their information not only with the desk research, but
also with representatives of national civil protection authorities in order to check whether the
information is still up to date.
During the second round of interviews, we will target in the first line national civil protection
authorities, programme evaluators, national experts with operation experience etc. These
stakeholders can provide us with information about the modules that are not up-to-date and not
appropriate any more to respond to the new and/or evolved risks; they can explicate the (national)
needs and actual supplies of civil protection in the country and the complementarity and
interoperability of capacities between different EU countries. We will enquire from these
stakeholders what changes, in their opinions, need to be undertaken to render response capacities
up-to-date, how definitions and quality requirements have to be changed/ adjusted, what general
requirements for experts are necessary and how their interoperability has been ensured in practice,
how the interoperability can be enhanced, whether the suggested changes make sense at the
national and European level.
During the third round of interviews, we will be further collecting from national level stakeholders
the missing data that is necessary to complete the analysis of modules under Task 1. At the same
time, we will validate the developed definitions, quality requirements and general requirements. For
the validation task, we will address respective questions not only to national level stakeholders, but
also to EU-level and international stakeholders. The latter stakeholder categories can provide
feedback on whether the suggested revisions are compliant with international standards and can be
used across all UCPM Participating Countries. We will consult thematic experts in civil protection in
order to check whether the suggested definitions and quality requirements are forward-looking
enough.
While the questionnaire with questions related to Task 1 will be developed during the inception
phase and during the initial weeks of desk research, here we provide an indicative list of topics/
issues to be covered by the interviews:
Issues for the first and second round of interviews:
- National Civil Protection risks
- National Civil Protection needs
- Match between risks and needs on the national level
- Match between needs on national level and UCPM capacities in case of emergency in
participating states;
- Capacities of UCPM Voluntary Pool
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 53
- Capacities of CECIS
- Capacities of European Civil Protection Pool
- Capacities foreseen in rescEU
- National Civil Protection capacities of the participating states that could be used in UCPM
operations (not included in the current UCPM Voluntary Pool or CECIS either in the planned
European Civil Protection Pool or rescEU);
- Risks regarding civil protection outside the UCPM participating states
- What additional Civil Protection assets or support is be needed regarding disasters outside
UCPM participating states
Additional issues for the second round of interviews:
- Adjustments in modules/ capacities necessary in the view of new/ changed risks
- Adjustments in response capacities necessary based on past experiences (exercises,
operations, training)
- Interoperability issues for experts
- Other requirements to experts
Additional issues for the third round of interviews:
- What implications a specific definition X (quality requirement Y) has for your national civil
protection response capacities (application of the definition, possibility to implement, cost
factors)?
- Does a specific requirement X improve/ ensure interoperability of the expert?
As the targeted consultations with stakeholders is a large overarching activity that informs all tasks
of this study and also needs inputs from all other tasks, its detailed description was included in
Section 2.3.1, even though this activity will be coordinated under Task 1. In this section, we only
cover those survey issues that are specific to Task 1 topic, that is review and (re) definition of the
existing response capacities, which mainly refers to topics to be covered by the survey.
Suggestions for a survey questionnaire will be developed during the inception phase, presented at
the kick-off meeting and then elaborated and finalised subsequently, in accordance with the
instructions by the Client. In relation to Task 1, the topics/ issues presented above as topics to be
addressed during the interviews will also be considered to be included in the survey. Based on the
initial round of interviews and the discussions with the Client, this list will be further narrowed down
and specified.
54 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
All in all, 15 modules63 plus technical assistance teams (TAST) and experts will be examined under
activity 1.4.
In the first step, on the basis of the data and information collected about the existing response
capacities, we will determine which of them comply with the established international standards
(specifically, USAR and EMT). For these, we will contrast the existing UCPM capacities, registered
by Participating Countries, with the data from international directories of EMT and USAR for
certified modules. We will also compare the legal requirements to those modules with international
standards. Such modules are category A modules and require no further examination under Task 1.
Category A modules will be immediately transferred for cost analysis under Task 2 (first iteration).
In the second step, we will determine which modules are still adapted to the actual risks and
anticipated responses (category B). To this end, for each individual response capacity, we will
compile the findings about potential risks (risk assessment under Task 3), information about the
performance of the modules in past operations, missions, projects and trainings, recommendations
by national and international experts and information about any gaps established in the past. We
will contrast this information with the current definitions and quality requirements applicable to the
specific response capacity. Those capacities will be included in category B, for which none of the
following issues are identified:
no underperforming in the past operations, missions, projects and trainings;
no gaps in the performance established;
no expert recommendations issued.
Category B modules also need to fully cover responses to the eventualities of relevant risks.
Once the category B modules are identified, they will be immediately transferred for cost analysis
under Task 2 (second iteration). At the same time, we will maintain this category under observation
as the progress of the Task 4 analysis may have impact on our assessment.
The remaining response capacities (i.e. not fulfilling one of the conditions above) will be categorised
as category C (in need of revision) and “other response capacities” and expert as category D.
Categories C and D will first undergo an analysis under activity 1.5, then submitted to the client for
approval. Only after the approval, category C will be transferred for cost analysis under Task 2
(third iteration).
The recommendations for the revision of definitions and quality requirements for category C
modules will be developed based on the data and information collected in relation to new or
evolved risks (from Task 3), gaps and underperformance established during past operations,
missions, projects and trainings and following recommendations of national and international
experts. We will also take into consideration the definitions and quality requirements of category A
and B modules, which did not need revision, and analyse whether some of the definitions and
quality requirements can be applied to or adjusted to category C modules. Finally, we will take into
account the findings on the revision of capacity goals under Task 4. The suggested changes will be
63
Aerial forest fire fighting (plane and helicopter) is not part of this study, according to the Tender specifications, p. 6.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 55
validated during stakeholder consultations, and their final version will be presented to the client for
approval.
For category D, we will develop general requirements that also include quality and interoperability
requirements. In drafting our recommendations, we will rely on international standards and best
practices (if existing); on past experiences of relevant operations, missions, projects and trainings
where the respective expertise was required or developed; and on the topical risk assessment
(Task 3) which identifies skills and characteristics of (expert) teams necessary for response. To
develop interoperability requirements, we will take into account the quality requirements drafted for
modules. The suggested general requirements will be validated during stakeholder consultations,
checked against the finding of Task 4 and their final version will be presented to the client for
approval.
2.5.3 Deliverables
The deliverables resulting from Task 1 are:
Deliverable 2: Survey questionnaire – a questionnaire comprising questions and answers
(answer categories) for all stakeholder groups, with routing.
Deliverable 3: Report on the survey and desk research, including at least the following
elements:
Table 2.10 below presents several specific risks and mitigation measures that refer directly to the
analysis to be performed under Task 1.
56 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Risk I P Mitigation
requirements acceptable and suitable requirements with them during the project.
for all Participating Countries. Lastly, we will also validate the suggested
definitions with stakeholders before presenting
them to the client. This approach ensures that all
stakeholder considerations are taking into
account, and they have been included in the
process of the revision.
2.6.1 Objective
The overall project objective is to respond better and prepare for natural and man-made disasters
including more efficient and effective process with an optimal cost. Task 2 contributes to this
objective by aiming to provide updated data with regard to the development, deployment,
operational and maintenance costs of the capacities used in response to disasters. Moreover, such
update should include the most recent market data such as labour costs and expenditures.
The costs to be analysed for 34 countries (all EU Member States and 6 additional countries
participating in the UCPM) relate to services handling in response to natural, technological, and
radiological disasters or environmental accidents, including accidental marine pollution, wildfires,
floods, earthquakes, hurricanes, industrial accidents. The cost categories to be analysed are:
development, transport, operational and maintenance. The project data collection activities will not
include personal data, therefore no issues are envisaged in relation to General Data Protection
Regulation but overall project includes very sensitive data, therefore the highest confidentiality
requirements will be followed.
Data collection for all tasks, including Task 2, will be conducted in a centralised manner by the
dedicated team of national correspondents. The task team will ensure that relevant questions are
included in the survey, interviews and data needs are made explicit to the national correspondents
doing the desk research. Following the data collection, different capacities scenarios will be
developed and this will be only Task 2 related activity.
Team VVA
The link between tasks relates to experts involved, data collection activities and task outputs. The
summary of interaction of Task 2 with other tasks is presented below:
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 57
Experts involved. Task 1, 3, 4 experts will provide information about different stakeholders and
data sources where cost information could be obtained. Experts will review the capacities cost
scenarios defined and final results of capacities cost calculation.
Desk research. During the desk research for Task 1, 3, 4 capacities cost information will be
shared with the Task 2 project team.
Survey. The survey sent out to different stakeholders will include capacity cost related
questions to collect information for Task 2.
Interviews. The interviews conducted will include questions related to capacities costs.
Outputs of Task 1, 3, 4 in relation to capacities definition. New capacities definitions will be
used to collect capacities related cost information.
These activities will be executed in three iterations, but in effective and efficient way by using
synergies between activities. The process iterations are based on the reporting requirements, which
include three reports, and on data availability. The main project iteration blocks are presented
below.
According to Commission Implementation Decision of 16 October 2014 laying down rules for the
implementation of Decision No 1313/2013/EU, Annex II “General requirements for modules and
technical assistance and support teams”, the following elements of emergency operations are
defined:
58 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
15 modules64 - for each module tasks, capacities and main components are defined. The
examples of modules include high capacity pumping; water purification; medium urban search
and rescue, etc.
Capacities are defined for each module, more than 100 if different sub-categories are taken
into account. The examples of main capacities include – medium and high capacity pumps;
sufficient personnel to fulfil the task; mobile water storage unit etc.
The capacity costs will be analysed for all UCPM Participating Countries (34 countries). Each
module and capacities short description is included in the Implementation Decision. During the
project the description of selected modules and capacities will be revised, therefore also related
costs to be revised. According to the Terms of Reference, cost analysis to be performed based on
the revised definition of response capacities, therefore the activity will start once the revised
capacity definitions will be available but taking into account very close deadlines the task will start
after Inception Report approval.
In order to understand potential shortcomings of the disaster response capacities under the UCPM,
it is important to define the category of costs that should be analysed while deploying the response
capacities. The following category of cost will be considered:
Development costs are the costs incurred to establish a given model or a team which would
be ready for immediate deployment if disasters strike. The definition applies to both material
and human resources at the same time and refers to such activities as renting, dry leasing, pre-
positioning at a provider’s location.65
Transport costs are the costs incurred to transport material and human resources where the
disaster occurs. It covers installation and all necessary process needed to render the module
operational.
Operational costs are the costs incurred for the running of the model during the entire period
of the mission deployment. The operation costs cover a wide range of items such as material,
staff salaries, accommodation, food, fuel, electricity. These costs are clearly the heaviest
burden for the response capacities.66 However, the operational costs inside and outside the EU
territory can vary considerably.
Maintenance costs refer to costs which are needed for maintain the modules in standby while
they are not used for the future deployment in the case a disaster strikes. Maintenance costs
cover a wide range of expenses such as trainings, storage, warehousing as well as replacement
of expired medication.67
In addition, costs will be split between equipment and human resources costs. For several cost
categories, EU and non-EU costs to be analysed.
The cost categories for different capacities can have a different meaning, therefore for each module
a Module Fiche will be prepared. The Module Fiche will be prepared with support of Task 1
experts, and draft document will be discussed with some Member States representatives and DG
ECHO. Data collection activities will start only after approval of Module Fiche to ensure a common
understanding. If needed, during the data collection activities Module Fiche will be updated to
64
According to the Terms of Reference, aerial forest fire fighting modules (2 modules – by plane and by helicopter) are
excluded from the scope of the study.
65
Following the indication provided in the TOR.
66
Ibid.
67
Ibid
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 59
reflect additional assumptions made or additional data sources found. Example of information
included in Module Fiche is presented below.
As stated above costs will be analysed for 34 Participating Countries, therefore it is important to get
the factual information about all capacities for countries included in the project scope. The basis for
country level information will be Common Emergency Communication and Information System
(CECIS) database where Member States shall register their modules, technical assistance and
support teams, other response capacities and experts. Given the CECIS database will not be fully
available to the project team, the data will be complemented by the survey and stakeholder
interviews.
All data received and obtained during an analysis will be recorded in Excel spreadsheet with a
separate sheet per capacity, where in columns different cost categories will be presented and
each raw will represent a different country. An example of information collection table is presented
below. The table also will include a reference to data resource used and during the project it will be
decided whether reference is needed at country level or it is sufficient to have it at cost category
level. In some cases, the table will also include additional columns where number of similar
elements in the capacity will be defined, for instance, number of tents, and cost per one unit will be
included in the cost table.
60 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
serve as the foundation upon which the cost analysis will be conducted. To this end, Task 2 team
will provide concrete input for both the interviews (to be conducted by the national correspondents)
and survey. This input will take shape of a list of questions that need to be asked during the
interviews and/or survey. An initial outline of the items related to Task 2 that shall be addressed
during the interviews and survey includes:
Capacities costs
Development costs
Transport costs
Operational costs
Maintenance costs
Background and technical information on capacities
Cost scenarios defined for certain capacities
At this stage, we assume that all topics will be covered by both the interviews and the survey.
However, during the inception phase of the project, a distinction will be made outlining which topics
will be addressed in which data collection method (and why). Furthermore, the Task 2 team will
provide the national correspondents with guidance with respect to the desk research exercise. In
order to collect relevant data of high quality and detail, national correspondents will be trained by
Task 2 team and civil protection experts involved in the project. This will be done in a briefing (on
the basis of the developed guidance) explicating the specifics of the data required and potential
data sources. Throughout the data collection period, the Task 2 team will remain at disposal of the
national correspondents and can provide advice and additional coaching if necessary.
As the stakeholder consultation activities are envisaged to take place throughout the various tasks,
a broad set of stakeholders will be approached. The following stakeholders are likely to hold data
and information necessary for Task 2 and will be interviewed or surveyed:
DG ECHO and other DGs (e.g. MARE, CLIMA)
EU-level agencies (e.g. European Defence Agency, European Maritime Safety Agency)
Emergency Response Coordination Centre (ERCC)
Member States' UCPM contact points
Competent national authorities
Relevant industries
Specialised non-government organisations (e.g. Samariterbund)
Other stakeholders - International Organization for Migration; International Federation of Red
Cross and Red Crescent Societies; Organisation for the Prohibition of Chemical Weapon, World
Bank, OECD, ANO, United Nations
Given the priority indicated by the Client to receive the cost analysis, data collection will consist of
three individual iterations. These iterations will receive input from the data collection methods that
have been carried out at the time the deliverable for the specific iteration is due. However, as the
first iteration is due relatively fast after the kick-off of the project, limited data collection methods can
be employed and, therefore, this iteration will be predominantly built on the consultation of publicly
available data (through desk research) and the knowledge of the project team’s experts. The
second iteration will be based, in addition to the above source, on the findings from the survey and
the interviews. The third iteration will add an additional round of interviews to its analysis and
combine all data and analysis obtained at that point, including the third round of interviews.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 61
Public data (first, second and third iteration)
During the first project iteration, the main data source will be public data, therefore during the
proposal stage already several data sources have been identified. These data sources will primarily
be used to gather information regarding the appropriateness of capacities as well as to collect data
on the various kinds of costs that those capacities have. The following sources have been identified
to be of use in this exercise:
for instance the following data to be used in desk research:
Information from Country card template68 - contact information at Member State level;
Recent economic data of the European market (EU Member States) - the costs of equipment
and human recourses. The main data source will be EUROSTAT;
Recent economic data of Iceland, Montenegro, Norway, Serbia, the Former Yugoslav Republic
of Macedonia and Turkey. The main data source will be World Bank;
OECD statistics (detailed information available about health and education);
Suppliers publicly available data about different equipment categories;
Relevant market and manufactures data on modules and capacities;
Relevant studies commissioned by EU institutions.
Studies and reports from Participating Countries.
Based on the discussions with DG ECHO and selected project stakeholders, the list will be
elaborated further and the suggested sources will be further explored to provide the Client with a
more detailed list of concrete reports and publications.
The data retrieved by means of this exercise will be gathered and documented in the module fiche.
Based on the publicly available information, the Task team will execute the first iteration of cost
analysis, which will be conducted for modules of category A and B identified under Task 1.
The experts involved in other tasks will also be asked to provide information about public data
sources on capacities, capacities suppliers and contact information, capacities’ technical
descriptions, expert knowledge about capacities costs and functionality, feedback on cost scenarios
defined and feedback on initial cost information.
68
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.320.01.0001.01.ENG
62 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
to integrate into the survey in order to be able to effectively conduct the cost analysis in the second
and third iteration. These suggestions for questions will be discussed with the Commission in week
5 when the draft of the survey is envisaged to be ready.
As not all stakeholders approached through the survey will have knowledge on the costs related to
the capacities, the project team will ensure that respondents can either choose to skip those
questions or the survey will be programmed in such a way that only a particular type of
stakeholders will be shown the questions related to costs.
The interviews conducted during the first and second round will feed into the second iteration of the
cost analysis. The third iteration of the cost analysis will also include the findings from the third
round of interviews.
During the interviews, the prepared Module fiches will serve as the basis for data collection as
these highlight exactly which information needs to be collected in order to conduct accurate cost
analysis.
The main steps in the cost adjustment / scenario definition process will be as follows:
To understand for which cost categories cost adjustments to be made (e.g. wages)
To understand capacities which will require scenario definitions, for instance, different
technologies;
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 63
To identify and agree with the Client the cost data adjustment sources, (for instance
EUROSTAT – purchasing power parity, Harmonised Consumer Price Index, EUROSTAT labour
market statistics)
To update Module Fiche capacities cost categories descriptions regarding cost adjustment
methodology applied for specific cost category and to update in relation to scenarios defined.
2.6.3 Deliverables
Deliverable 4: Cost analysis
Task 2 deliverables include three reports on capacities which will be prepared in three iterations.
The reports will be prepared based on the required deadline, data collection principles and data
sources, and capacities to be covered by specific report. The summary of reports to be prepared
are presented in the table below.
The reports to be submitted will include the following sub-reports, which could be further tailored
and developed, based on ECHO needs. During the Inception phase the reporting requirements will
be discussed in detail with ECHO, for instance country reports, consolidating reports for several
capacities.
64 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Cost variation scenarios in cases when different technologies were used, including details of the
new technologies, their costs and how such innovation could improve the intervention in case of
disasters.
Combined cost scenario per capacity will be prepared for each capacity and cover scenarios in
relation to wages, similar elements in capacity and technologies.
Combines cost scenario report in table format where three cost variation scenarios are linked.
All reports described above will be available to DG ECHO as internal versions. Based on the
internal versions one public version of the report will be prepared including summary information
and information, which has been obtained from public sources.
Risk I P Mitigation
Insufficient number of Medium Medium Close cooperation and planning with other project
participants selected for tasks, involvement of industry experts from other
targeted consultation tasks, cooperation with national stakeholders
Insufficient cost data about Medium Medium Close cooperation with experts from other tasks in
capacities order to identify suppliers
Close cooperation with national experts who can
provide an information about capacities and suppliers
Subjective view in the Low Medium Close cooperation with industry suppliers in order to
technology scenario get their views on technical aspects of capacities and
definition data verification from different data sources
2.7.1 Objective
The objective of Task 3 is two-fold:
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 65
1. To carry out an assessment of the EU internal and external safety and security risks; and
2. To carry out risk-informed gap analysis of the existing capacities under the EERC and CECIS.
The outputs of the Task 3 will feed into all other tasks. However, before engaging in risk mapping, it
is important to identify the time horizon to be ascertained for risk assessment. This will need to be
discussed with the DG ECHO prior to the start of the risk assessment.
Objective To carry out comprehensive risk-based capacity assessment and capacity gap
analysis
Timing Week 0 - 8
The findings from the risk assessment and capability-gap analysis will inform all other tasks. Hence,
the overview of the existing and emerging safety and security risks for Europe, which is one of the
outputs of Task 3, will inform Task 1. This overview will be critical to analyse the appropriateness of
the existing response capacities. The findings of Task 3 will be used to define capacity categories
(A, B, C, and D respectively).
The Task 3 team will also work in cooperation with Task 4 team to review the capacity goals in light
of the findings from risk assessment. Based on the findings of risk-based capacity analysis from
Task 3, the categories D and C will be analysed in details: what are the emerging risks? Which
capacities the UCPM is missing? Which capacities needs to be revised and strengthened? Which
are the requirements to the new capacities and those that need revision? Answers to these
questions will be summarised in a list of preliminary recommendations, which will include the
definitions of new capacities, typologies of the UCPM’s capacities, and when appropriate, the
quality requirements of new capacities. The list will be submitted to DG ECHO and only after its
approval the further work on cost analysis for categories C and D will be organised by Task 2.
The findings from the risk assessment and risk-based capacity analysis carried out under Task 3
will further inform (a) the content of the survey 1 and (b) the series of interviews and stakeholders’
consultations. Further, the Task 3 team will be engaged in conceptualising the methodological
framework to be finalised under the Task 4.
66 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Activity 3.1 Risk data collection and preliminary analysis
The available national risk assessments from all 34 countries participating in the UCPM will be
analysed. The focus will be on the high likelihood and medium to high impact risks as well as low
likelihood and high impact risks. The risk mapping will be carried out based on specific criteria to be
fine-tuned during the desk research. The criteria would include, but not limited to, the following:
Geographic distribution of risks per countries: the national risks assessments from each
participating state will be analysed, cross-country comparisons made, and when relevant
general trends revealed in terms of risks geographic distribution across the Europe
External risks will be analysed (a) based on regional perspective: Eastern neighbourhood and
Southern neighbourhood; and (b) global perspective for humanitarian risks where the UCPM
might be activated.
Typology of risks: additional analysis will take place to reveal the typology of the risks the
Union is facing. The categorisation for the typology will be further fine-tuned, however, it will be
based on the conceptualisation of ‘crisis’ within the EU and will include (a) man-made crisis
(including armed conflict, terrorism, etc.), (b) natural disasters (based on UNISDR’s hazard
definition - earthquakes, floods, droughts, wild fires, health epidemics, industrial or transport
accidents, fires), and (c) complex emergencies, a combination of natural and man-made crisis
which result in ‘a total or considerable breakdown of authority, which require international
response that goes beyond the mandate or capacity of any single agency, and which has been
assessed to require intensive and extensive political and management coordination’. 69
Risk severity (i.e. likelihood X impact): the EU-level overview of the risks will be analysed
based on the first-ever EU-wide risk overview carried out by JRC in collaboration with DG
ECHO in 2017. When relevant the information will be combined with the national and external
risks, the compatibility of the findings would require additional elaboration from the project team.
Various sources will be analysed to map the existing and emerging risks outside the EU that can
impact the safety and security of the European Union as well as humanitarian risks where the
UCPM is likely to be activated to provide necessary support for disaster/ emergency response. As
mentioned above, external risks will be analysed from regional (Eastern and Southern) and global
(humanitarian risks) perspective. All three typologies of risks will be explored: caused by natural
hazards, by man-made crisis, and risks of humanitarian crisis. For the risk of natural disaster there
are some reliable early warning mechanisms in place already (for instance, the European Flood
Awareness System (EFAS)) and some global indexes developed (such as INFORM70); however,
the risk mapping for man-made and compound crisis would require additional analysis.
The imperative to understand the contextual risks71 and ensure adequate preparedness has
triggered multiple innovative and useful developments at national, regional and global levels. They
include risk monitoring tools, early warning indicators, various platforms for combined data analysis
and visualization, etc. Given the paramount complexity of this task, the state-of-art developments
by leading international organisations will be analysed. This includes, but is not limited to, the work
carried out by:
WB on Global Risks Platform72,
UNDP on Global Risk Platform73,
69
https://interagencystandingcommittee.org/system/files/legacy_files/WG16_4.pdf
70
http://www.inform-index.org/
71
https://irgc.epfl.ch/page-158246.html
72
http://www.europe.undp.org/content/geneva/en/home/partnerships/the-global-risk-platform.html
73
http://www.europe.undp.org/content/geneva/en/home/partnerships/the-global-risk-platform.html
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 67
OECD on High Level Risk Forums74, and many others.
The Task 3 team will analyse all recent developments to map the external risks (both natural, man-
made, and compound) that can be of relevance for the safety and security to the Union and to the
operational effectiveness of the UCPM. Special attention will be paid to the recent developments
regarding conceptualization of emerging and systemic risks carried out by International Risk
Governance Council from 2018 (IRGC).75 Special attention will be paid to emerging risk areas such
as CBRN, terrorism, (cybersecurity) risks to critical infrastructure, amongst others. And last but not
least, due attention will be paid to cascading and compound risks that may cause a chain of
reactions and multiply disaster impact.
The risk mapping will be carried out through desk research and relying on inputs from interviews
with leading experts/ stakeholders conducted during the first round of interviews.
When the response capacities and response capacity requirements are analysed vis-à-vis identified
risks, then the existing capacity goals will be compared with the availability of capacities in the
EERC. The purpose is to identify if necessary capacities are easily available for the participating
states and/or when the assistance from the UCPM might be requested. The initial response
capacity gaps will be identified by Task 3 team through exploring which are the missing capacities
within UCPM and which capacities are only partially address the risks. The work will be carried out
via desk research, however, if required additional questions in stakeholders’ interviews may be
included.
When initial response capacity gaps have been identified, the Task 3 team will review the
capacities registered in CECIS as well as any available buffer capacities to determine if they are
sufficient to close the gap. The analysis will be based on the access to the data available from
CECIS to the Team. Given restrictions in accessing the CECIS, the team will submit the overview of
the new capacities to the DG ECHO and request their feedback regarding which capacities should
be considered as priority during the remaining period for the capacity gap analysis. With this, the
team is expecting to receive some additional direction from DG ECHO regarding the capacities
about which there is some assurance at the EU level without actually getting into the specifics of
the quantity and quality of the resources available as buffer capacities for the UCPM.
After that, the Task 3 team will identify (a) those capacity goals and capacity requirements that
would require revision, and (b) those new capacities that are necessary to meet the emerging risks.
On the latter issues, the Task 3 team will cooperate with the Task 1 team because the (preliminary)
findings can already inform the design of the survey and the definitions of categories A, B, and C
modules. The response capacity gap analysis will be carried out in close consultation with the
74
http://www.oecd.org/gov/risk/5th-oecd-high-level-risk-forum.htm
75
https://irgc.epfl.ch/page-158246.html
68 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
UCPM experts that are part of the project team. Also, the engagement of the Task 4 team will be
requested to ensure coherence of deliverables.
To address all three objectives the workshop will be designed in highly engaging way with
presentations by the Task 3 team and group discussions to ensure the whole variability of options
are duly collected. Representatives of other Task teams will be invited to gain insights from the
workshop and engage with experts/ stakeholders. The findings of the workshop will be summarised
and inform the final recommendations. The detailed organisation and process of workshop is
described in Section 2.3.1.
The work will be carried out in cooperation with Task 1 by providing input to shape the list of
capacities and definitions/ requirements in categories C and D. The recommendations validated
during the workshop will directly contribute to shaping the methodological framework for the
assessment of the suitability of the existing capacity goals as planned under the Task 4.
2.7.3 Deliverables
Deliverable 5: Risk based capacity gap analysis
There is one deliverable expected from Task 3 and that is Deliverable 5: Risks based Capacity Gap
Analysis report, which will be developed and finalised eight weeks after the approval of the
Inception Report. It will include all findings from Task 3, namely risk mapping, analysis of the
response capacity goals in light of the identified risks, availability of capacities in the EERC vis-à-vis
risk-informed capacity goals, comprehensive overview of the capacity gaps and the availability of
buffer capacities that can be mobilised through CECIS. The report will also include preliminary
recommendations on new capacity typology and new capacity goals.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 69
Analysis of the response capacity goals in the light of the identified risks
Availability of capacities in the EERC vis-à-vis risk-informed capacity goals
comprehensive overview of the capacity gaps and the availability of buffer capacities that can be
mobilised through CECIS
Preliminary recommendations on new capacity typology and new capacity goals
2.8.1 Objective
The objective of Task 4 is to assess the suitability of existing EERC capacity goals and to prepare
recommendations on their revision. The task involves the development of a methodological
framework assessing EERC capacity goals and a list of potentially significant capacity goals.
Depending on the findings of this task, our team will also provide DG ECHO with a list of capacities
which are considered outdated and/or obsolete and with a draft request to national contact points
which:
Sets out the details of an assessment of potentially significant capacity gaps, and;
Invites them to provide information on any capacities available outside of the EERC.
70 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Outputs Deliverable 6: Capacity goals revision
Deliverable 7: Final report
Timing Weeks 6 - 38
Task 4 exhibits a high degree of interdependence with Tasks 1, 2, and 3 which provide inputs for
the activities and/or deliverables subsumed under it, while activities completed within Task 4 affect
several activities which take place during the later stages of Task 1. The desk research conducted
during the early stage of Task 4 provides inputs into the design of the 2 nd and 3rd rounds of
interviews. At the same time, Task 4 relies on:
Data collected from the 2nd and 3rd rounds of interviews;
Desk research and/or conclusions derived from Task 3, and;
The 3rd iteration of the cost analysis exercise, to provide it with input data for the suitability
assessment which constitutes the basis for all further analysis within the task.
At the same time, the revision of capacity goals in Task 4 will inform whether new definitions and
quality requirements or typologies of capacities are required under Task 1. If this is the case, an
approval by DG ECHO as specified in Task 1 is needed.
The collection of information which feeds into these processes requires a tailored approach to
conducting interviews. Our previous experience shows that interviewers with extensive knowledge
of subject-matter are able to deliver better results. This is because their subject matter expertise
allows interviewers to evaluate the relevance of responses in real time, and facilitates the process
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 71
of asking thoughtful follow-up questions. Put simply, well-informed interviewers are able to collect
better results because they are able to go beyond a standardized question list.
Task 4.1 is to ensure that the data collected over the course of the 2nd and 3rd rounds of interviews
benefits from well-informed interviewers. In concrete terms, this means that activity 4.1 will produce
(or rather contribute to producing) interview guidance and questions. The guidance will include a
‘digestible’ ‘background’ document that provides interviewers with relevant background information,
and a list of relevant interviewees.
In order to facilitate activity 4.1’s input into the 2nd and 3rd rounds of interviews, activities falling
under activity 4.1 will commence in week 6 and conclude in week 10. The deliverables will be
completed and sent to DG ECHO for review by the beginning of week 11, with the expectation
being that feedback will be provided (and addressed) within three weeks.
Methodology
Activity 4.1 is primarily desk research-oriented and supplements the information collected in Tasks
1 and 3. The guiding research questions could canvas the following issues:
1. How do other countries and/or organisations (if applicable) define the technical and/or
operational requirements for their capacity goals? In the event that these differ from those
outlined in ANNEX II of the UCPM implementing Decision (2014/762/EU), what lessons can
be learnt from the observed discrepancy?
2. What criticism has been regularly leveraged against the UCPM capacity goals (and on what
grounds)?
3. Which UCPM participating states have made the best use of the mechanism?
4. Which UCPM participating states are most likely to need the mechanism in the next 5 years?
Based on this, are there any omissions within the UCPM’s current capacity goals?
5. Who are the key stakeholders to interview when it comes to identifying shortcomings in the
UCPM (technical or otherwise) and/or to identifying its drivers?
The questions will be formulated through a combination of desk research and an analysis of inputs
from stakeholder consultations from previous tasks. Stakeholder analysis will be applied to
inform the results of the literature review. The expected input sources, in addition to an explanation
of the to-be-used data collection techniques, are featured in the table below.
72 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Question Input sources Comment(s)
4 Task 3 risk The risk assessment in Task 3 is expected to constitute a key input
assessment, literature into addressing question 4. Additional desk research will be
review conducted to provide contextual information to interviewers.
5 1st round interviews, Question 5 will be addressed by the desk research process, with
Task 3 risk each newly collected data source being collated into a larger
assessment, stakeholder matrix.
stakeholder analysis
The observations derived from the literature review will be tempered through the findings of the
stakeholder analysis. This step is included because it is expected that stakeholder motivations
and/or interests relating to the UCPM are likely to impact the reliability of the data derived from the
literature review (particularly within the public discourse data category), meaning that the team
proposes the stakeholder analysis as a contingency plan to offset the risk of respondent bias. The
stakeholder analysis also feeds directly into the process of compiling a list of stakeholders to
interview.
The reasoning behind the inclusion of a preliminary list of omitted capability goals is that, in order
for questions relating to the cost analysis and the assessment of these capacity goals to be
addressed through the interview process, questions and context pertaining to this subject should be
provided to interviewers at an early phase. Because the ‘omitted capability goal’ list ties in closely
with the list of ‘potentially significant’ capacity goals (hereafter referred to as the prioritisation
exercise) – and because the expectation is that the 3 rd iteration of the cost analysis will feed into
the aforementioned exercise – the Ecorys consortium will provide the baseline of these
deliverables. With this in mind, activity 4.1 has been designed to ensure all relevant information is
collected in an efficient and systematic manner.
In addition to a general overview (geared towards providing interviewees with context), the
‘background’ document will feature an explanation of the reasoning behind chosen questions,
outline feasible answers, and suggest follow-up questions which (depending on context and/or
interviewee) may improve the quality of results.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 73
(wherever possible) be identified during activity 4.1 (on the basis of how other countries conduct
similar analyses, etc.).
Activity 4.2 focuses on analysing data collected through activities of Task 1, the risk assessment
conducted as part of Task 3, and the desk research conducted in activity 4.1. If any research
should fall out of the scope of the aforementioned interviews, the consortium will make use of EU
databases (e.g, Eurostat and CECIS) before resorting to sources such as the World Bank or the
OECD.
Activity 4.2 is expected to be executed from week 23 to week 30. This timing should allow for:
a) the consortium to collect and address DG ECHO’s feedback on the proposed
methodological framework prior to applying it to the collected data, and
b) the required input data (i.e., from the 2rd iteration of the cost analysis, which will include an
exploration of the costs associated with omitted capability goals and with modifying and/or
expanding existing ones) to be collected.
One of the key sub-deliverables for activity 4.2 is a ‘weighting scheme’ which – based on the
information collected throughout Tasks 1, 3, and activity 4.1 – assigns a numerical value denoting
the ‘relative importance’ of the variables on which data has been collected throughout these tasks.
To facilitate this process, the interview questions delivered at the end of activity 4.1 will – where
feasible – be designed to be compatible with ‘Likert’ (a format in which respondents are asked to
score qualitative variables between 1 and 5) and Thematically-based techniques of analysis. Other
techniques which could feasibly be applied are sentiment analysis and word association mining.
The exact methodology for extracting and quantifying qualitative variables from interview data will
vary from variable to variable. As previously outlined, each extraction and/or quantification
methodology will be justified by means of an explanatory text which will be included in the annex of
the methodological report, and which will be prepared on a variable-by-variable basis. These
explanatory texts will (where possible) refer to relevant peer-reviewed (empirical) studies to
maximise the final results’ ability to withstand criticism from within academic circles.
The use of a weighting scheme, along with an accompanying handbook for ‘quantifying’ any
qualitative variables collected throughout the research process is proposed as part of the Ecorys
consortium’s commitment to adhering to a highly robust and replicable methodology.
Methodology
The proposed approach focuses on the quantification of variables which relate to the suitability of
the stated capacity goals. This exercise is done on a gap-by-gap basis and is subsequently applied
to the capacities of the UCPM as a whole. The logic is that an evaluation of the suitability of the
individual capacity goal connects naturally with evaluation of the UCPM as a whole as any
shortcomings in the individual goals will highlight the need for goal amendments and/or goal
additions within the mechanism. Based on our initial desk research, the following high-level
variables are expected to be of relevance for the evaluation of the suitability of the individual
capability goals (as outlined in the request for services):
1. The adequacy of each capability goal’s technical requirements as outlined in Annex II of
Decision No. 2014/762/EU, and;
2. The degree to which the existing capability goals have been met by the ERCC.
These variables will both be made operational through a range of sub-variables, the data for which
will be collected during Tasks 1, 3, and activity 4.1. Regarding the adequacy of each capability
74 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
goal’s technical requirements, the input data will largely be qualitative in nature, and will be derived
from the 2nd and 3rd rounds of interviews and from the risk assessment exercise conducted in Task
3. The overall goal is for the variable to clearly denote to what degree each of the current capacity
goals addresses the risk assessments specified in Task 3. The distinction between variables 1 and
2 is that – while variable 1 is largely theoretical (‘on paper, do the technical requirements outlined in
ANNEX II of Decision No. 2014/762/EU address risks outlined in the risk assessment?’) – variable
2 is far more operational and is geared more towards evaluating whether the current
implementation of these capacity goals is adequately equipped to tackle current (and future) risks.
Given the fact that activity 4.2 focuses on designing a methodological framework for assessing
these variables, activities will largely be comprised of desk research and stakeholder and/or expert
consultation. The consulted stakeholders will be identified under activity 4.1, with the interviews
expected to occur during the 2nd round of interviews. The goal of the respective questions will be to
validate assumptions regarding the relative importance of the previously outlined variables and sub-
variables.
The weighting scheme which will be used to operationalise the prioritisation exercise will be derived
from the 1st round of interviews. In addition, the consortium will seek feedback from DG ECHO to
ensure that the final weighting scheme aligns with your expectation. Because the prioritisation
exercise and the ‘omitted capability goals’ will be offered as baseline services, the methodology for
the prioritisation exercise (i.e. the weighting scheme) will incorporate information regarding the
omitted and/or adjusted capability goals identified as part of Task 3. As a result, the document
outlining the preliminary technical requirements for omitted and/or adjusted capability goals (the first
of the activity 4.2 deliverables) will play a central role in formulating this aspect of the methodology.
In addition to the variables (and sub-variables) outlined in this section, the weighting scheme is
expected to incorporate variables relating to a) the magnitude and/or severity of the risk(s) which a
capability goal is geared towards addressing, and b) the costs associated with implementing,
improving, or maintaining each individual capability goal. This means that activity 4.2 and, by
extension, activity 4.3 are highly dependent on the successful formulation of a set of definitions
and/or typologies of capability goals (both amended and omitted) at the end of Task 3.
Furthermore, activity 4.2 is also dependent on whether we could deliver robust cost estimates of the
resulting (amended) capability goals in Task 2. The final product is a weighting scheme which can
correct for diminishing returns to investments and optimise the process of appropriating public
funds towards capability goals.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 75
2. A draft version of a request to national contact points which aligns with the requirements
outlined in Article 20(3) of Decision No. 2014/763/EU, should these goals (or the
implementation thereof) be found to be inadequate;
3. A final list of revised and (where omitted) formulated capability goals;
4. An overview of ‘potentially significant’ capability goals (i.e., overview of which goals require
the most immediate attention and/or funding);
5. A list of obsolete capability goals, including explanation, and;
6. A final report.
Methodology
The methodology for executing activity 4.3 will be outlined in the methodological framework
developed in activity 4.2 and approved by DG ECHO prior to implementation. Because the
methodology is expected to have a strong quantitative component, the consortium will make use of
(among others) Microsoft Excel, Tableau, and Atlas.TI to process, catalogue and visualise data in a
systematic manner.
The execution of the methodological framework developed in activity 4.2 is expected to result in a
final overview of the suitability of the UCPM’s capability goals and an overview of ‘potentially
significant’ capacity goals (i.e. deliverables 1 and 4 of activity 4.3). The suitability analysis will be
conducted on a goal-by-goal basis and will incorporate goals which have been reformulated and/or
developed as part of the ‘omitted capacities’ exercise in activity 4.1 and the capacity revision’
exercise which takes place at the end of Task 3. A distinction will be made between theoretical and
operational suitability, with theoretical suitability tests being applied to goals which have been
reformulated and/or developed as part of the ‘omitted capabilities’ exercise in activity 4.1 and the
‘capability revision’ exercise which takes place at the end of Task 3, and operational suitability tests
being applied to capacities which are currently part of the UCPM capacity portfolio.
In the event that the aforementioned reformulated and/or newly introduced capacity goals fail the
‘theoretical suitability’ test, the Task 4 team will employ a combination of desk research, follow up
phone interviews, and (depending on results) a validation stakeholder workshop in Brussels to
identify the shortcomings and to elaborate on operational and/or technical requirements.
Reformulated and/or newly introduced capability goals that pass the ‘theoretical suitability’ test will
be submitted to the UCPM experts involved in the project for final validation, after which a final
version of the will be submitted to DG ECHO for final review. The final version of the capability
goals will feature (if applicable) the newly developed and/or reformulated definitions and typologies
delivered by the Task 3 task team, in addition to an overview of the corresponding technical and/or
operational requirements developed by the project expert panel. The technical and/or operational
requirements will adhere to the format used in Annex II of Decision No. 2014/763/EU. The
reasoning behind amendments and/or additions will be included as an annex.
For all capacity goals which are found to be ‘operationally inadequate’ from an implementation point
of view, a request will be drafted which outlines the methodology used in the to-be-completed study
and the results derived therefrom. The request will further invite Member States to provide
information on any capacities which are available outside of the EERC which may serve to increase
the suitability of the mechanism if integrated. The team will work under the assumption that the
aforementioned request will be sent out to all national contact points, and (as such) will make use of
the national correspondents to translate the drafted document to the relevant languages.
76 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Activity 4.3 deliverables will be developed in tandem, with the methodology for their development
being defined by the methodological framework developed during activity 4.2. It is expected that
those capability goals which are evaluated as of the ‘lowest priority’ will also typically be most likely
candidates for elimination. Information regarding this specific parameter will be worked into the
interview questions developed during activity 4.1. The approach towards identifying ‘potentially
significant’ capability goals will be grounded in the use of variable weights, which will incorporate
data derived over the course of the cost analysis. Because the 2nd iteration of the cost analysis will
cover (among others) the newly developed and/or reformulated definitions and typologies delivered
by the Task 1 team, these capabilities will be included in the prioritisation exercise. For all ‘low
priority’ capability goals whose status is obsolete will be validated on the basis of ad-hoc follow-up
phone interviews and/or desk research. An explanatory document highlighting the reasoning behind
this designation will also be made available.
2.8.3 Deliverables
As outlined in the request for services, Task 4 will culminate in – in addition to the ‘intermediary’
deliverables outlined in the previous sections – a list of revised capacity goals and a Final Report,
The contents of these deliverables are outlined below:
Final report
1. Title page;
2. Table of contents
3. A ±200-word abstract;
4. An executive summary of no more than 6 pages outlining a.) the motivation behind the study,
b.) the methodology applied, and c.) the identified results;
5. An introduction;
6. An explanation of the study background;
7. An in-depth overview of the methodology;
8. An in-depth analysis of the identified results;
9. An overview of the resulting conclusions and recommendations, and;
10. A series of annexes which provide information that supplements points 5-9.
The length of the final report will not exceed 80 pages (before annexes). It will be submitted to DG
ECHO no later than 38 weeks after the contract signature.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 77
Risk Probability Impact Mitigation
allow for the discrepancies between interviewees’ willingness to
development and/or volunteer information, and unavailability on the part of
application of the key interviewees. To compensate for this risk, the
methodological Task 4 team has planned activity 4.2 to allow for a.)
framework. the use of the 3rd round of interviews to collect relevant
data (in case the 2nd round proves insufficient, and b.)
the conduct of additional (phone-based) interviews by
the Task team itself. In addition, considerable
resources will be invested into ensuring desk research
is as productive as possible.
Desk research does not Medium Medium The Task 4 team will have access to highly
result in sufficient experienced experts, which will provide them with
information to facilitate relevant venues to pursue, and which will validate the
the preparation of quality of all Task deliverables.
interview material.
78 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
2.9 Planning and deliverables
Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
Task 0
Task 0.1
Task 0.2
Task 1
Task 1.1
Task 1.2
Task 1.3
Task 1.4
Task 1.5
Task 2
Task 2.1
Task 2.2
Task 2.3
Task 2.4
Task 2.5
Task 3
Task 3.1
Task 3.2
Task 3.3
Task 3.4
Task 4
Task 4.1
Task 4.2
Task 4.3
Task 5
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 79
Task 5.1
Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
Deliverables
D. 1
D. 2
D. 3
D. 4.1
D. 4.2
D. 4.3
D. 5
D. 6
D. 7
Meetings
M1
M2
M3
Workshop
M4
M5
M6
M7
M8
M9
80 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
M4: meeting to discuss survey and D4.1 (virtual) M9: Final meeting (in-person)
Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
Task 0
Desk
research
Interviews
Survey
Workshop
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 81
3 Project team
This chapter sets out the general organisation of the Consortium and introduces the project team
that will implement this project. Our approach to project management was in detail outlined in
Section 2.4, as we consider project management a separate horizontal task permeating all other,
content-related tasks of this study.
Building on our experiences and lessons learned from previously-conducted projects on behalf of
DG ECHO and other European Commission DGs and, we propose a clear project team structure
with clearly defined roles and responsibilities This will ensure the accomplishment of both the needs
of the project by means of smooth delivery of all required consultancy, technical support and
assistance services during the running time of the study.
82 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
experts who will support data collection activities that involve reaching out to stakeholders/
experts;
Back-up & support team consisting of staffers which provide the safety net in case a member of
the team is unexpectedly unable to fulfil his/her role.
In this section we present the short profiles of the project team members and provide descriptions
of the roles assigned to them.
Core team
Laura Birkman, MPA, is Senior Consultant at Ecorys, where she is Deputy
Head of the Security and Justice Unit. She has led a number of studies for the
European Commission at the intersection of security and innovation policy. An
expert in disaster risk and crisis management and evaluation studies, Laura has
led a number of projects in this field for the European Commission. Laura is
currently in charge of the design and implementation of the EU disaster risk
management peer review programme run by DG ECHO. In 2015, she led a study for DG HOME
investigating at European level First Responder capability goals and corresponding technological
requirements. In addition to her thematic expertise, Laura is an active mediator and community
builder in the security research domain. She is currently leading advisory and support services
concerning the building of a Community of Users in Secure, Safe and Resilient societies, which
aims to improve the integration of practitioner requirements into EU funded research. She is also
leading the support facility for DG HOME in its chairmanship of the International Forum to Advance
First Responder Innovation (IFAFRI), a global initiative focused on enhancing and expanding the
development of affordable, innovative technology for first responders worldwide. Laura is a Member
of the Dutch Analyst Network National Security and a Member of the Roster of Experts of the UN
Security Council. In 2012, Laura obtained her MPA from Harvard University with a Fulbright
scholarship. She also holds an Executive Master in International European Relations and
Management from the University of Amsterdam and a Joint M.A. (Honours) in Philosophy and
International Relations from the University of St. Andrews.
Project director
The project director (Laura Birkman) is responsible for the overall implementation of the project and
and is charged with ensuring proper communication between members of the project core team,
ensuring that the various task teams are on-track, referring deliverables to the quality assurance
team, and facilitating the process of forging synergies within the consortium itself.
The main external responsibility of the project manager is to represent the project team to DG
ECHO. A critical element of this role will be to ensure that the decentralised consortium structure
outlined under task 1 is functioning smoothly. The project director will maintain transparent and
regular communication with DG ECHO (e-mail and phone, but also regular and ad-hoc face-to-face
meetings as much as needed but at least during the 9 foreseen meetings) to keep the client up to
date with the progress of the individual task teams. An increase in communications frequency is
anticipated in the lead up to the delivery of key outputs and milestones. This will include an update
on any barriers, challenges or other issues that may impact either the quality or timely delivery of
outputs. The project manager will respond to requests and any other issue raised by DG ECHO.
The project manager has as internal responsibility to chair the project core team and to take
general decisions with respect to the contract. She will orchestrate and supervise the process of
quality assurance, with the objective of guaranteeing that all deliverables are of high quality and are
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 83
submitted in a timely manner. She will additionally monitor budgetary and financial matters, provide
content expertise, and contribute (where necessary) to the execution of the tasks. The project
director has a proven track record in the management of projects relating to crisis management.
Her understanding of the nature and scope of the requested services, as well as the context
underpinning them, will thus inform the project core team’s approach to organising workshops,
delivering on community outreach, and aligning the content of training programs to Commission
preferences.
For the delivery of specific requests for services, the project manager will be responsible for:
1. Discussions with the Commission on the scope of services to be carried out under specific
tasks, including responding to potential requests for clarification on behalf of the task leaders;
2. Assisting in the development of work plans for individual assignments and/or tasks (i.e.:
reviewing the project roadmap prepared by the junior project experts);
3. The monitoring of performance metrics;
4. Maintaining contact and communication with DG ECHO regarding progress and notification of
any issues that may impact either the quality or timely delivery of outputs as they emerge;
5. Proposing mitigation actions in the event of delays or quality issues;
6. Ensuring that outputs are delivered to schedule, and;
7. Dealing with any contractual issues, which may arise between the Client and the Consortium.
Alexandra has a strong academic track-record and has published extensively, mainly focusing on
the migration-security nexus. Her latest publications include two book chapters on EU-Belarus
border management and on the impact of the 9/11 terrorist attacks had on the migration policies of
the European Union. She has also co-authored a conference paper on the parliamentary control of
Frontex. Alexandra studied Political Science & International Relations at the Ludwig-Maximilian-
University in Munich, Germany and at The University of Auckland, New Zealand. She holds a M.A.
in Political Science and is currently finalizing her PhD on the EU’s external migration governance. In
her previous career, Alexandra worked as a research fellow and university lecturer at the Jean-
Monnet Chair for European Politics at the University of Passau, Germany, (4 years) as well as a
journalist for leading German press (5 years).
Project coordinator
The main general responsibility of project coordinator (Alexandra Schmid) is to support the project
director in ensuring the effective implementation of the services outlined in the ToR. In particular,
the project coordinator will be responsible for the daily executive management of the consortium.
This includes maintaining regular contact and communication with task leaders, providing guidance
and feedback on deliverables prior to their submission to the quality assurance team, responding to
84 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
any questions or issues raised, and ensuring that quality protocols and provisions are upheld (in
partnership with the Quality Assurance Group outlined below).
The project coordinator’s role differs from that of the project director in that the project coordinator is
expected to be much more hands-on. Where the project director interacts with the consortium
largely through project core team meetings or when task leaders take up contact proactively, the
project coordinator will work closely with the task leaders and with the junior team members to
ensure the project’s implementation aligns with the goals of the project roadmap. The project
director will thus ensure that the agreed-upon approach, direction and responsibilities set out in the
inception phase are executed as planned (both in terms of quality and work plan), and will be
involved in all parts of project and will be charged with maintaining a clear overview of progress.
Mr. Esa Ahlberg has more than 20 years experiences in planning and
conducting national and international civilian crisis management, civil
protection and humanitarian aid trainings and exercises in close collaboration
with International and regional organizations (EU, UN and NATO).
He has been working as a Senior Emergency Planning Officer in the United
Nations Interim Administration Mission for Kosovo (UNMIK) and as a Staff
Officer at NATO Headquarters Euro-Atlantic Disaster Response Coordination
Centre (EADRCC). He is trained EU Civil Protection expert, NATO Civil
Expert, and OPCW’s Qualified Expert. He has EU Civil Protection and NATO
mission experience. He has been involved in various EU projects in the field of civil protection.
Mr. Ahlberg will be involved in Task 1, 2 and 4 and will be predominantly concentrated on
data collection, data analysis and developing recommendations.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 85
Martin Jensen is Managing Director of VVA Brussels SPRL., a public policy
consultancy providing economic, market and regulatory advice to public
sector institutions. Mr Jensen provides direction and leadership on projects
and in business development activities (tenders, proposals, setting up
consortia etc.) in his capacity as Project Director. He steers the work focused
on providing top quality economic, market and regulatory advice. Mr Jensen
has more than 20 years of experience providing research and consultancy
services to the European Commission and the European Investment Bank
(EIB). Mr Jensen has specialised in providing evaluations and impact
assessments of EU policies and programmes, analysing EU governance models, activity based
management and budgeting, and the setting up of executive agencies).
Mr Jensen has participated in the evaluation of regulatory agencies (ECHA-2016, EMSA (2008)
and FRONTEX), has worked on activity based management with DG Budget and assisted the
Commission and EEAS in the preparatory CBA to establish a Shared Service Centre for EU
missions (2014). Mr Jensen has acquired extensive experience in carrying out CBA and
evaluations of EU executive agencies and governance structures (executive agencies and joint
undertakings). He was CBA expert on the initial Cost-Benefit analysis performed to establish the
TENT Executive Agency (2004 and 2005), responsible for modelling and carrying out the actual
cost calculations as well as the updating study part of the proposal to extend the mandate of
TENEA (2005). Since then Mr Jensen was responsible for the retrospective CBA carried out in the
interim evaluation of the EACEA (2009 and TEN-TEA (2007 and 2012). Mr Jensen was leading the
interim evaluation of EAHC (2010) and SESAR JU (2010 and 2014). He is therefore very
acquainted with agency evaluations as well as with the institutional set up and the underlying
regulations (financial regulation, staff regulation and agency regulations). Mr Jensen holds a BA in
Agriculture Economics and a MA in Economics from the University of Copenhagen.
She holds two Master's degrees: MA Sociology (from Yerevan State University, Armenia) and MSc
Public Administration with specialization in Crisis and Safety Management (from the Leiden
University, the Netherlands). Also, she holds several advanced certificates from the world leading
universities: certificate in Leadership in Crisis from the Kennedy School of Government (KSG),
Harvard University, USA; certificate in Blue Ocean Strategy from INSEAD, France; certificate in
International Cooperation and Development from the UN University, Japan, certificate in Risk
Management from the Institute of Risk Management, UK, and many others. She is the CEO &
Founder of the Risk Society organisation (www.risk‐society.com) based in the Netherlands. She is
a member of the Sustainability Board of the DRIVER (Driving Innovation in Crisis Management for
European Resilience) and a visiting lecturer in the Crisis Management State Academy of Armenia.
She is fluent in Dutch, English, Armenian, Russian with good skills in French. She has co-authored
the National Disaster Risk Assessment (NDRA) Guideline (2017) commissioned by UNISDR under
86 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
‘Words into Action’ and authored Risk and Risk Management in Development Aid Practice chapter
in Risk Management Handbook: A practical guide to managing the multiple dimensions of risk as
well as several other publications on risk management and governance for development and
humanitarian fields.
Ms. Stepanyan will lead Task 3 on risk assessments and capability gap analysis.
Mr. Felfernig will be involved in Task 1, 2 and 4 and will be predominantly concentrated on
data collection, data analysis and developing recommendations.
Agnes Rajacic is Senior Disaster Management officer at the IFRC. Her tasks consist of monitoring
on-going, developing and potential future disasters in Europe and in the Caucasus regions,
disasters and crisis structures activation, supporting necessary disaster response upon the need
and helping in applying response tools, supporting Europe Region National Societies in responding
to disaster, in planning, implementing operations in response to different types of disasters,
supporting the Red Cross societies to implement Disaster Risk Reduction efforts and actions and
provide technical input, supporting the implementation of Urban Risk Reduction Programmes in
Europe and Asia, and providing technical support to Red Cross National Societies in the Europe
Region to develop disaster management capacity in line with Federation policies and strategies.
Agnes will fulfil the function of subject expert in the core team, and will contribute to Task 4
activities.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 87
and public sector, Gabriëlle is comfortable coordinating projects with different types of stakeholders
and clients and she enjoys working in international environments.
Ms. Op ‘t Hoog will serve as support in Task 0 and 1 and will be involved in the general
project management, data collection and stakeholder consultation.
Task members
His most recent contracts include the recent thematic management of the Interreg Europe Policy
Learning Platform on Low-carbon economy, the management of the Secretariat for the Urban
Agenda Air Quality Partnership (DG REGIO), the management of the Assistance Mechanism for
the Atlantic Action Plan (DG MARE), the ongoing evaluation of the INTERREG North-West Europe
programme, the mid-term evaluation of the LIFE programme (DG ENV), and the Support to
Copernicus user uptake promotion, communication and dissemination activities (DG GROW). Luigi
is also a certified PRINCE2 project manager with experience in leading teams for EU funded large
projects. He has an MA in Political Science and a Master’s degree in European Politics. He is
Italian native speaker, fluent in English, French and Swedish, with a good knowledge of Spanish.
Dr. Olga Batura is an associate consultant with Ecorys. Olga is a lawyer specialised in EU and
international law. She holds LL.M. Eur. (University of Bremen), Master of European Studies
(Europa-Kolleg Hamburg) and Dr.jur. from the University of Bremen. Before joining Ecorys in
December 2016, Olga worked as a researcher and lecturer at several universities in Germany, was
an associate professor at the European Humanities University in Vilnius, Lithuania, where she
conducted interdisciplinary research and/or taught courses with a strong interdisciplinary
perspective, and she also worked as a senior fellow at the University of Münster, Germany. At
Ecorys, Olga focuses on EU policy research and analysis, including impact assessments and
evaluations. Olga is experienced in various data collection methods, she has conducted surveys
and interviews herself and also supervised teams of national experts gathering data. Both in her
academic work and for various Ecorys projects, she has carried out qualitative data analysis and
case studies.
Ms. Batura will be involved in the data collection and stakeholder consultations of Task 1 as
well as she will be engaged in the analysis of the capacities.
88 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Agnese Macaluso is a consultant in the Regions and Cities sector of Ecorys.
She specializes in urban innovation and sustainable urban development,
regional and local economic development. Before joining Ecorys, Agnese was
a researcher in the domain of security and conflict prevention (violet extremism
and radicalization, urban security, education in post conflict settings). She has
vast experience with data collection and analysis, primarily with qualitative
methodologies, and has delivered methodology trainings internationally, including in DR. Congo
and Macedonia. Agnese has organized several stakeholder consultations, primarily through the use
of interviews and workshops. Notably she coordinated a large stakeholder engagement project on
violence and conflict prevention in The Hague (which included representatives from the education
sector, police, large and small businesses etc.). She has also run the secretariat of the Knowledge
Platform Security and Rule of Law, a network of NGOs, CSOs, research institutes and businesses
set by the Dutch Government to promote knowledge brokering between research and policy where
stakeholder mobilization was essential. Agnese is proficient in Italian, French, Spanish and English.
Ms. Macaluso is the teams in-house stakeholder consultation expert and will support Mr. Lo
Piparo.
Daina Belicka, Board Member, CSE COE, has in-depth knowledge and
understanding of finance and cost accounting boasting over 20 years of
experience at international audit companies. Daina is experienced Project
Manager for EU wide studies including DG MOVE Core Transport Corridor
Studies (phase 1 and 2), DG REGIO major projects quick appraisals, DG ENV
LIFE project financial monitoring and others. Daina’s cost accounting
experience was gained through different consulting projects, for instance,
developing complex business model for all Post Offices in Latvia, developing
business model for Public Services Delivery Agency in Albania covering all branches and more
than 20 different cost categories (project financed by World Bank), further elaborating Agency
Business model by including new cost items (project financed by Europeaid). Daina has obtained
Certified Accountant qualification (ACCA, UK), Certified Internal Auditor qualification (CIA, US),
Certified Information Systems Auditor qualification (CISA, US), Certified Project Manager (Prince 2)
and has MBA (Riga Business School in cooperation with University of Buffalo, US). Daina has been
lecturing at Riga Business School Financial Accounting and Managerial Accounting course to MBA
and executive MBA programs. Daina speaks Latvian, English and Russian.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 89
Thiago Tavares is a civil engineer with a specialisation in transport with over eight years of work
experience in private organisations, public and international institutions. Throughout his career, he
has worked on various transport and infrastructure projects (i.e. road, rail, air) including economic
and policy analysis. He has acquired large experience in the collection and analysis of large
quantities of data as he was responsible for the data collection and analysis in several transport
projects. He has sound experience in the preparation of feasibility studies as well as policy impact
assessments and CBAs. He has developed strong knowledge about the challenges and costs
involved in the development and maintenance of transport and infrastructure projects but also on
different financing instruments such as blending and PPP.
Niels van Wanrooij works at the Security and Justice sector at Ecorys as a
consultant. He studied international political economic issues (MSc, London
School of Economics) and international relations (MSc, Radboud University
Nijmegen). As a Political Scientist, he has been extensively schooled in
qualitative and quantitative research methods (ranging from multiple regression
analysis to process tracing and case studies). He has worked on evaluation and
impact assessment projects for DGs HOME, RTD, ECFIN, and MOVE. Recent studies include the
‘impact assessment lite’ for Explosives Precursors (DG HOME), an evaluation for the SES initiative
(DG MOVE) and research into terrorism and organised crime (Dutch Ministry of Security and
Justice).
Mr. van Wanrooij will be involved in Task 3 and supports the risk assessments and capacity
gap analysis.
Prior to joining Ecorys, she was an Economics Manager at Deloitte Australia for 2 years. She held
an Assistant Director and Principal Economist position at the Australian Bureau of Agricultural and
Resource Economics and Sciences (ABARES) for 6 years. Her earlier roles are Economist at
Mekong Economics in Vietnam, a Junior Economist at the Centre for International Economics in
Australia, and a Research and Teaching Assistant at Penn State University in the USA. Her PhD
research focused on the assessment of water quality trading market performance in the US context.
Ms. Nguyen will be involved in Task 4, the development of the assessment methodology and
the development of recommendations.
Hugo van Manen works as a consultant in Ecorys’ Security and Justice sector.
He is currently involved in the International Forum to Advance First Responder
Innovation, Community of Users, and DRIVER+ projects, in addition to DG
ECHO’s peer review programme. Hugo previously worked as an assistant
analyst at the Hague Centre for Strategic Studies, where he conducted
research (both qualitative and quantitative) into security-related issues that
advised policymaking within the Dutch Ministries of Defence and Foreign
Affairs. He holds an MSc in International Public Management and Policy (Public Administration)
90 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
from the Erasmus University in Rotterdam, as well as a BA in International Studies (International
Relations) from Leiden University in The Hague.
Mr. van Manen will be involved in Task 4, the development of the assessment methodology
and the development of recommendations.
Quality control
Brigitte Slot is director at Ecorys, and head of the Security and Justice team.
As such she has led many international projects in the field of migration
(participatory evaluation of European Immigration Liaison officers, study the
fraud and return policies related to the Schengen visa system), illicit financial
flows (money laundering and terrorist financing, financial investigation),
corruption, judicial institutions and organised crime. She has been director of
various justice related studies on behalf of the European Commission (DG Justice, DG Home and
OLAF) and the Dutch Ministry of Justice. Prior to her work as consultant she was senior policy
advisor on anti-money laundering and terrorist financing at the Ministry of Finance in the
Netherlands, where she was project manager of two international studies in money laundering on
behalf of the Financial Action Task Force (FATF): on money laundering in the football sector and on
maritime piracy. She was lead researcher on a study on ill-gotten money in two African countries
(Malawi and Namibia) on behalf of the World Bank. She started her career as international macro-
economic researcher in the financial sector and worked at was associated with the Delft Technical
University for her PhD.
Ms. Slot will be involved in the quality control team in which she will be specifically looking
at the consistency and coherence of the different deliverables.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 91
(2016). Paul holds a B.Sc. in Economics and Law (University College, Cardiff, UK) and a M.A. in
Economics (Queen’s University, Kingston, Canada). He has previously been employed at Ecorys
(Rotterdam, the Netherlands), BIPE Conseil (Paris, France), the Institute for Fiscal Studies
(London, UK) and University College, London (UK).
Mr. Baker will be involved in the Quality control team and will specifically look quality of the
UCPM and evaluation aspects of the project.
Ms. Peeters is part of the quality control team and will concentrate on English language
checks.
Vibeke Weigelt is an experienced consultant with experience from private sector, public sector,
academia and civil sector. She has comprehensive experience with policy formulation, studies,
evaluations, Impact assessments and analysis. She has been in charge of numerous projects for
the EU including DG EMPL, DG JUST, DG Sanco, DG ENV and DG ECHO. Sectors include
different aspects of society: Civil protection, disaster prevention, employment, health, gender, and
social sector. She has solid experience with European policies. Her skills also include innovation
and process competence. She has solid knowledge of humanitarian aid and of the volunteer sector
both from heading a volunteer programme and from performing assignments for the sector, most
recently for Save the Children. She was in charge of the Preparatory study for an Impact
Assessment on the establishment of a European Voluntary Humanitarian Aid Corps for DG ECHO.
She has worked intensively with disaster risk reduction (DRR) and disaster risk management
(DRM), and was in charge of evaluating Evaluation of European Commission's Actions in the field
of Civil Protection 2007-2009 and involved in an expert capacity in the Ex post evaluation of Civil
Protection Financial Instrument and Community Civil Protection Mechanism (recast) 2007-2013.
Through numerous years of international work she has acquired solid project management skills
and experience with qualitative and quantitative methods. She has some 20 years of experience
with international projects and programmes of which 3 years with the European Commission as a
task manager in the EC Tacis programme at DG RELEX.
Ms. Weigelt is part of the quality control team and will specifically check the quality of the
UCPM-related analysis, evaluations and recommendations.
The data collection team consists of a large group of national correspondents who are presented in
the table below.
92 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Country VVA Ecorys
Belgium Elodie Salle
Bulgaria Diliana Mitovska
Croatia Tea Paulovic
Cyprus Alexandros Vigkos
Czech Republic Veronika Brantova
Martin Rune Jensen
Denmark
Estonia Triin Muuk-Adrat
Finland Vilma Kuuliala
France Agnese Macaluso
Germany Alexandra Schmid
Greece Ioannis Giannelos
Hungary Iva Plašilová
Iceland Martin Rune Jensen
Ireland Gabrielle op ‘t Hoog
Italy Agnese Macaluso
Latvia Vesna Sredanovic
Lithuania Milda Butkute
Luxembourg Luc Fischer
Macedonia Vesna Sredanovic
Malta Hugo van Maanen
Montenegro Tea Paulovic
Netherlands Niels van Wanrooij
Norway Neja Bergant
Poland Magdalena Klebba
Iva Plašilová
Portugal
Romania Madalina Nunu
Serbia Vesna Sredanovic
Slovakia Alena Cierna
Slovenia Neja Bergant
Spain Agnese Macaluso
Sweden Alma Agusti Strid
UK Nga Nguyen
Turkey Serdar Demirel
3.2.1 Overview
Core team
Table 3.2 Core team composition
Name Team role Level Organisation
Laura Birkman Project Director Senior Ecorys
Alexandra Schmid Project Coordinator Junior Ecorys
Gabriëlle op ‘t Hoog Project support Junior Ecorys
Esa Ahlberg Lead Task 1 Senior CMC
Martin Jensen Lead Task 2 Senior VVA
Magda Stepanyan Lead Task 3 Senior Risk Society
Michael Felfernig Lead Task 4 Senior CMC
Agnes Rajacic Subject expert Senior IFRC
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 93
Task teams
Table 3.3 Task 0 team composition
Name Team role Level Organisation
Laura Birkman Task lead Senior Ecorys
Alexandra Schmid Task support Junior Ecorys
Gabriëlle op ‘t Hoog Task support Junior Ecorys
94 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
3.2.2 Expertise of our team members
III = highly experienced, II = experienced, I = Some experience
Laura Birkman
Brigitte Slot
Paul Baker
Anita Peeters
Magda Stepanyan
Alexandra Schmid
Luigi Lo Piparo
Nga Nguyen
Agnese Macaluso
Gabriëlle op ‘t Hoog
Olga Batura
Agnes Rajacic
Martin Jensen
Daina Belicka
Magdalena Klebba
Weigelt Vibeke
Thiago Tavares
1 Project management and coordination III III III III I III ||| III I I I II III III III III III I III III
2 Quality control II III III II II III || II I I III III III I III III
3 Logistical support to consortium II II II III I II | I I I I
4 Conducting and analysing interviews III III III III II III ||| II I I III II I III III III III III III
5 Designing and conducting survey III III II II II III | III II I I III I III III I III III
6 Literature analysis III III III III II III ||| III II I I III I I III III III III III III
7 Evaluation of EU programs and initiatives III III III III II III | II II II I III III III III III III
8 Cost analysis II II II II || I I I III III III
9 Quantitative analysis II II III II I III ||| II I I I II I III III III III III
10 Gap analysis II II III III III || II II I III III III II III III
11 Development of assessment methodology II II III III I III || II II I I II I III III II III III
12 Stakeholder engagement and consultation III III III III II III || III I I I III II I III III III III III
13 Organising and conducting workshops II II II III I III || III I I I III II III III III II III III
14 UCPM I II I III III III III III
15 Operational experience with UCPM II III III III II
16 UCPM response capacities I II II III III III
17 Assessment of UCPM capacities I I III III
18 Revision of UCPM capacities I I II III
19 Risk assessments I III II II III III III I III III
20 EERC II III II II III
21 CECIS II III II II III
22 Disaster Risk Reduction II III I
23 Disaster preparedness II III I
24 Post disaster needs assessment and recovery II III
25 Civil Protection II III I
26 International cooperation in disaster management II III
27 Strategic planning II III
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 95
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 97
4 Quality Control
This chapter sets out our bespoke quality control and assurance approach, which we have tailored
specifically in the light of the assignment requirements. The approach incorporates the extensive
experience of the consortium, and in particular, of the lead company Ecorys, with the
implementation of projects in the field of crisis management for the European Commission.
Many of the quality control measures proposed below have been tested in practice and adjusted to
suit the unique needs of the tendered-for project and the assembled consortium. Quality measures
described below concern both content and process quality assurance and are applied at both the
project level as well as at assignment-deliverable level. They follow the work flow from receiving a
request for service to completing the final quality compliance check of all resulting deliverables and
subsequent submission to DG HOME.
In the sections that follow, we first outline a number of principles informing our approach to quality
assurance within the context of this project, and describe our overall QA approach (Section 4.1).
We then elaborate the QA measures that we will put in place for guaranteeing high-quality
deliverables, tailored to the different types of outputs (Section 4.2), followed by a description of how
we address the process-related aspects of quality control (Section 4.3). The chapter then sets out
the risks and constraints as well as the risk mitigation measures (Section 4.4).
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 99
established a multi-layered quality assurance system, which ensures that all deliverables
undergo a dedicated quality checks for: (i) content, (ii) methodology, (iii) language, and (iv) form
and format. We have allocated appropriate resources (in persons and man-days) to properly
assess each deliverable, provide written and verbal comments on its content as well as provide
guidance on sources, data and references to information that will lead to its improvement;
The designated junior project experts will act as ‘first-line’ quality controllers for all deliverables
produced by their respective task team, which will be screened on content, soundness of the
utilized methodology, etc. To ensure that all deliverables are screened by subject experts, we
have consciously included subject-matter experts (Expert Advisors) in the quality
assurance team in order to provide ad-hoc expert advice to our quality manager on difficult,
and/or politically sensitive issues when necessary.
With the above principles in mind, we have ensured that there is a robust, multi-layered review
process from the moment it is received by the quality control team to the moment it is submitted to
DG ECHO. Our procedures ensure that no deliverable will arrive on the Client’s desk without having
been thoroughly reviewed by the quality assurance team.
In this section, we elaborate and tools and approach that will be used to ensure high quality
deliverables and services throughout the project’s 13-month duration. The main components of our
approach are:
A multi-layered quality control process;
A tailored approach to the different types of deliverables;
A dedicated team of experts with clearly defined tasks and roles, and;
Regular collection and analysis of feedback to enable lessons learned.
Table Figure 6 Scope of quality control across quality dimensions describes the scope of the
quality control measures covered at each level and who is responsible for carrying out the controls.
100 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Scope of quality control
Accuracy: Analysis is factual and correct; information can be substantiated in more than one
reliable source; opinions can be supported by facts;
Objectivity: Information and sources are unbiased;
Relevance: Information and analysis address the assistance needs of the EC; information is
concise, avoiding repetition or superfluous content;
Completeness: The content addresses the assignment requirements effectively; or in terms
of completeness of data and information presented;
Coherence and Consistency: Analysis follows a logical flow and avoids logical
inconsistencies in argumentation and contradictory views.
Across-deliverables
Across deliverables within a given assignment, checks cover:
Consistency in terms of overall quality of analysis (scope, detail, policy utility, etc.);
Coherence and consistency in terms of terminology and definitions applied, aligned to those
of the Commission.
Responsible partner
Junior project experts (Project Core team), Quality Manager (Quality Assurance Group).
Linguistic Per deliverable
Proofreading by a linguistic editor or native English speaker shall include:
Checks for errors in the use of the English language, including words, phrasing and
grammar;
Detection and correction of typos, including incorrect punctuation.
Responsible partner
Linguistic editor (Quality Assurance Group). Project assistant (a native English speaker)
provides general language checks on deliverables not requiring professional proofreading.
Editorial Per deliverable
For each deliverable, editorial quality checks are carried out to ensure consistency in terms of:
Form and structure of the information presented;
Visual presentation (layout) of the reports adhere to style and formatting rules, including for
the use of visual graphics, where relevant. Compliance with EC requirements as contracted.
Responsible partner
Project director, supported by deputy project director (Project Core team).
All deliverables will be subject to this layered control process, with some variation in scope and
intensity depending on the nature of the deliverable.
Following the ‘first-line quality control’ carried out by the junior project experts, the Quality
Manager will conduct a second, high-level review of all deliverables, focusing on the overall quality
(content and methodology), coherence and consistency of the deliverables. In some cases, an
additional round of revision and review may be necessary, which will be factored into the work plan
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 101
and resourcing accordingly. In the case of complex and/or politically sensitive deliverables, the
Quality Manager will consult our Expert Advisors within the Quality Assurance Group.
The rationale behind our two-stage quality review of the content and methodological quality of the
deliverables is to ensure that all deliverables produced within the consortium’s various task teams
undergo a thorough review by subject matter experts rather than generalists .
Coherence and consistency of the review procedure will be ensured via the provision of both written
and verbal guidance, standardised checklists and reporting templates. These materials will be
prepared by the Project Core team, and will (where applicable) be tailored to the individual task
outputs. More specifically, the guidance will set out the review criteria and how each criterion is to
be understood in the context of the given assignment.
For all other deliverables, general language and linguistic style checks will be performed as part of
the final layer of quality control (see Layer 4 below), by the project director and deputy project
director, the latter being an English native speaker. At the onset of the contract, we will additionally
provide ‘style of writing’ guidance materials.
We have an established quality procedure (see chapter 3.2.1) for ensuring a language quality
check as required for all outputs and deliverables.
For questions of English style, the Inter-institutional Style Guide76 and the English style guide77 will
be used as a reference. Once the final version of a given deliverable has been agreed, the
deliverable will be subject to a final check and detailed proofreading by our Linguistic Editor before
76
Inter-institutional Style Guide: http://publications.europa.eu/code/en/en-000100.htm.
77
http://ec.europa.eu/translation/english/guidelines/documents/styleguide_english_dgt_en.pdf.
102 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
submission to DG ECHO. The final check, editing and proofreading process will be coordinated by
the project’s project director. Furthermore, our QA process for written outputs provides a guide to
common referencing. All staff are required to be rigorous in the ways in which literature references
are stored, and expect that all written outputs will have a consistent structure. Our junior project
experts will check drafts for bibliographic consistency. Anti-plagiarism checks will also be carried
out as outlined below.
At regular intervals, the project director and the quality manger, in consultation with the
Commission, will take stock of any identified quality issues regarding specific assignments. The
consortium, led by our project directors, additionally proposes to establish a regular feedback
mechanism with the respective project officers at DG ECHO. In particular, we will request
feedback regarding the quality and policy utility of the services offered through the pilot programme
and/or the functionality of tools such as the CMINE and eFIRE-IN. This feedback mechanism could
take the form of teleconferences organized following the completion of a given assignment, which
would be specifically dedicated to evaluating the performance and quality of service delivery on the
previous assignment; or during final report meetings with the Client within a given assignment.
Such discussions and feedback will serve the explicit purpose of helping the Project Core team to
continuously review and improve the quality of our support, as well as to identify any issues
requiring action and a forum for proposing appropriate and timely mitigation actions.
The project director will brief the Project Core team on the outcomes of these sessions, particularly
regarding lessons learned to be adopted in the execution of future activities. This system will
ensure a learning approach within the consortium and foster a strong sense of network coherence
and consistency.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 103
The Consortium has therefore put a few quality control processes in place to address this issue:
1. The Ecorys-led Consortium has obtained ‘Safe-Assignment’ software to check for plagiarism;
2. In the internal Consortium Agreement as well as in the sub-contracting agreements there is a
penalty provision for plagiarism.
When plagiarism is detected, both the project director and the quality manager will be notified and
pointed to the section or sections where the plagiarism was detected. These will be rewritten
immediately. In the Consortium Agreement between the partners, a specific clause states that the
project director will issue a first warning to the junior project expert whose task team is responsible
for authoring the offending content. If the plagiarism is not taken care of fully by the relevant task
team, sanctions will be executed accordingly.
Graphical requirements
We will adhere to all the graphic requirements with regard to the form of the final deliverable(s). The
consortium will make use of standardised Word templates (with automatic headings and paragraph
styles) during the drafting of each deliverables. For deliverables in which specialised visual work is
of relevance, the consortium shall ensure that formatting and layout conform to the corporate visual
identity of the EC by applying the graphic rules set out in the EC’s Visual Identify Manual, including
its logo. The final layout for all deliverables will be verified by our experienced secretarial staff
based in the Ecorys offices in Brussels and Rotterdam.
The consortium has embedded various measures to warrant the smooth and efficient functioning of
the project execution. Next to the overall management structures, as explained in detail the
previous chapter, in this section we elaborate the quality mechanisms that have been put in place
to ensure timely execution of assignments and the continuity of service in view of replacements of
staff and dealing with multiple assignments.
The project director and deputy project director will be constantly monitoring progress and respect
of deadlines. Our project director – in close consultation with the junior project experts – will be in
charge of ensuring the operational adherence to the timetables as set out in each contract and the
precise implementation of a detailed production plan for each deliverable, setting deadlines and
responsibilities for each step of the process.
In order to efficiently monitor the progress of the large number of deliverables foreseen by several
of the tasks outlined in the ToR, we will create an excel database in order to track the status of all
deliverables, including their movement through the multi-layered quality assurance procedure. It will
be maintained by the deputy project director, who will monitor the completion of all steps in our
review process, and flag risks of delay to our project director in the event that particular deliverables
require multiple rounds of content and methodology revision.
104 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
4.3.2 Ensuring continuity of service
The consortium views the continuity of the service during a project’s implementation phase as
imperative to its overall success. Because – as is the case with (among others) Tasks 3, 4, and 5
as outlined in the ToR – the interim outputs of several this project’s Tasks feed into the completion
of deliverables in others, a key challenge presents in the simultaneous management of various
processes and assignments. Outside of requiring a robust management structure, this challenge
can be (partially) mitigated through contingency planning. In principle, the team has (through the
FIRE-IN network) access to expert networks – both formal and informal – in the field wildfire
prevention which extend well beyond those which are presented within this proposal. This
guarantees continuous access to sufficient capacity at all times. In the (unlikely) eventuality that
project workloads surpass team capacity and/or that key knowledge is found to be absent, the
consortium will supplement its capacity by subcontracting high level experts from the extensive
network to which it has access.
Subcontracting arrangements
We have built into our subcontracting arrangements (and foresee in the contractual agreement) a
provision to accommodate the replacement of a subject matter expert if required. This will also
provide flexibility in the case of changes in the needs of DG ECHO. This procedure will be
instigated if:
The project officers at DG ECHO do not feel that the task teams’ approaches and/or outputs are
acceptable or of high enough quality;
The subject matter expert continually produces sub-standard quality deliverables- this includes
producing badly written, poorly analysed, poorly referenced outputs which are not to a standard
expected of Ecorys and DG ECHO;
The subject matter expert continually produces late deliverables- including outputs that are sent
to Ecorys beyond the deadline given.
We will inform the client of a proposed change of expert in writing, explaining the reason behind this
decision. We will then seek to source a replacement expert and send their CV and background
details to the client for formal approval. After client agreement, we will then fully brief the new expert
on expectations and tasks before their commencement of the role.
It is our aim to be transparent and pro-active about the potential risks that the consortium may face
during the provision of the requested services. In the tables below we present potential risks,
including their probability and impact (L = low, M = medium and H = high), as well as our preventive
and contingency measures. These comprise more generic risks that are valid in the overall
management of the contract as well as specific risks related to the Tasks subsumed under this
project.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 105
Challenge I P Mitigation measure
implementation. Inclusion of safety margins around
critical milestones to reduce the impact of encountered
delays by having regularly team meetings.
Difficulty in identifying the H M Mobilise national experts who have direct access and can
stakeholders engage the stakeholders, or reach out to international and
local platforms to facilitate access
Low/participation response H H Pre-alert participants before survey/interviews are held,
rate explain context, confidentiality, potential issues with
sensitivity
Send reminders and use networks above to facilitate
Design of questionnaires with a reasonable number
of questions (mostly closed type of questions)
Test of questionnaires prior to publishing
Support from the client in identifying the most proper
stakeholders and provision of email contacts
Use of interviews and to provide further answers
Follow up e-mails and phone calls
Need to use local language M M Deploy experts able to cover all languages, or most of them.
with Survey can be translated easily with no need of a local
surveys/interviews/workshops expert.
Little availability of data H M As such, for a proper evidence-based evaluation, we will rely
on input from coming from different sources. We expect that
much of this information requirement can be covered with
input from targeted survey and interviews.
106 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Challenge I P Mitigation measure
Non-coherent presentation M M Detailed and rigorous preparation of the templates
of information across various and protocols and guidelines for the interviews,
Member rigorous quality control of the performed activities by
States/interviewees/other the task leaders, regular meetings with the team to
categories update on such a process and quality of the
intermediate deliverables
Difficulty to retrieve data by M M In case the national correspondents experience severe
national correspondents difficulties obtaining information, the project team has set
aside budget to involve national experts who are better
connected and will most likely be able to retrieve this
information more effectively. The list of national contact
points in Annex I will be consulted and national experts will
be approached based on which national correspondents
experience most difficulties.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 107
Risk I P Mitigation
taking into account and they have been included in the
process of the revision.
108 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Risk I P Mitigation
collect relevant data (in case the 2nd round proves
insufficient, and b.) the conduct of additional (phone-based)
interviews by the Task team itself. In addition, considerable
resources will be invested into ensuring desk research is as
productive as possible.
Desk research does not M M The Task 4 team will have access to highly experienced
result in sufficient experts, which will provide them with relevant venues to
information to facilitate the pursue, and which will validate the quality of all Task
preparation of interview deliverables.
material.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 109
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 111
Annex I National contact points UCPM
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 113
Country Contact point Contact person E-mail address
Poland State Fire Service Wiesław Puchalski bwm@kgpsp.gov.pl
Portugal Civil Protection Authority Rute Ricardo Rute.ricardo@prociv.pt
Romania Permanent loan Nitica ioni6I@yahoo.com
Representation/Ministry
of Interior
Serbia Ivan Baras ivan.baras@mup.gov.rs
Slovakia Permanent Ingrid Stanova ingrid.stanova@mzv.sk
Representation
Slovenia Permanent Srečko Šestan srecko.sestan(at)mors.si
Representation/Civil
Protection
Administration
Spain Ministry of Interior Juan Crespo Sanchez juan.crespo@policia.es
Sweden MSB/Ministry of Defence Anders M. Johansson anders.m.johansson@civil.se
United-Kingdom Cabinet Office Simon Lewis simonlewis@redcross.org.uk
Turkey Prime Ministry Disaster Hüseyin Alp Kaya halp.kaya@afad.gov.tr
and Emergency
114 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Country Authority Name
Austria Federal Ministry of the interior Institute for Johann BEZDEKA
science and research Robert STOCKER
National Civil Protection and Disaster Relief Karin ZETTELMANN
Affairs Mirjana JAKOPEC
Kurt KALCHER
Hubert VETTER
Belgium Home Affairs, National Centre of expertise for Jérôme GLORIE
civil protection Marc LOOZE
Sebastiaan DRUYTS
Anne BAETENS
Bulgaria Ministry of State Policy for Disasters and Nikolay NIKOLOV
Accidents DG
National Civil Protection Antoaneta BOYCHEVA
Maria STANIMIROVA
Croatia National Protection and Rescue Directorate Jadran Perinić
Cyprus Ministry of Interior - Cyprus Civil Defence Force Maria PAPA
HQ
Czech Republic General Directorat of Czech Fire and Rescue Jiri CHALUPA
Service - Ministry of Interior Drahoslav RYBA
Miloš SVOBODA
Jiří MUSÍLEK
Denmark Danish Emergency Management Agency Simon Søborg AGGER
(DEMA), Centre for Education and HR Michael Boesgaard BRØNDEL
Development Flemming S. NIELSEN
Ministry of Defence
Estonia Estonian Rescue Board Kuno TAMMEARU
Alo TAMMSALU
Jevgeni Jutkevitś
Finland Ministry of the Interior Esko Koskinen
Eero KYTÖMAA
France Ministry of Interior, Direction of defence and Manuel VALLS
civil security Philippe NARDIN
Germany Federal Ministry of the Interior Franz-Josef HAMMERL
Dr. Klaus-Georg MEYER-
TESCHENDORF
Juergen POLKA
Franz Josef MOLITOR
Greece Ministry of Public Order and Citizen Protection Patroklos GEORGIADIS
General Secretariat for Civil Protection Dr. Kalliopi SAINI
Dr. Olga KAKALIAGOU
Hungary Ministry of the Interior - National Directorate Dr. György BAKONDI
General for Disaster Management (NDGDM) Dr. Attila NYIKOS
Ireland Department of Environment, Community & Kenny BRIAN
Local Government
Italy Presidenza del Consiglio dei Ministri - Dr. Franco GABRIELLI
Dipartimento della Protezione Civile Luigi D’ANGELO
Latvia Latvian State Fire and Rescue Service Oskars ĀBOLIŅŠ
Jānis IVANOVSKIS
Lithuania Fire and Rescue Department, Ministry of General Remigijus BANIULIS
Interior Tatjana MILKAMANOVIC
Raimondas BOGDEVICIUS
Luxemburg Ministry of Interior Jean-Marie HALSDORF
Malta Civil Protection Department John RIZZO
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 115
Albert TABONE
Michel GALEA
Peter-Paul COLEIRO
Netherlands Ministry of Security and Justice Theo LODDER
Martijn Meeder
National Crisis Centre Chris van Duuren
Poland National Headquarters of the State Fire Service Wieslaw LESNIAKIEWICZ
Malgorzata TRZCINSKA
Tomasz KOLODZIEJCZYK
Maciej GODLEWSKI
Adrian BUCALOWSKI
Dariusz KADUKOWSKI
Anita KOŁODYŃSKA
Portugal ANPC - National Authority for Civil Protection Major-General Manuel COUTO
Ana FREITAS
Romania General Inspectorate for Emergency Situations Colonel Nicolae CORNEA
Ministry of Internal Affaires Colonel eng. Marius DOGEANU
Major Vlad PETRE
Slovakia Ministry of Interior of the Slovak Republic Lenka Hmírová
Section of Crisis Management Andrea Geguššová
Jaroslav Valko
Slovenia Ministry of Defence -Administration for Civil Darko But
Protection and Disaster Relief Danica Babič
Spain Ministerio del Interior -Dirección General de Juan Antonio DÍAZ CRUZ
Protección Civil y Emergencias uan Pedro LAHORE LACOSTE-
PEDELABORDE
Sweden Swedish Civil Contingencies Agency (MSB)- Helena Lindberg
Swedish Rescue Services Agency Therese Wikström
Selma Ilijazović
UK Cabinet Office
Civil Contingencies Secretariat
Iceland Icelandic Police – Civil Protection Vidir REYNISSON
Guðrún JÓHANNESDÓTTIR
Norway Ministry of Justice and Public Security Ørjan N. KARLSSON
Geir Arne FREDRIKSEN
Hanne B S ERIKSEN
Turkey Prime Ministry Disaster and Emergency Fuat OKTAY
Management Presidency
FYROM Protection and Rescue Directorate of the Shaban Saliu
former Yugoslav Republic of Macedonia
116 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Annex II Interview guidelines
Lot 1
Task 1
National Civil Protection needs
Match between needs on national level and UCPM capacities in case of emergency in participating
states;
Capacities of UCPM Voluntary Pool
Capacities of CECIS
Capacities of European Civil Protection Pool
Capacities foreseen in rescEU
National Civil Protection capacities of the participating states that could be used in UCPM operations
(not included in the current UCPM Voluntary Pool or CECIS either in the planned European Civil
Protection Pool or rescEU);
Risks regarding civil protection outside the UCPM participating states
What additional Civil Protection assets or support is be needed regarding disasters outside UCPM
participating states
Adjustments in modules/capacities necessary in the view of new/ changed risks
Adjustments in response capacities necessary based on past experiences (exercises, operations,
training)
Interoperability issues for experts
Other requirements to experts
Task 2
Capacities costs
Development costs
Transport costs
Operational costs
Maintenance costs
Background and technical information on capacities
Cost scenarios defined for certain capacities
Task 3
National Civil Protection risks
Match between risks and needs on the national level
Lot 2
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 117
Interview Guide Lot 2
Introduction of the study
Introduction of the interviewee
Task 1
National Civil Protection needs
Match between needs on national level and UCPM capacities in case of emergency in participating
states;
Capacities of UCPM Voluntary Pool
Capacities of CECIS
Capacities of European Civil Protection Pool
Capacities foreseen in rescEU
National Civil Protection capacities of the participating states that could be used in UCPM operations
(not included in the current UCPM Voluntary Pool or CECIS either in the planned European Civil
Protection Pool or rescEU);
Risks regarding civil protection outside the UCPM participating states
What additional Civil Protection assets or support is be needed regarding disasters outside UCPM
participating states
Adjustments in modules/ capacities necessary in the view of new/ changed risks
Adjustments in response capacities necessary based on past experiences (exercises, operations,
training)
Interoperability issues for experts
Other requirements to experts
Task 2
Capacities costs
Development costs
Transport costs
Operational costs
Maintenance costs
Background and technical information on capacities
Cost scenarios defined for certain capacities
Task 4
How do other countries and/or organisations (if applicable) define the technical and/or operational
requirements for their capacity goals? In the event that these differ from those outlined in ANNEX II of
the UCPM implementing decision (2014/762/EU), what lessons can be learnt from the observed
discrepancy?
What criticism has been regularly leveraged against the UCPM capacity goals (and on what grounds)?
Which UCPM participating states have made the best use of the mechanism?
Which UCPM participating states are most likely to need the mechanism in the next 5 years? Based on
this, are there any omissions within the UCPM’s current capacity goals?
Who are the key stakeholders to interview when it comes to identifying shortcomings in the UCPM
(technical or otherwise) and/or to identifying its drivers?
Lot 3
118 Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism
Interview Guide Lot 3
Introduction of the study
Introduction of the interviewee
Task 1
What implications a specific definition X (quality requirement Y) has for your national civil protection
response capacities (application of the definition, possibility to implement, cost factors)?
Does a spoecific requirement X improve/ ensure interoperability of the expert?
Task 2
Capacities costs
Development costs
Transport costs
Operational costs
Maintenance costs
Background and technical information on capacities
Cost scenarios defined for certain capacities
Task 4
How do other countries and/or organisations (if applicable) define the technical and/or operational
requirements for their capacity goals? In the event that these differ from those outlined in ANNEX II of
the UCPM implementing decision (2014/762/EU), what lessons can be learnt from the observed
discrepancy?
What criticism has been regularly leveraged against the UCPM capacity goals (and on what grounds)?
Which UCPM participating states have made the best use of the mechanism?
Which UCPM participating states are most likely to need the mechanism in the next 5 years? Based on this,
are there any omissions within the UCPM’s current capacity goals?
Who are the key stakeholders to interview when it comes to identifying shortcomings in the UCPM (technical
or otherwise) and/or to identifying its drivers?
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 119
Annex III Checkmarket survey
120 FJK97864
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 121
122 FJK97864
About Ecorys
Ecorys is a leading international research and consultancy company, addressing society's key
challenges. With world-class research-based consultancy, we help public and private clients make
and implement informed decisions leading to positive impact on society. We support our clients with
sound analysis and inspiring ideas, practical solutions and delivery of projects for complex market,
policy and management issues.
In 1929, businessmen from what is now Erasmus University Rotterdam founded the Netherlands
Economic Institute (NEI). Its goal was to bridge the opposing worlds of economic research and
business – in 2000, this much respected Institute became Ecorys.
Throughout the years, Ecorys expanded across the globe, with offices in Europe, Africa, the Middle
East and Asia. Our staff originates from many different cultural backgrounds and areas of expertise
because we believe in the power that different perspectives bring to our organisation and our
clients.
We value our independence, our integrity and our partners. We care about the environment in
which we work and live. We have an active Corporate Social Responsibility policy, which aims to
create shared value that benefits society and business. We are ISO 14001 certified, supported by
all our staff.
Evaluation study of definitions, gaps and costs of response capacities for the Union Civil Protection Mechanism 123
P.O. Box 4175
3006 AD Rotterdam
The Netherlands
Watermanweg 44
3067 GG Rotterdam
The Netherlands
W www.ecorys.nl
BELGIUM – BULGARIA – CROATIA – INDIA – THE NETHERLANDS – POLAND – SPAIN – TURKEY – UNITED KINGDOM