You are on page 1of 60

PWP 6-461G1 DOE MARTIE Impact to Energy

Sector and MARITE SRS


Donovan Truitt
Dwight Beaver
Brad Runyon

March 2018

SPECIAL REPORT
CMU/SEI-2018-SR-019

CERT Division
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited
distribution. Please see Copyright notice for non-US Government use and distribution.

http://www.sei.cmu.edu

REV-03.18.2016.0
Copyright 2018 Carnegie Mellon University. All Rights Reserved.

This material is based upon work funded and supported by the Department of Energy under Contract No.
FA8702-15-D-0002 with Carnegie Mellon University for the operation of the Software Engineering Institute, a
federally funded research and development center sponsored by the United States Department of Defense.

The view, opinions, and/or findings contained in this material are those of the author(s) and should not be
construed as an official Government position, policy, or decision, unless designated by other documentation.

This report was prepared for the SEI Administrative Agent AFLCMC/AZS 5 Eglin Street Hanscom AFB, MA
01731-2100

NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE


MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO
WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT
NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR
RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT
MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK,
OR COPYRIGHT INFRINGEMENT.

[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited
distribution. Please see Copyright notice for non-US Government use and distribution.

Internal use:* Permission to reproduce this material and to prepare derivative works from this material for
internal use is granted, provided the copyright and “No Warranty” statements are included with all
reproductions and derivative works.

External use:* This material may be reproduced in its entirety, without modification, and freely distributed in
written or electronic form without requesting formal permission. Permission is required for any other external
and/or commercial use. Requests for permission should be directed to the Software Engineering Institute at
permission@sei.cmu.edu.

* These restrictions do not apply to U.S. government entities.

DM18-0405

CMU/SEI-2018-SR-019
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Sections of the Document

Section 1 Malware Analysis Repository and Information Exchange – Impact to Energy Sector .. 3

Section 2 Virtual Forensics Exchange System Requirement Specification .................................. 16

CMU/SEI-2018-SR-019 i
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
SECTION 1
MALWARE ANALYSIS REPOSITORY AND
INFORMATION EXCHANGE - IMPACT TO
ENERGY SECTOR

CMU/SEI-2018-SR-019 1
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Executive Summary

MARTIE, the Malware Analysis Repository and Threat Information Exchange, is the SEI’s
prototype implementation of a Virtual Forensics Exchange (VFE) platform that automates critical
steps in recognizing and responding to cybersecurity threats to the national electricity
infrastructure. This document discusses the rationale, approach, and capabilities of the current
system.

CMU/SEI-2018-SR-019 2
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Table of Contents

Executive Summary 2

Introduction 4

1 The Threat to the Sector 6

1.1 Targeted Malware 6

1.2 Commodity Malware 7

2 Stakeholders 8

2.1 Asset Owner/Operator Community 8

2.2 DOE and FERC 8

2.3 NERC 8

2.4 E-ISAC 9

2.5 The SEI 9

3 The Current State 10

4 A Possible Future State 12

4.1 E-ISAC 12

4.2 Asset Owners and Operators 13

4.3 DOE 13

4.4 The SEI 13

4.5 Research Organizations 13

4.6 Equipment Vendors 13

5 Next Steps 14

6 Requirements 15

6.1 Collect 15

6.2 Triage 15

6.3 Analysis 15

6.4 Archive and Consensus 15

CMU/SEI-2018-SR-019 3
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Introduction

The following factors underlie the need for the capabilities of the VFE/MARTIE system.

1. The Energy sector owns and operates a large number of systems that are critical
infrastructure or are connected (e.g., through corporate networks) to critical infrastructure.
2. The Energy sector is not homogenous, complicating security efforts in significant ways.
a. Assets are owned and operated by thousands of independent organizations. Their
personnel vary considerably in security expertise and time allocated to security
responsibilities. Some organizations simply lack the resources to be as effective as they
would like.
b. Supply chains include software and equipment from dozens if not hundreds of vendors.
The age of the elements within these supply chains likewise varies, and there is a
considerable amount of equipment in the field that was not designed with security in
mind.
3. Securing systems in the Energy sector requires ongoing vigilance against evolving,
significant threats.
4. Today, the community does not commonly share potential cyber artifacts or conduct trend
analyses. This results in inefficient use of already stretched resources (e.g., rediscovery of
known malware) and missed opportunities to react more quickly to new threats.

VFE/MARTIE is a prototype system designed to help Energy sector operators work together more
efficiently to improve their individual and collective security postures. The core capabilities
provided by the system are automated surface analysis and information sharing.

The automated surface analysis capability processes suspicious cyber artifacts to extract indicators
of compromise. It also collects metadata that can be combined with other data sources to improve
situational awareness and analyze trends both across the community and over time.

VFE/MARTIE also provides information sharing capabilities for the community, including
mechanisms allowing
• community members to upload cyber artifacts for analysis by the E-ISAC
• the E-ISAC to communicate analysis results back to the provider
• the E-ISAC to share cyber intelligence with the broader community

Taken together, these capabilities provide an opportunity for significant improvements, such as
• significantly increasing the number of artifacts being analyzed by simplifying the upload of
cyber artifacts and providing a commitment to rapid turnaround of results that automated
analysis enables
• improving the reliability of cyber analyses by relying on expert knowledge embedded in
tools instead of relying on potentially inconsistent analyst training

CMU/SEI-2018-SR-019 4
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
• improving and speeding acquisition of situational awareness and rollout of appropriate
responses by exploiting the analysis results across a growing number of community members
sharing cyber artifacts

VFE/MARTIE allows these benefits to accrue in a scalable way. Community members can upload
cyber artifacts with minimal effort, removing a barrier to participation while freeing local
resources that may be devoted to enhanced prevention and detection efforts. The E-ISAC can
scale analysis to a larger number of artifacts by shifting the analysis burden from analysts to
VFE/MARTIE. VFE/MARTIE can also be extended with new analyses over time, increasing the
number of types of attacks that can be detected. The community can shift their training from
instilling deep expertise in analysts to knowing what artifacts to upload and how to interpret
results.

VFE/MARTIE is a necessary, but not sufficient, element of achieving these benefits. To be


successful, the Energy sector will have to use the system and share more cyber artifacts than are
being shared today. Fortunately, there exists the possibility to kick off a virtuous cycle. As
community members upload artifacts, they will get reliable results quickly, improving confidence
in the analyses. As more community members upload files, the E-ISAC will have more data to
work with, and the VFE will provide more useful situational awareness. As this happens, the E-
ISAC will be able to provide more valuable insights to the community, further encouraging them
to upload artifacts.

CMU/SEI-2018-SR-019 5
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
1 The Threat to the Sector

The Energy sector relies on industrial control systems (ICS) and digital networks to maintain
efficient, continuous operations, much like other critical infrastructure sectors (CIKR). However,
given the dependence of other CIKR and non-CIKR sectors on Energy, the results of a
coordinated cyber-attack could have extreme and lasting effects on the rest of the country. VFE
should be used to minimize this risk by allowing analysts to share important information, identify
potentially malicious files, and aid in network defense missions across the sector.

Network defenders use various technologies to stop or prevent cyber-attacks. Most of these
technologies focus on identifying or blocking network indicators of compromise (IoCs) such as
domain names, IP addresses, or URLs. Operationalizing IoCs from outside the organizations by
way of alerts, bulletins, or indicator feeds can be challenging because these are slow, in the
timescale of attacks, and are not in a format that can be automatically integrated into defense.
Further, for organizations that produce these operational reports, it is increasingly difficult to both
identify host-based or network IoCs of an intrusion and provide early warning support for
network defenders. Personnel in IT organizations need a more efficient way to prioritize threats
and to identify potentially malicious files.

The Energy sector is susceptible to two types of malware: targeted and commodity. The VFE
infrastructure supports analysis of both types. Longitudinal analysis, or tracking particular
malware families over time, allows analysts to uncover previously unknown files during a
compromise. VFE provides longitudinal analysis results, a capability that enables proactive
defense by helping network defenders understand the techniques, tactics, and procedures (TTPs)
of an adversary. VFE also helps more network IoCs share with other organizations. One
particularly advantageous use of VFE is to share the results of reverse engineering, an effort that
requires significant expertise, time, and resources. This could be a boon to resource-strapped
organizations.

1.1 Targeted Malware


The Energy sector could be the target of sophisticated malware meant to collect and exfiltrate
data. Malware could also cause major interruptions or outages due to the significant risks the
sector poses to the country if compromised. These types of malware families are pervasive and
not easily detected by many common security technologies. Adversaries nearly always restructure
and redeploy their tools rather than build new ones from scratch for each target. Sophisticated
adversaries have targeted nations in the recent past, which makes targeted malware concerning for
the United States. We assess that adversaries will continue to target the Energy sector with
targeted malware in the long and short term. The malware families discussed below are examples
that could have been pushed through VFE. Efficient information sharing is critical to prevent
more organizations from being compromised by targeted malware.

• Havex: Havex is a remote access Trojan (RAT) used by sophisticated adversaries to


compromise the Energy sector. The malware family used its Open Platforms Communication
(OPC) module in targeted reconnaissance campaigns of ICS components. Based on compile

CMU/SEI-2018-SR-019 6
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
times, the malware was likely used from September 2011 to at least October 2014.
Adversaries used this tool primarily for intelligence gathering of specific details relating to
the sector. i
• BlackEnergy: While BlackEnergy did not start as an Energy- or ICS-specific tool, it quickly
gained notoriety in its third version, which specifically targeted ICS networks. This version
included several exploits tailored for ICS components, a self-destruction module, and other
anti-reverse engineering functions. BlackEnergy v3 provided an intelligence-collecting
component and modules that caused outages well into 2016. ii Most notably, BlackEnergy was
used to wipe and destroy serial-to-Ethernet devices. iii
• CRASHOVERRIDE: In December 2016, attackers used the CRASHOVERRIDE malware to
take down one-fifth of the total power in Ukraine with a simple test of the code. iv
CRASHOVERRIDE is the first malware framework to target energy-specific systems. Its
functionality is modular and uses an OPC module similar to Havex for intelligence gathering
and exfiltration. The adversaries using this malware could repurpose it for use against the
United States, which could cause severe impacts for days or weeks. v

1.2 Commodity Malware


The Energy sector faces additional exposure due to patch prevalence and adoption of ICS
technologies. In many cases, the technologies within the Energy sector are either end-of-life,
difficult to access, or mission critical, all of which make patching inconsistent or nonexistent. vi
The Energy sector is vulnerable to both targeted and non-targeted attacks, the latter of which
relies on the prevalence of old, seemingly innocuous vulnerabilities in Energy-sector ICS or
corporate networks.

Certain technologies are ubiquitous across all sectors and are easily exploitable at scale by
motivated adversaries. Crimeware and exploit kits favor out-of-date or non-patched systems,
which provide low-hanging fruit for attackers. Because the Energy sector relies on digital
networks to maintain day-to-day operations, these computers are equally as likely to be
compromised as those of any other sector. VFE should be used to support commodity malware
analysis to reduce the occurrence of duplicative analysis within the sector. This class of malware
should be prioritized below targeted malware; VFE can help an analyst make this determination
for unknown files.

• Crimeware kits: Criminal actors have a much lower barrier for entry due to the vast number
of exploit kits and malicious tools for purchase. Because the attack surface is broad for these
tools, the enterprise networks of Energy companies could also be compromised. Crimeware
kits provides criminals both with a way into the network to move laterally and, with
motivation, to compromise the availability of the organizations. vii
• Ransomware: Several global energy providers were the target of the Petya ransomware
campaign in late 2017. Energy organizations in Denmark, Russia, and Ukraine were offline
for several hours as incident response teams remedied the infections. Based on conversations
with the E-ISAC, ransomware has been a problem for energy providers in the United States.

CMU/SEI-2018-SR-019 7
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
2 Stakeholders

The stakeholders who could benefit from VFE/MARTIE include the owners and operators of the
electric grid, DOE and FERC, NERC, the E-ISAC, FFRDCs, and equipment vendors.

2.1 Asset Owner/Operator Community


The asset owner/operator community comprises the owners and operators of the various physical
assets that generate, transmit, and distribute electricity in the United States. Due to the grid
interconnections at the northern and southern borders, the community also encompasses Canada
and Mexico. These include investor-owned electric utilities, electric cooperatives, and municipal
electric companies.

Stakeholders of this class of benefit from VFE/MARTIE by reducing the need to provide their
own expertise for all aspects of malware detection, analysis, and remediation. However, realizing
this benefit requires a corresponding willingness to share the files and other data that are needed
to populate VFE/MARTIE.

2.2 DOE and FERC


The United States Department of Energy (DOE) is a Cabinet-level department of the Executive
Branch charged with various aspects of energy production and consumption in the country. The
Federal Energy Regulatory Commission (FERC) is an independent federal agency that regulates
the interstate commerce aspects of electricity, gas, and oil in the United States. The DOE and
FERC have the ongoing responsibility of ensuring the security and survivability of the national
electrical grid.

As the sponsor of VFE/MARTIE development, the DOE intends to provide a capability to


measurably improve the community’s security posture in terms of the number of artifacts
analyzed and the number and quality of actionable warnings provided to the community, without
putting a heavy burden on the community.

2.3 NERC
The North American Electric Reliability Corporation (NERC) is the not-for-profit international
regulatory authority whose mission is to assure the reliability and security of bulk power systems
in North America. Its key programs are based on four pillars of continued success: reliability
(addressing events and identifiable risk); assurance to the public, industry, and government for
reliable performance; learning to provide continuous improvement for improved reliability; and a
risk-based approach to focus attention on the most important issues affecting bulk power system
reliability. As part of its mission, NERC created the E-ISAC.

CMU/SEI-2018-SR-019 8
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
2.4 E-ISAC
The Electricity Information Sharing and Analysis Center (E-ISAC) is operated by NERC and is a
member of the National Council of ISACs. E-ISAC establishes situational awareness, incident
management, and communication capabilities within the electricity sector.

E-ISAC is the future operator of the VFE/MARTIE system. The system is expected to enable E-
ISAC to handle a much greater number of threats, as well as a far more diverse set, by allowing E-
ISAC to shift its resources to more automated aggregation and analysis, providing richer, more
actionable feedback to its members in the owner/operator community.

2.5 The SEI


The Software Engineering Institute (SEI) is a college-level unit of Carnegie Mellon University,
operating as a Federally Funded Research and Development Center (FFRDC) under contract with
the United States Department of Defense. The SEI, through its CERT Division, is the developer
of the current prototype VFE, also known as MARTIE.

CMU/SEI-2018-SR-019 9
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
3 The Current State

The U.S. Energy sector shares a fundamental goal across many varied government and private
stakeholders: the protection of its critical physical and, increasingly, cyber-physical infrastructure.
To achieve this goal, the various stakeholders undertake various sub-goals relating to their
particular missions and objectives, such as the following:

• The community should develop and adopt standards that are secure.
• Vendors should supply products secure against known threats and proactively patch issues as
new threats emerge.
• Asset owners should deploy equipment and networks in secure configurations and keep those
elements updated.
• Asset operators should monitor their environments—for example, network traffic, enterprise
communications, and potential patches—for any signs of suspicious activity.
• Any suspicious activity should be analyzed, and appropriate responses taken, for malicious
and potentially malicious activities.
• The community should share its knowledge regarding suspicious and malicious activities in
order to improve situational awareness, improve responses to such activities, and make future
plans for better securing the critical infrastructure.

The SEI designed the VFE specifically to support the last two items. The mission of the E-ISAC
specifically addresses the last item. Through conversations with the E-ISAC during the
development of VFE/MARTIE, the SEI learned about the current state of cybersecurity
information sharing within the community. E-ISAC members currently have the ability to submit
information to the E-ISAC for manual analysis, but relatively few members do so. One estimate is
that fewer than 20 members submit suspicious artifacts to the E-ISAC. This suggests tremendous
room for improvement.

Consider the following possible explanations for why so few members submit suspicious artifacts
for E-ISAC analysis:

1. There are not very many suspicious artifacts to report because the Energy sector is not a
target for malicious actors.
2. There are more suspicious artifacts in the Energy sector, but asset owners have all the
expertise and resources that they need to perform in-house analyses.
3. There are more suspicious artifacts in the Energy sector, but asset owners do not have the
expertise to reliably determine what is and is not suspicious.
4. There are more suspicious artifacts in the Energy sector, but asset owners do not have the
resources to examine them.

Scenario (1) seems improbable; see the section on Threats for counter-examples. Scenario (2)
seems unlikely, at least in the general case, given the range of asset owners and operators in terms
of size, resources, staff, and expertise. Even if true, however, this scenario points to tremendous
inefficiencies in the sector. Thousands of owners and operators would be spending resources to
CMU/SEI-2018-SR-019 10
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
acquire or train their own leading-edge experts. They would also be performing duplicative
analyses, given the likelihood that multiple owners and operators would be subject to the same
kinds of malicious activities. Some combination of scenarios (3) and (4) seems closest to reality.

The VFE/MARTIE system can help with, though not completely solve, both scenarios (3) and (4).
It helps with scenario (3) by reducing uncertainty. As the cost of analysis decreases with
automation and resource-sharing, it becomes more acceptable to submit something that is
probably benign, which can then change the bias from only submitting something probably
malicious to always submitting the merely suspicious. This can lead to asset owners requiring less
skilled resources because making a decision on submission would require less expert
discrimination. VFE/MARTIE most obviously helps with scenario (4) by encouraging asset
owners and operators—especially the smaller ones—to offload a resource- and expertise-intensive
task, the analysis of the suspected threat, to a trusted and shared resource.

Through the deployment of VFE/MARTIE, the Energy sector should see significant, measurable
improvements in measures such as

• the number of suspicious artifacts submitted


• the number of members submitting suspicious artifacts
• the number of suspicious artifacts analyzed and determined to be malicious
• the accuracy of the analyses (e.g., reduction of false positives as analyses are improved and
more analyses are added)
• the quality of the situational awareness the E-ISAC is able to generate and disseminate to
members
• reductions in the amount of time and other resources individual owners and operators have to
spend on malware analysis

CMU/SEI-2018-SR-019 11
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
4 A Possible Future State

The VFE/MARTIE system was designed to improve the cybersecurity posture across the Energy
sector. To achieve this vision, several classes of stakeholders have important roles at various
stages of deployment.

• E-ISAC, to
− deploy and begin operating the VFE
− improve and disseminate situational awareness as results accumulate
• assets owners and operators, to contribute suspicious files and deal with any malicious
findings
• asset owners, operators, and equipment vendors, to take proactive steps based on these results

Each stakeholder can take many smaller steps to enable or improve the effectiveness of the high-
level steps above. We break these suggestions down by stakeholder below. The bulk of the next
steps fall upon the E-ISAC to get the VFE operational and to entice the larger community to use it
much more aggressively than current data-sharing efforts.

Many of these suggested steps are part of a near-term kickoff effort. At this phase, it is important
to ensure the community is aware of the existence and benefits of VFE/MARTIE and to quickly
identify and remove as many barriers as possible. This is intended to achieve a critical mass of
users and usage on the system such that its benefits are clear and apparent and its viability is
established. Longer term activities are oriented toward ongoing operations, including
improvements and enhancements. Because vigilance is needed to keep up with evolving threats,
VFE/MARTIE must anticipate such threats and be kept current to be effective. This will require
some form of sustainment, which will be best justified by ongoing, measurable benefits to the
sector.

4.1 E-ISAC
• Deploy the VFE/MARTIE system, confirm correct operations, and make the system available
to members for their submissions.
• Conduct an awareness campaign to inform members of the VFE/MARTIE system’s existence
and capabilities, the benefits to using it, and how to do so.
• Interact with members to explore any barriers to use. Collect information on the range and
frequency of barriers and move to remove or address each one. For example, observed
barriers might include
− failures of knowledge or awareness, such as
 submitting all activities, not just suspicious ones
 how to submit suspicious artifacts
 how to tell which artifacts are suspicious and should be submitted
− failures in resourcing (time/staff) to do this kind of work
− fears of the ramifications of information disclosure (e.g., business, legal, or regulatory)
CMU/SEI-2018-SR-019 12
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
• Collect metrics on the use of VFE/MARTIE and its impact on the sector. Compare this to
baseline metrics and how they change over time. Measure the effectiveness of different
awareness and outreach efforts.
• Gather success stories (probably anonymized) that can be shared with the community to
reinforce the benefits of VFE/MARTIE and encourage additional participation.
• Collect data on what kinds of malicious activity cannot be detected by VFE/MARTIE and
how frequently these are encountered in the sector.

4.2 Asset Owners and Operators


• Submit all suspicious files!
• Provide E-ISAC with feedback on what is working well, what can be improved, and what
new capabilities would be most valuable.
• Share positive experiences with the community to help promote participation and secure
ongoing resources.

4.3 DOE
• Monitor adoption of the VFE/MARTIE capabilities, for example through metrics that the E-
ISAC collects directly from asset owners and operators.
• Monitor positive impacts, for example in detections of malicious activity, increased number
of organizations sharing artifacts, and reports on reduced burdens on asset owners and
operators.
• Work with the E-ISAC and asset owners and operators to remove barriers to use, as
appropriate.
• Support extensions to the VFE/MARTIE system as warranted. Examples include the
automation of additional analyses, integration with other analysis tools, improved reporting
and metric collection capabilities, improved maintenance functions, or improved usability if
such issues are reported.

4.4 The SEI


• Help the E-ISAC get the VFE/MARTIE prototype deployed and operational. Ensure the E-
ISAC can move quickly to operation without reliance on the SEI.
• Help the E-ISAC with community awareness activities, such as by co-presenting at a sector
event or supporting a webinar for members on VFE/MARTIE capabilities.

4.5 Research Organizations


• Propose and prototype extensions that expand the capabilities of VFE/MARTIE.

4.6 Equipment Vendors


• TBD

CMU/SEI-2018-SR-019 13
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
5 Next Steps

• E-ISAC
− Deploy VFE.
− Start awareness campaign to increase upload numbers.
− Explore and remove barriers.
− Enable comparison of a baseline by using curated reporting metrics.
− Gather success stories (anonymized) to promote successes for members and to
encourage others to participate.
• DOE
− Monitor adoption.
− Monitor success (compromises, community advisories, etc.).
− Support extension of analyses when warranted.
• The SEI
− Smoothly transition VFE to E-ISAC.
− Help with community awareness (e.g., co-present).
− Recommend candidate analyses or features for the future.

CMU/SEI-2018-SR-019 14
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
6 Requirements

The following are the current high-level functional requirements for the VFE system under
development. These requirements along with architectural and design information are covered in
the related document, VFE-1.0 Software Requirements Specification.

6.1 Collect
1. The system shall accept malware samples from users through a web page.
5. The system shall accept malware samples programmatically through a RESTful API.

6.2 Triage
6. The system shall be hosted within the E-ISAC web portal.
7. The system shall be generic enough to be deployed in multiple locations.
8. Malware files should be downloadable by approved users through a RESTful API.

6.3 Analysis
9. The system shall execute automated analytics on uploaded malware samples.
10. The system shall have asynchronous malware analytics.
11. The system shall output malware analytics results to a database.
12. Analytic results should be available to users by the web page.
13. Analytic results should be queriable by users through a RESTful API.

6.4 Archive and Consensus


14. The system shall notify users when analytics are complete.
15. Analytic results should be queriable by users through a web browser.
16. The system shall allow sharing of analytic output by a user-determined method.
17. The system shall authenticate and authorize users with E-ISAC’s directory service.
18. E-ISAC will have full access to results; members can delineate the extent of their sharing
with other community members or externally, outside the community.
i
https://www.scmagazine.com/havex-malware-strikes-industrial-sector-via-watering-hole-
attacks/article/538721/
ii
https://ics.sans.org/media/E-ISAC_SANS_Ukraine_DUC_5.pdf
iii
https://dragos.com/blog/crashoverride/CrashOverride-01.pdf
iv
https://www.wired.com/story/crash-override-malware/
v
https://dragos.com/blog/crashoverride/CrashOverride-01.pdf
vi
https://ics-cert.us-cert.gov/Abstract-Patch-Management-ICS-RP
vii
https://www.darkreading.com/attacks-breaches/ransomware-scada-access-as-a-service-emerging-
threats-for-ics-operators-report-says/d/d-id/1325952?

CMU/SEI-2018-SR-019 15
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
SECTION 2
MALWARE ANALYSIS REPOSITORY AND
INFORMATION EXCHANGE - SOFTWARE
REQUIREMENTS SPECIFICATION

CMU/SEI-2018-SR-019 16
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Executive Summary

The purpose of the MARTIE is to automate surface analysis of cyber artifacts uploaded by ISAC
community members. Automating surface analysis will allow the ISAC to rapidly gain situational
awareness into ongoing cyber-attacks on members and thus accelerate incident response efforts
across the community. In addition to increasing situational awareness, the system will act as a
knowledge management platform, allowing information gained from prior incidents to be applied
to all future incidents. This system will also enable automated peer-to-peer communication of
intelligence and indicators of compromise.

The MARTIE’s initial capabilities are the following:


1. surface analysis of potentially malicious files uploaded by ISAC community members
2. pivoting on atomic indicators of compromise
3. sharing of specific indicators and intelligence

The MARTIE consists of the following major components:


1. Portal
2. Job Controller
3. Queuing System
4. Raw File Storage
5. Static and Dynamic Malware Analysis System
6. Analysis Output Storage

The MARTIE’s architecture includes the following:


• The architecture of each component is designed to be modular and scalable. This means
that the capacity of each component of the system can be increased independently to meet
user demand.
• The system is designed to support multiple independent and asynchronous backend
analytics systems. While the File Scanning Framework (FSF) is the initial backend
analytic system, others can be added to the system to further enhance capabilities without
affecting operations.
• The system will support two modes for user interaction. The first is through a standard
web portal. The second is programmable through a RESTful API. Both modes of
interaction will support and use the same feature set.
• The system will be hosted by the ISAC in commercial Azure.
• The ISAC portal will provide the primary means for members to access MARTIE with
seamless access to capabilities.
• The system will use the ISAC’s directory services to authorize and authenticate users.

CMU/SEI-2018-SR-019 17
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Table of Contents

Executive Summary 17

Goal 19

Value Proposition 19

Purpose 20

Scope 21

References 23

Definitions and Acronyms 24

System Requirements 25

High-Level Design 26

Revision History 48

Appendix 49

MARTIE Administration 49

(v2) E-ISAC MARTIE Deployment Instructions 51

E-ISAC MARTIE Migration Instructions (v1-v2) 54

User Guide – Within FAQ 56

Troubleshooting – Within FAQ 56

CMU/SEI-2018-SR-019 18
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Goal

The goal of the Malware Analysis Repository and Information Exchange (MARTIE) System
detailed in this document is to allow the ISAC1 to rapidly gain situational awareness into
malicious software cyber-attacks targeting its community members. Rapidly gaining situational
awareness insights will allow the ISAC to better triage events, coordinate incident response
efforts, allocate resources, and disseminate important information in a timely manner. The
MARTIE system will accomplish this goal by automating surface analysis of attack artifacts and
disseminating results.

Value Proposition
Automating surface analysis will benefit the ISAC community in the following ways:

• Response: Improve the quality and timeliness of analysis of both generic malware and more
targeted incidents affecting multiple community members.
• Knowledge Management: Seamlessly allow knowledge gained from one incident to be
applied to all future incidents within a protected community while continuing to improve
capabilities.
• Knowledge Sharing: Reduce the costs and improve the quality of sharing attack information
with community members.
• Situational Awareness: Gain a better understanding of how attackers are specifically targeting
community members.

CMU/SEI-2018-SR-019 19
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Purpose

This purpose of this document is to translate the system requirements from the MARTIE Business
Requirements Document (BRD) into a verifiable specification that will be used to design,
prototype, build, test, and approve this system.

CMU/SEI-2018-SR-019 20
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Scope

This document applies to the MARTIE developed for the DOE under the 6-461G1 DOE PWP.
This section describes a horizontally scalable system designed to automate surface analysis of
potentially malicious cyber artifacts within uploaded files.

MARTIE will use commercially available, sustainable components and solutions, as opposed to
custom-developed alternatives, to the extent feasible.

High-level capabilities
1. surface analysis of potentially malicious cyber artifacts
2. querying and pivoting based on surface analysis results
3. user-controlled sharing of analysis results

Major system components


1. Portal
2. Job Controller
3. Queuing System
4. Raw File Storage
5. Static and Dynamic Malware Analysis System
6. Analysis Output Storage

Capabilities
Surface analysis
Uploaded files will be processed through analytics that produce metadata about those
files. These results will be stored in a database that allows querying, pivoting, and
sharing. Surface analysis results could contain but are not limited to the following
information:
• indicators of compromise (IOCs)
• antivirus hits
• hashes, strings, IP addresses, domain names, and position-independent code
signatures

Pivoting
Pivoting allows an analyst to take information gleaned from examining one file and then
enrich it with other data to gain better situational awareness. Examples of pivoting and
enrichment include
• finding groups of files that contain the same characteristics
• determining if other community members have seen files with similar
characteristics
• grouping files into families based on similar characteristics
• supplementing data using multiple datasets
CMU/SEI-2018-SR-019 21
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Sharing
As new analysis results are generated, this information needs to be disseminated to
community members so they can take appropriate actions. This system will support
sharing by
• allowing users to share specific indicators of compromise with the ISAC,
community members, and the government
• allowing community members to share gathered intelligence

CMU/SEI-2018-SR-019 22
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
References

Internal References
PWP 6-6461G1

MARTIE BDR

External References
Reference Content Management System
http://www.ingeniux.com/resources?type=&category=38

CMU/SEI-2018-SR-019 23
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Definitions and Acronyms

Term or Acronym Definition

ISAC Information Sharing and Analysis Center

DOE Department of Energy

AMD Asynchronous, Modular, and Distributable

CMS Content Management System

Malware software that is intended to damage or disable computers and computer


systems

RESTful uses HTTP requests

RESTful API an application program interface that uses HTTP requests to GET, PUT,
POST, and DELETE data

NERC North American Electric Reliability Corporation

SEI Carnegie Mellon University Software Engineering Institute

MARTIE Virtual Forensics Exchange

SLA Service Level Agreement

Azure Microsoft’s Cloud Solution for E3 licenses

FERC Federal Energy Regulatory Commission

SSO Single Sign On

Threat Connect Malware Analysis Software Tool

OT Operational Technology

JSON JavaScript Object Notation

FSF File Scanning Framework

Atomic Cannot be broken down further

CMU/SEI-2018-SR-019 24
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
System Requirements

Collect
The system shall accept malware samples from users through a web page

The system shall accept malware samples programmatically through a restful API

Triage
The system shall be hosted within the ISAC web portal

The system shall be generic enough to be deployed in multiple locations

Malware files should be downloadable by approved users through a RESTful API

Analyze
The system shall execute automated analytics on uploaded malware samples

The system shall have asynchronous malware analytics

The system shall output malware analytics results to a database

Analytic results should be available to users by the web page

Analytic results should be queriable by users through a RESTful API

Archive and Consensus


The system shall notify users when analytics are complete

Analytic results should be queriable by users through a web browser

The system shall allow sharing of analytic output by a user-determined method

The system shall authenticate and authorize users with the ISAC’s directory service

The ISAC will have full access to results; members can delineate the extent of their
sharing with other community members or externally, outside the community

CMU/SEI-2018-SR-019 25
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
High-Level Design

CMU/SEI-2018-SR-019 26
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Deployment and Operations
SecurityRequirements – the system shall use security requirements provided by the
ISAC to run within its environment

security requirements will be provided by the ISAC

SLARequirements – the system shall be available during core business hours

the system shall be available to registered members of the ISAC

the system shall provide three tiers of support

Level 1 – ISAC

Level 2 – ISAC

Level 3 – Vendor/Provider

SystemOperations – the System shall be operated by a team at the ISAC familiar


with MS Azure and the Linux operating system

SystemDeployment – system deployment shall be automated

CMU/SEI-2018-SR-019 27
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Portal
The purpose of the Portal is to provide an interface between the end user and the system.
Users will be able to access the Portal through a graphical web interface and through a
RESTful API. The Portal will be responsible for authenticating and authorizing user access to
the system.

This system will be constructed in such a way as to allow it to scale according to user
demand.

Physical Attributes
BasePlatform – Microsoft Azure

The system shall use MS Azure licensed by the ISAC.

Commercial license of Azure is acceptable.

The basic system platform shall consist of the following components:

Operating System: Red Hat Linux

CMU/SEI-2018-SR-019 28
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Application Programming Language: Python

Application Framework: Django and Flask

Queue: Redis on Red Hat

Metadata store: MongoDB

Application Services
VolumeAndBandwidth – data volume and bandwidth shall be expected to be low

expected uploaded file volume shall be 100 to 1000 items per day

expected daily queries shall be 1000 per day

the high-level design shall be based on acquired information on data volume and
bandwidth estimates

QueriesSupport – servers shall support 10 queries per second

UploadSupport – servers shall support 10 concurrent uploads

MaxFileSize – the maximum file size for a user uploaded file shall be 10MB

QueryResults – servers shall support caching query results

ApplicationScaling – the application architecture shall be modular and support


horizontal scaling. Specifically:

the application architecture shall support scaling load balancers if necessary

the application architecture shall support scaling web servers if necessary

the application architecture shall support scaling the application servers if


necessary

ApplicationQueryBasics – the application shall support the following queries:

keyword searches

CMU/SEI-2018-SR-019 29
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
CIDR (classless inter-domain routing) for IP searches

IP ranges for searches

basic Boolean searches

wildcard search terms

searches shall be on metadata

metadata shall include user and timestamp

Reporting – the application shall provide reporting functionality to users:

reports shall be available in human-readable formats

reports shall be available in machine-readable formats

reports shall be exportable in PDF, XML, JSON

UserDelete – users shall be able to delete a submission

if a user deletes a submission that was also submitted by another user, the
submission shall not be deleted in the system

Sharing – the end user shall determine how other users can access the end user’s
data

users shall be given three options for sharing:

option 1 shall not share with any ISAC members

option 2 shall share with all ISAC members

option 3 shall share with all ISAC members and the government

with options 2 and 3 users shall have another option to share without attribution

ISAC staff shall be able to access all data uploaded to the system

CMU/SEI-2018-SR-019 30
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Authentication and Authorization Module
UserSessions – the application shall handle user sessions

ValidCertificates – the application shall use valid certificates

the application shall use certificates provided by the ISAC

TokenAndSecret – the application shall support token and secret authentication

the application shall support authentication using Microsoft Active Directory

the application may support OAUTH

Authenticating and Authorizing Users – the application shall authenticate and


authorize users using the operator’s directory servers

the application shall use the ISAC’s directory for authenticating and authorizing
users

the vetting of users shall be done by the ISAC

the application shall support authentication and authorization using usernames


and passwords

the ISAC directory shall allow the application to distinguish between ISAC
community members, ISAC staff, and government users

CMU/SEI-2018-SR-019 31
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Job Controller
The purpose of the Job Controller is to process, route, and track user requests. The Job
Controller will create jobs for the analysis system, process results from the analysis system,
store raw files, and update and query the analysis output.

Processing Servers
Application Programming Interface – the application will support a RESTful API
for processing data

the application shall process queries by the user

the application shall allow users to upload potentially malicious files to the
system

the application shall allow users to query for the status of their jobs

the application shall allow users to control access to their samples and results

the application shall allow users to delete data they uploaded

FileUpload – the application shall allow users to upload files

the application will store the uploaded files in the raw file storage

the application will store metadata about the files in the analysis output storage

the application will include a source for each uploaded file

file uploads shall be scalable based on user demand

CMU/SEI-2018-SR-019 32
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
QueryOutputStorage – the application shall be able to query the analytic output
storage

the application shall not re-run analytics for duplicate uploaded files

the application shall return data from the analytic output storage for duplicate
files

JobCreateMalware – the application shall create jobs for the malware analysis
system

Processing Server Architecture – the processing server architecture shall be


modular and support scaling

the application shall be written in Python

CMU/SEI-2018-SR-019 33
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Queuing System
The purpose of the Queuing System is to temporarily store new jobs and job results for
processing by the Analysis System and the Job Controller.

Queuing Cluster
Queuing – the application shall use a queuing system where necessary

the queuing system shall be scalable

the queuing system shall initially be Redis

CMU/SEI-2018-SR-019 34
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Raw File Storage

Distributed File System


StorageSystem – shall use a reliable, redundant, and durable block storage system
storing uploaded raw files

Azure block storage shall be the storage system for raw files

Storage system shall scale as needed

Encryption – uploaded files shall not be encrypted when at rest

Backup – uploaded files shall be backed up by the operators of the system

DeleteAll – ISAC shall be able to delete uploaded files and all metadata

Storage Duration – uploaded files shall be kept as long as possible

CMU/SEI-2018-SR-019 35
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Access to uploaded files – access to uploaded files shall be controlled by the ISAC

users shall not have the ability to download uploaded files

ISAC shall have access to all uploaded files

ISAC shall be responsible for controlling access to uploaded files

CMU/SEI-2018-SR-019 36
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Static and Dynamic Malware Analysis System

Database Analytic Results Collection


JobAccept – the application shall accept jobs through the queuing system

JobStatusNotification – the application shall communicate job status to the end user

the application shall support email notifications of job statuses

the portal shall visually display the status of submitted jobs

job statuses shall be

In Progress

Completed

Failed

Levels of Failed

Upload Failed

Job Failed

IndependentAnalytics – analytics shall be independent of one another

the analytics system shall be designed to allow analytic backends to be added


and replaced without affecting the operations of the system

Initial Analytic Backend – File Scanning Framework (FSF) shall be the initial
analytic backend for the MARTIE
CMU/SEI-2018-SR-019 37
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
FSF (https://github.com/EmersonElectricCo/fsf) initial capabilities are below:

scan uploaded files against a series of YARA signatures

extract files from archives and process them (ZIP, RAR, SWF, UPZ, GZIP,
etc.)

collect basic meta information such as file hashes, sizes, types, etc.

collect executable file metadata

extract certificate information from PE files

collect java metadata

collect portable document file (PDF) metadata

collect legacy office document (OLECF) metadata

collect modern office document (OOXML) metadata

FSF is extendible through modules

malware configuration dumpers can be written for malware families and run
against files that match certain YARA signatures

FSF is scalable

supports multiple servers and multiple clients

supports load balancing

scales horizontally and vertically

FSF outputs JSON

JSON shall be stored in MongoDB

the application shall support exporting data to CSV and PDF

Example FSF Output:

1. {
2. "Scan Time": "2017-03-28 10:12:34.465426",
3. "Filename": "/home/cartman/malware/Backdoors/TidePool/14fbe3a1b0b5d40f66
946a58ab86f8e1/1.dll",
4. "Source": "Analyst",
5. "Object": {
6. "META_BASIC_INFO": {
7. "MD5": "6c970f3a6977f8b645b7aa6f8a010700",
8. "SHA1": "09d5ae700b99f7532c35c338731c486ed98087b1",
9. "SHA256": "b107831326c6b374f698ea3ad8b571fa72f4d50fb1e37f60703df
f1002933475",
10. "SHA512": "856585debdf58acb260ad8cf9e62d8af0d28c9ed44921c2fa8f09
e7b654a4060cd18088bc740dfcdc00f7f11734fa773c8c956f1126df6ebd6e1537bfd4210bd"
,
CMU/SEI-2018-SR-019 38
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
11. "ssdeep": "1536:h+8FoaHORnK/gTn/zZfUmMvhYLmj4Ysne6W:c8FosInKSzet
vmLmj4Yse6",
12. "Size": "72192 bytes"
13. },
14. "SCAN_YARA": {
15. "tidepool_jump_table": {
16. "Caveats": "Not for public dissemination. Do NOT upload thi
s signature to the Internet.",
17. "Author": "CERT/CC, Software Engineering Institute",
18. "KN_last_modified": "2016-012-13",
19. "KN_rat": "TidePool",
20. "KN_rat_confidence": "High"
21. },
22. "ft_strict_dll": "No Meta Provided",
23. "tidepool_c2_decode": {
24. "Caveats": "Not for public dissemination. Do NOT upload thi
s signature to the Internet.",
25. "Author": "CERT/CC, Software Engineering Institute",
26. "KN_last_modified": "2016-012-13",
27. "KN_rat": "TidePool",
28. "KN_rat_confidence": "High"
29. },
30. "ft_strict_exe": "No Meta Provided",
31. "ft_exe": {
32. "company": "Emerson",
33. "lastmod": "20141217",
34. "desc": "Simple signature to trigger on PE files.",
35. "author": "Jason Batchelor"
36. },
37. "ComputeJSHash": {
38. "Caveats": "Not for public dissemination. Do NOT upload thi
s signature to the Internet.",
39. "Author": "CERT/CC, Software Engineering Institute",
40. "KN_last_modified": "2016-012-13",
41. "KN_hashing_alg_confidence": "High",
42. "KN_hashing_alg": "JSHash"
43. }
44. },
45. "META_PE": {
46. "File Type": {
47. "EXE": "True",
48. "SYSTEM": "False",
49. "DLL": "True"
50. },
51. "CRC": [
52. "Claimed: 0x0",
53. "Actual: 0x1924d"
54. ],
55. "Compiled": "Wed Jun 8 00:55:49 2016 UTC",
56. "Architecture": "x86",
57. "EntryPoint": "0x4c7d",
58. "ImageBase": "0x10000000",
59. "Characteristics": {
60. "ASLR": "Disabled",
61. "DEP": "Disabled",
62. "WDM_DRIVER": "Disabled",
63. "FORCE_INTEGRITY": "Disabled",
64. "TERMINAL_SERVER_AWARE": "Disabled",
65. "NO_ISOLATION": "Disabled",
66. "SEH": "Enabled"
67. },
CMU/SEI-2018-SR-019 39
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
68. "Sections": [
69. ".text",
70. ".rdata",
71. ".data",
72. ".reloc"
73. ],
74. "Resource Names": "None",
75. "Resource Types": "None",
76. "Export Functions": [
77. "??0CIEComDll@@QAE@XZ",
78. "??4CIEComDll@@QAEAAV0@ABV0@@Z",
79. "?fnIEComDll@@YAHXZ",
80. "?nIEComDll@@3HA",
81. "DeleteMe",
82. "IEHelper"
83. ],
84. "Import DLLs": {
85. "KERNEL32": [
86. "DeleteFileW",
87. "CreateDirectoryW",
88. "WaitForSingleObject",
89. "GetDiskFreeSpaceExA",
90. "GetDriveTypeA",
91. "FindClose",
92. "FindNextFileW",
93. "FindFirstFileW",
94. "VirtualFree",
95. "GetProcAddress",
96. "LoadLibraryA",
97. "ExitProcess",
98. "CreateProcessW",
99. "lstrcmpiW",
100. "GetCommandLineW",
101. "lstrcatW",
102. "lstrlenW",
103. "VirtualAlloc",
104. "CreateThread",
105. "Sleep",
106. "FreeLibrary",
107. "GetLongPathNameW",
108. "CreateFileW",
109. "GetModuleHandleW",
110. "GetLastError",
111. "ResetEvent",
112. "SetErrorMode",
113. "lstrcpyW",
114. "lstrlenA",
115. "IsBadReadPtr",
116. "GetUserDefaultLCID",
117. "GetSystemDirectoryW",
118. "GetStartupInfoW",
119. "CreatePipe",
120. "PeekNamedPipe",
121. "GetComputerNameA",
122. "GlobalMemoryStatus",
123. "SetEnvironmentVariableA",
124. "CompareStringW",
125. "CompareStringA",
126. "GetFileSize",
127. "ReadFile",
128. "CloseHandle",
CMU/SEI-2018-SR-019 40
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
129. "MoveFileW",
130. "SetFilePointer",
131. "WriteFile",
132. "EnterCriticalSection",
133. "LeaveCriticalSection",
134. "SetEvent",
135. "InitializeCriticalSection",
136. "CreateEventA",
137. "WideCharToMultiByte",
138. "GetModuleFileNameW",
139. "MultiByteToWideChar",
140. "FlushFileBuffers",
141. "LCMapStringW",
142. "LCMapStringA",
143. "SetStdHandle",
144. "GetOEMCP",
145. "GetACP",
146. "GetCPInfo",
147. "InterlockedIncrement",
148. "InterlockedDecrement",
149. "GetStringTypeW",
150. "GetStringTypeA",
151. "IsBadCodePtr",
152. "SetUnhandledExceptionFilter",
153. "IsBadWritePtr",
154. "HeapReAlloc",
155. "HeapCreate",
156. "HeapDestroy",
157. "GetVersionExA",
158. "GetEnvironmentVariableA",
159. "GetModuleHandleA",
160. "GetEnvironmentStringsW",
161. "GetCurrentThreadId",
162. "TlsSetValue",
163. "TlsGetValue",
164. "ExitThread",
165. "RtlUnwind",
166. "GetTimeZoneInformation",
167. "GetSystemTime",
168. "GetLocalTime",
169. "GetCommandLineA",
170. "GetVersion",
171. "HeapAlloc",
172. "HeapFree",
173. "TlsAlloc",
174. "TlsFree",
175. "SetLastError",
176. "TerminateProcess",
177. "GetCurrentProcess",
178. "UnhandledExceptionFilter",
179. "SetHandleCount",
180. "GetStdHandle",
181. "GetFileType",
182. "GetStartupInfoA",
183. "DeleteCriticalSection",
184. "GetModuleFileNameA",
185. "FreeEnvironmentStringsA",
186. "FreeEnvironmentStringsW",
187. "GetEnvironmentStrings"
188. ],
189. "WS2_32": [
CMU/SEI-2018-SR-019 41
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
190. "WSACleanup",
191. "gethostbyname",
192. "WSAStartup"
193. ],
194. "OLEAUT32": [
195. "VariantClear",
196. "SysFreeString",
197. "VariantInit",
198. "SysAllocString",
199. "SafeArrayCreateVector",
200. "SafeArrayAccessData",
201. "SafeArrayUnaccessData"
202. ],
203. "OLE32": [
204. "CoCreateInstance",
205. "OleInitialize",
206. "OleUninitialize"
207. ],
208. "SHELL32": [
209. "SHFileOperationW",
210. "SHGetSpecialFolderPathW",
211. "ShellExecuteW"
212. ],
213. "USER32": [
214. "wsprintfA",
215. "wsprintfW"
216. ],
217. "ADVAPI32": [
218. "GetUserNameA",
219. "RegOpenKeyExA",
220. "RegDeleteValueW",
221. "RegQueryInfoKeyA",
222. "RegEnumKeyExW",
223. "RegQueryValueExW",
224. "RegDeleteKeyW",
225. "RegCreateKeyExW",
226. "RegOpenKeyExW",
227. "RegSetValueExW",
228. "RegSetValueExA",
229. "RegCloseKey",
230. "RegQueryValueExA"
231. ],
232. "WININET": [
233. "InternetSetOptionA"
234. ],
235. "NETAPI32": [
236. "Netbios"
237. ]
238. },
239. "Import Hash": "64718f7bb4dc62b7aacae89650102583",
240. "StringFileInfo": "None"
241. },
242. "META_TIDEPOOL": {
243. "C2": "hXXp[:]//shooterop.zzux.com",
244. "Class ID": "{2625E0BA-BF1E-3C23-D1B1-B640040AE720}"
245. }
246. },
247. "Summary": {
248. "Yara": [
249. "ComputeJSHash",
250. "ft_exe",
CMU/SEI-2018-SR-019 42
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
251. "ft_strict_dll",
252. "ft_strict_exe",
253. "tidepool_c2_decode",
254. "tidepool_jump_table"
255. ],
256. "Modules": [
257. "META_BASIC_INFO",
258. "META_PE",
259. "META_TIDEPOOL",
260. "SCAN_YARA"
261. ],
262. "Observations": []
263. },
264. "Alert": true
265. }

ResultsOutput – the application shall output analytic results to a database

results shall be stored in a document-based database

the document-based database shall support different fields for different files and
data

analytic output storage shall be MongoDB

MongoDB documentation: https://docs.mongodb.com/

CMU/SEI-2018-SR-019 43
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Analysis Output Storage

Metadata Storage

Metadata Retention – metadata shall never be purged

Backup – the operator of the system shall be responsible for backup up the metadata
storage

Engine – the application shall us an enterprise-grade distributed document database

the system shall use MongoDB for storing metadata

CMU/SEI-2018-SR-019 44
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Key – the application shall store metadata from uploaded files using a unique key

the unique key for storage shall be the sha256

the application shall also store md5 and sha1

the application shall support searching the metadata store by sha256, md5, or
sha1 hashes

Access – the end user shall determine how other users can access their data

See section 5.6.2 Application Services – Sharing

CMU/SEI-2018-SR-019 45
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
System Integration

AcceptAnalyticOutput – shall accept analytic output from the Static and Dynamic
Malware Analysis System

StoreAnalyticOutput – shall store analytic output based on malware hash

QueryData – the job controller shall query data

CMU/SEI-2018-SR-019 46
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Security and Policy
NDA
ISAC NDA is in effect.

Traffic Light
TLP – shall follow the Traffic Light Protocol defined by FIRST: https://www.first.org/tlp
• Red – only the ISAC
• Amber – all ISAC members
• Green – all members, ISAC, and the DOE, government partners
• White – do not expect the system to be publicly available and so this will not
be a selectable status

CMU/SEI-2018-SR-019 47
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Revision History

Rev Description of Change

1 Initial Release

2 Edits from 2/8/2017 SEI DC onsite

3 Edits from 3/31/2017 EISAC onsite

CMU/SEI-2018-SR-019 48
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Appendix

MARTIE Administration
1. Admin User
To login as administrator:
https://<martie-url>/admin
Username: docker@cert.org
Password: diidTartans@1
By default, docker@cert.org does not belong to any groups. You must manually go into the admin panel and add
groups to the docker@cert.org user.

2. Adding New Users


To add new users through the admin panel:
Log in as https://<martie-url>/admin
Before adding new users, you must create group classes and groups. A group class is a collection of 1 or more
groups. For example, FFRDC might be a group class that contains groups such as CERT and MITRE.
To add a user from the command line:

docker exec -it martie_web_1 python manage.py adduser [username] [password]


--first [firstname] --last [lastname] --email [email address] --group cert
--admin

The "adduser" custom command requires that the given group already exists. Groups must be created
through the web admin interface. You must use the --admin option in order for the user to be able to change his or
her password. This gives users permissions to the admin panel but only allows them to change the password. If a
user needs true admin privileges, this access must be given through the admin panel by a superuser.

3. Permissions
• Groups can be created through the admin panel (https://<martie-url>/admin)
• Only users that are given "staff" status are permitted to view the admin panel to change passwords.
• Before creating groups, the admin should create a “GroupClass”
• A GroupClass is a collection of Groups (e.g., FFRDC, GOV).
• When creating a group, the very last option is to select a GroupClass for this Group. (e.g., when creating the
CERT group - add to the FFRDC GroupClass).
• Each user should belong to at least one group, this can be selected when using the command line option or
through the admin panel.
• By default, a user's actions are shared with members of his/her Group.
• GroupClasses are used as an option for sharing permissions.
• A user has the option to share a file with a GroupClass or GroupClasses
• When a file is shared with a GroupClass - every user in that GroupClass will see an ALERT on their
home page that a file has been shared with their GroupClass.
• The ALERT will be colored RED until someone from that GroupClass downloads the file.
• A user can suppress the alert by clicking the (X) on the alert. Once the alert is clicked, the user
will not see the notification on the home page.
• All alerts (remediated or not) can be seen on the ACTIVITY page under “Recent Notifications.”
• It is acceptable if an admin prefers that a GroupClass have a one-to-one relationship with Groups, but the
GroupClasses should still be created (with the same name as the Group).

Other Viewing Permissions


• File TAGS can only be seen by users in the same group

CMU/SEI-2018-SR-019 49
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
• When a file is shared by a user, any comments and metadata is also shared to the GroupClass that the file is
shared with.
• If a file is shared with a user's group (default), then any user in the group has permission to share the file with
a GroupClass. A log of any sharing is available to everyone who can view the file.
• Once the file is shared to a GroupClass, any user in the GroupClass can share the file with another
GroupClass.

4. Login Lockout
A user is locked out after 5 failed logins (configurable in martie/web/malcert/settings.py:
AXES_LOGIN_FAILURE_LIMIT). The user's IP Address is locked out for 1 minute (configurable in settings.py:
AXES_COOLOFF_TIME). This means that the user cannot log in with the locked username for the cool off time. An
IP can be disabled from logging in with any username by setting
AXES_LOCK_OUT_COMBINATION_USER_AND_IP to False. Other configuration variables currently not set (but
that may be useful):
• AXES_NEVER_LOCKOUT_WHITELIST: If True, users can always login from whitelisted IP addresses. Default:
False
• AXES_IP_WHITELIST: A list of IP’s to be whitelisted. For example: AXES_IP_WHITELIST=[‘0.0.0.0’]. Default:
[]
• AXES_USE_USER_AGENT: If True, lock out / log based on an IP address AND a user agent. This means
requests from different user agents but from the same IP are treated differently. Default: False
An admin can view/clear the lockouts with the following commands:

docker exec -it martie_web_1 python manage.py axes_list_attempts


docker exec -it martie_web_1 python manage.py axes_reset [ip x.x.x.x]
docker exec -it martie_web_1 python manage.py user [name]

5. Bulk File Loading


Copy files into martie/web and create a text file with paths to files (one file per line).

docker exec -it martie_web_1 python manage.py bulkload


list_of_files_to_import.txt <username>

<username> must be the username of a user already added to MARTIE. The user also must have a valid email
address. Anyone in the user's group(s) will be able to see the uploaded files.
Alternatively, you can add an additional volume into the docker-compose.yml file (instead of copying files).
The "bulkload" custom command uploads and runs FSF over the imported files and stores the results in the Postgres
DB and makes them accessible via MARTIE Web UI.
Bulk loading CAN accept encrypted ZIP files but they must all have the same password. The default password is
"infected" but this can be modified if you provide the --password option to the bulkload command.

CMU/SEI-2018-SR-019 50
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
(v2) E-ISAC MARTIE Deployment Instructions
1. Generate an Ubuntu 16:04 Linux VPS in Azure. It is recommended that the default user is martie.

2. Log into the VPS.

ssh martie@yourdomain.com

3. Copy the MARTIE tarball from martie.cert.org.

scp eisac@martie.cert.org:/home/eisac/to_eisac_martie_*.tar.gz .

4. Extract the tarball.

sudo tar -xvzf to_eisac_martie_*.tar.gz

5. Install docker and docker-compose.

sudo apt-get install docker docker-compose

6. Enable and start docker.

sudo systemctl enable docker


sudo systemctl start docker

7. Add the 'martie' user to the docker group. You will need to login again before the new group settings will
take effect.

sudo usermod -a -G docker martie


su - martie # Login again

8. Copy NGINX certificate and keys to 'martie/data/certs/'.

cp martie.cert.org.key martie/data/certs/
cp martie.cert.org.crt martie/data/certs/

CMU/SEI-2018-SR-019 51
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
9. Edit 'martie/nginx/conf.d/default.conf' and set ssl_certificate and ssl_certificate_key.

client_max_body_size 10M;

server {
listen 80;
return 301 https://$host$request_uri;
}

server {

listen 443;
server_name martie.cert.org;
charset utf-8;

ssl_certificate /certs/<CERT_FILENAME>.crt;
ssl_certificate_key /certs/<KEY_FILENAME>.key;

...

client_max_body_size 10M;

server {
listen 80;
return 301 https://$host$request_uri;
}

server {
listen 443;
server_name martie.cert.org;
charset utf-8;

ssl_certificate /certs/martie.cert.org.crt;
ssl_certificate_key /certs/martie.cert.org.key;

...

10. Edit 'env/env_mass_app' and set the 'ALLOWED_HOSTS' variable to include your domain name.

DB_NAME=postgres

CMU/SEI-2018-SR-019 52
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
DB_USER=postgres
DB_PASS=postgres
DB_HOST=postgres.local
DB_PORT=5432
ALLOWED_HOSTS=localhost
MONGO_HOST=mongo.local
...

DB_NAME=postgres
DB_USER=postgres
DB_PASS=postgres
DB_HOST=postgres.local
DB_PORT=5432
ALLOWED_HOSTS=localhost,martie.cert.org
MONGO_HOST=mongo.local
...

11. Build the MARTIE application.

cd martie
docker-compose build

12. Run the MARTIE application.

docker-compose up -d

13. Copy static files for nginx.

docker exec martie_app_1 /bin/sh -c "python manage.py collectstatic


--noinput"

CMU/SEI-2018-SR-019 53
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
E-ISAC MARTIE Migration Instructions (v1-v2)
1. Log into the VPS.

ssh martie@yourdomain.com

2. Copy the MARTIE tarball from martie.cert.org.

scp eisac@martie.cert.org:/home/eisac/martie_v2_20180213.tar.gz .

3. Stop MARTIE.

cd martie
docker-compose down
docker-compose ps # no containers should be running

4. Backup MARTIE.

cd
sudo tar -cvzf $(date +%Y%m%d-%H%M%S)_martie_backup.tar.gz martie

5. Extract the new MARTIE.

tar -xvzf martie_v2_20180213.tar.gz

6. Copy the Postgre database.

sudo cp -r martie/pgdata martie_v2_20180213/pgdata

7. Copy the uploaded files.

cp martie/services/flask/uploads/* artie_v2_20180213/files/uploads/

8. Copy the nginx certs.

cp martie/nginx/certs/* martie_v2_20180213/data/nginx_certs/

9. Edit 'martie_v2_20180213/services/nginx/conf.d/default.conf' and set ssl_certificate and


ssl_certificate_key.

client_max_body_size 10M;

CMU/SEI-2018-SR-019 54
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
server {
listen 80;
return 301 https://$host$request_uri;
}

server {

listen 443;
charset utf-8;

ssl_certificate /certs/<CERT_FILENAME>.crt;
ssl_certificate_key /certs/<KEY_FILENAME>.key;

...

client_max_body_size 10M;

server {
listen 80;
return 301 https://$host$request_uri;
}

server {

listen 443;
charset utf-8;

ssl_certificate /certs/martie.cert.org.crt;
ssl_certificate_key /certs/martie.cert.org.key;

...

10. Edit 'martie_v2_20180213/env/env_mass_app' and set the 'ALLOWED_HOSTS' variable to include your
domain name.

DB_NAME=postgres
DB_USER=postgres
DB_PASS=postgres
DB_HOST=postgres.local
DB_PORT=5432

CMU/SEI-2018-SR-019 55
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
ALLOWED_HOSTS=localhost
MONGO_HOST=mongo.local
...

DB_NAME=postgres
DB_USER=postgres
DB_PASS=postgres
DB_HOST=postgres.local
DB_PORT=5432
ALLOWED_HOSTS=localhost,martie.cert.org
MONGO_HOST=mongo.local
...

11. Build the MARTIE application.

cd martie_v2_20180213
docker-compose build

12. Run the MARTIE application.

docker-compose up -d

13. Copy static files for nginx.

# docker exec <NAME OF APP> /bin/sh -c "python manage.py collectstatic


--noinput"
docker exec martiev220180213_app_1 /bin/sh -c "python manage.py
collectstatic --noinput"

User Guide – Within FAQ

Troubleshooting – Within FAQ

CMU/SEI-2018-SR-019 56
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.
Form Approved
REPORT DOCUMENTATION PAGE OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions,
searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments
regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington
Headquarters Services, Directorate for information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to
the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.
1. AGENCY USE ONLY 2. REPORT DATE 3. REPORT TYPE AND DATES
COVERED
(Leave Blank) March 2018
Final
4. TITLE AND SUBTITLE 5. FUNDING NUMBERS
PWP 6-461G1 DOE MARTIE Impact to Energy Sector and MARITE SRS FA8702-15-D-0002
6. AUTHOR(S)
Donovan Truitt, Dwight Beaver, Brad Runyon
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
REPORT NUMBER
Software Engineering Institute
Carnegie Mellon University CMU/SEI-2018-SR-019
Pittsburgh, PA 15213
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING
AGENCY REPORT NUMBER
AFLCMC/PZE/Hanscom
Enterprise Acquisition Division n/a
20 Schilling Circle
Building 1305
Hanscom AFB, MA 01731-2116
11. SUPPLEMENTARY NOTES

12A DISTRIBUTION/AVAILABILITY STATEMENT 12B DISTRIBUTION CODE


Unclassified/Unlimited, DTIC, NTIS
13. ABSTRACT (MAXIMUM 200 WORDS)

14. SUBJECT TERMS 15. NUMBER OF PAGES


60
16. PRICE CODE

17. SECURITY CLASSIFICATION OF 18. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION 20. LIMITATION OF
REPORT OF THIS PAGE OF ABSTRACT ABSTRACT

Unclassified Unclassified Unclassified UL


NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std. Z39-18
298-102

CMU/SEI-2018-SR-019 57
SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
[DISTRIBUTION STATEMENT A] This material has been approved for public release and unlimited distribution. Please see
Copyright notice for non-US Government use and distribution.

You might also like