DO MORE

www.rise.com.br
Assessing Security in Software Product
Lines: A Maintenance Analysis

Paulo Anselmo da Mota Silveira Neto

Ph.D. Candidate

Advisor: Vinicius Cardoso Garcia

Co-Advisor: Eduardo Santana de Almeida

Federal University of Pernambuco (UFPE)

Reuse in Software Engineering, Brazil (RiSE)
2
Agenda

Context
Software Product Lines
Software Security
Motivation
Problem Statement
Summary of Proposal
Test Beds
Family of Experiments
Concluding Remarks and Future Work
Software Product Lines

a set of software-intensive systems sharing a common,

managed set of features that satisfy the specific needs of
a particular market segment or mission and that are
developed from a common set of core assets in a
prescribed way. (PAUL CLEMENTS, 2001).
SPL (Main) Motivation: economics

Enables large-scale (systematic) reuse of artifacts

Design for family of systems
High productivity
Higher quality
Amortize costs of building variants of program
Key idea of Software Product Lines

Variability management
Enables a multitude of products to be built/composed that share
a set of artifacts

Shows how the SPL allows for and facilitates the differences
between the products.

Implementation side: Aggregation - D legation

AOP Dynam ic C ass Loading

e
ilo
Conditional C m pilation
Fram es Reflection

Dynam ic L nk L braries
I nheritance
Overloading i
Properties Param eterization
Static L braries

Source: [van Gurp et al., 2001][Svahnberg et al., 2005][Kim et al., 2005]

[Mohan and Ramesh, 2007][Deelstra et al., 2009]
Software Security

Security has became an important requirement with the

advent of multi-user computers in the late 1960s
(DENNING; DENNING, 1979).

With increased number of people connected, web and

banks transactions and apps purchasing.

It is even more important to the companies to protect its

and customer data against unauthorized access.
Security Logical Areas

Procedural Security: which includes operational,

administrative and accountability procedures

Environmental or Physical Security: it comprehends

personnel and physical security, e.g., physical controls
such as door locks

Technical Security: which consists in security related to all

communications, data, and automated information
system security.
Security Properties

Confidentiality: The data or service are protected from

unauthorized access.

Integrity: the data or service are not subject to

unauthorized manipulation

Availability: the system will be available for legitimate use.

Software Security Tactics

Security

Recover from
Detect Attacks Resist Attack React to Attacks
Attacks
-Verify message integrity
- Authenticate subject Alert Actors Audit actions
-Verify storage integrity
- Authorize subject Apply institution Apply institution
-Maintain audit trail - Manage security policies policies
-Identify Intrusions
information
- By signature
- Filter data
- By behavior - Verify origin of the
message LEGEND
- Establish secure
channel Quality Attribute
- Hide data Security Tactics
- By encryption
- By steganography Techniques

(FERNANDEZ; ASTUDILLO; PEDREZA-GARCIA, 2015)

Security at Code Level

Semantic gap between company policies and software

code. (FERNANDEZ; ASTUDILLO; PEDREZA-GARCIA, 2015).

Architectural tactics are described

as "measures" or "decisions"
taken to improve some quality
factor or "architectural building
blocks from which architectural
patterns are created" (BASS;
CLEMENTS; KAZMAN, 2012).
SPL, Software Security and Quality
Attributes: Motivation

Some non functional properties are continuous and

nonlinear (REGNELL; SVENSSON; OLSSON, 2008),
which means that, instead of viewing the quality level as
binary property of "in" or "out" as functional requirements
behaves.
It is worthwhile to know how the quality attribute
differences in a product variant can be distinguished to
the customers.
Some functional properties (i.e. manipulating complex
graphical images) might be inherently complex, making a
non functional property (i.e. performance) very difficult
to achieve.
Few studies were performed in the SPL context regarding
to how to implement security tactics at code level.
Problem Statement
Existing gap between security requirements/design and
how it can be implemented at code level.

Lack of evidence regarding to the most suitable

variability implementation mechanism to implement
security techniques.

Lack of evidence regarding to how to implement

software security with low impact on maintainability
(size, separation of concerns, cohesion and coupling).

Lack of evidence regarding to how to implement quality

attributes at code level and show how they vary.
Proposed Solution

Identify the most suitable variability implementation

mechanism to implement security techniques by
reducing code size, separation of concerns, coupling and
cohesion and increasing the benefits of reuse and
maintenance activities.

It also aims to decrease the semantic gap between

company policies and software code. By providing
different ways to implement security tactics.

Provide evidence regarding to how the quality attributes

are implemented and can vary in the SPL context.
Software Product Lines TestBeds
 RiSE Events SPL

RiSE Events SPL

Number of features: 34
Domain: Event management Number of Classes: 496
Developers: 2 Number of Methods: 1673
Ph.D candidate and Ms.C. candidate Lines of Code: 26395
 RiSE Store SPL

RiSE Store SPL

Number of features: 40
Domain: Shop Store
Number of Classes: 74
Developers: 11
Number of Methods 443
Ph.D candidates and Ms.C. candidates
Lines of Code: 5426
Law Office SPL

Law Office SPL

Domain: Law Office Number of features: 34
Developers: 3 Number of Classes: 116
Undergraduates Number of Methods 734
Lines of Code: 16665
Family of Experiments
Variability Implementation Mechanisms
Conditional compilation
#ifdef directives

Aspect-Oriented Programming
Point cuts and advices
Software Security Techniques
Experiment: Goal

The goal of this study is to evaluate different techniques to

implement security tactics for the purpose of assessing
conditional compilation and aspect-oriented programming
as variability mechanisms with respect to maintainability
by accessing size, separation of concerns, coupling and
cohesion from the point of view of software architects in
the context of Software Product Lines projects.
Question

RQ1. What is the impact on Maintainability when

evaluating security techniques implemented using
conditional compilation and aspect-oriented
programming?
It aims to understand how security techniques impact and how
they are distributed regarding to Size, Separation of Concerns,
coupling and Cohesion to support software architects design
decisions. In addition, it provides important insights regarding the
most suitable way to implement the security techniques.
Quality Model
Metrics
Size:
Lines of code (LOC)
Number of attributes (NOA)
Weighted Operations per Component (WOC)
Vocabulary Size (VS)
Cohesion:
Lack of Cohesion over Operations (LCOO)
Coupling:
Coupling Between components (CBC)
Depth Inheritance Tree (DIT)
Separation of Concerns
Concern Diffusion over Components (CDC)
Concern Diffusion over Operations (CDO)
Concern Diffusion over LOC (CDLOC)
Scattering
Family of Experiments
Hypotheses

Conditional compilation (CC) and aspect-oriented

programing (AOP) (from T1 to T15)

Null Hypotheses:

Alternative Hypotheses:
Hypotheses

Comparing the authorization patterns(from T8 to T13).

Null Hypotheses:

Alternative Hypotheses:
Hypotheses

Comparing the Detect attack tactics (from T1 to T5) and

Resist attack tactics (from T6 to T15).
Null Hypotheses:

Alternative Hypotheses:
Experiments Execution and Operation

{
Three SPLs 90 Releases

RiSE R1: T1 - CC
R2: T1 - AOP
Event
SPL

{
R30: T15 - AOP

R1: T1 - CC
RiSE R2: T1 - AOP
Store
SPL R30: T15 - AOP

Law
Office
SPL { R1: T1 - CC
R2: T1 - AOP

R30: T15 - AOP
Data Collection

Metric Tools
Metrics
Ckjm
Data
N of classes,
N of interfaces,
N of methods,
N of static methods,
N of static attributes,
N attributes,
N of parameters.
Analysis Procedure

Shapiro-Wilk normality test

does not follow a normal distribution
We choose two non-parametric tests to analyze the results,
they are following described:
Kruskal-Wallis.
Mann-Whitney-Wilcoxon Test.
Randomization test
Boxplot graph
Analysis and Interpretation

Size

- AOP vs CC. The null

hypotheses H01 was rejected.
CC techniques implementation
presented better results when
compared with AOP
implementations, when
considering most of the metrics
(LOC, WOC and VS).
Analysis and Interpretation

Size

- Authorization patterns. The

LOC, NOA and VS values
rejected the null hypotheses
H02. Authorization patterns can
be ordered starting from the best,
such as: T8, T11, T12, T13, T9
and T10.
Analysis and Interpretation

Size

- Detect Attack vs Resist

Attack.
Null hypotheses H03 was not
rejected since there is no
significant statistical difference
among these tactics.
Analysis and Interpretation

Separation of Concerns

- AOP vs CC The null

hypotheses H01 was rejected
by Concern diffusion over
Components (CDC) metric.
Analysis and Interpretation

Separation of Concerns

- Authorization. The null

hypothesis H02 was rejected for
CDC, CDO when considering CC
implementations and only CDC
when considering AOP
implementations.
Analysis and Interpretation

Separation of Concerns

- Detect Attack vs Resist

Attack. The null hypothesis H03
was not rejected.
Analysis and Interpretation

Lack of Cohesion

- AOP vs CC. The null

hypothesis H01 was only
rejected by LCOO metric for the
Multilevel Security Design Pattern
(T10) technique.
Analysis and Interpretation

Lack of Cohesion

- Authorization. There is no
significant difference for CC
or AOP, this way the null
hypothesis H02 was not
rejected.
Analysis and Interpretation

Lack of Cohesion

- Detect Attack vs Resist

Attack. The null hypotheses
H03 was not rejected, for either
variability implementation
mechanism AOP and CC.
Analysis and Interpretation

Coupling

- AOP vs CC. The null

hypothesis H01 was
rejected considering CBC
metric since the CC
implementation showed to be
more effective for all studied
techniques.
Analysis and Interpretation

Coupling

- Authorization. The null

hypothesis H02 in both AOP
and CC implementations
when considering CDC metric
was rejected.
Analysis and Interpretation

Coupling

- Detect Attack vs Resist

Attack. The null hypotheses
H03 was not rejected.
Threats to the Validity

Metrics selection
Apply some well known metrics and already empirically validated
Metric Tools selection
Cross-check tools results.
Small data sample
Randomization Tests combined with Statistical Methods
Biased data analysis
Double-check of analyzed data
Generalization
Used three SPL developed in different domains.
Related Work

RYOO; KAZMAN;ANAND, 2015

Combine architectural analysis techniques based on tactics,
patterns, and vulnerabilities in order to understand how to create a
secure architecture.
Vulnerability-oriented architectural analysis (VoAA),
Patterns oriented architectural analysis (PoAA),
Tactic-oriented architectural analysis (ToAA).
CERVANTES et al., 2016
They studied three industrial projects and one open source project
to gather empirical evidence about security practices.
how secure these systems were?
how difficult and costly it was to create the security controls?
how much it would cost to maintain them in the future?
HORCAS; PINTO; FUENTES, 2016
Generic process to model and automatically inject quality
attributes into the application without breaking the base
architecture.
Main Contributions

Software Product Lines TestBeds

Evidence about how to implement quality attributes in SPL.

Evidence about the impact of security techniques

implementation on maintainability.

Evidence about different variability implementation

mechanisms and how they impact on maintainability.

Ways to implement software security as SPL variability,

which contributes to reduce the semantic gap between
security policies and code.
Future Work

Perform the same assessment in a greater variety of test

beds.

Software security evolution.

Evaluate Security Techniques regarding to Security Degree

(e.g., Low, Medium and High)

Combination of Security Techniques.

Metric Tool.
Any Questions?

Any Questions?
Assessing Security in Software Product
Lines: A Maintenance Analysis

Paulo Anselmo da Mota Silveira Neto

Ph.D. Candidate

Advisor: Vinicius Cardoso Garcia

Co-Advisor: Eduardo Santana de Almeida

Federal University of Pernambuco (UFPE)

Reuse in Software Engineering, Brazil (RiSE)
50