Microservices

… how loose is loosely coupled?

1
 What am I going to cover?

• Overview of Microservices

• Overview of Docker Containers

• Why Docker Containers become Vulnerable

• How I built a Container Vulnerability Service using loosely coupled

microservices
3
4
 Evolution of App Designs

C, C++ Java, J2EE Ruby on Rails, Python Django Node, Angular, Swift, Whisk
Code and Manual Test Manual Deployment Automated Testing Automated Deployment
Monolithic Programs Tiered Applications Service Oriented, RESTful Stateless Microservices
Bare Metal Virtual Machines Platform As A Service Docker Containers

5
Microservices

6
 What Are Microservices?

"…the microservice architectural style is an approach to developing a single application

as a suite of small services, each running in its own process and communicating
with lightweight mechanisms, often an HTTP resource API. These services are built
around business capabilities and independently deployable by fully automated
deployment machinery."

- James Lewis and Martin Fowler

https://www.martinfowler.com/microservices/#what
7
 Microservice Architecture
• An architecture style aimed to achieve flexibility, resiliency and control, based on the following
principles:
• Single Purpose Services that are Loosely Coupling with a Bounded Context
• Independent life cycle: developed, deployed and scaled... and hopefully, fail independently
• Design for resiliency and owns it’s own data
• Polyglot — independent code base
• Built by autonomous teams with end-to-end responsibility, doing Continuous Delivery
• Communicates with other services over a well defined API
8
 Monolithic vs Microservices
Monolithic Applications Microservices-based Applications
Web / Presentation User Interface
(Apache/Nginx) Service
Customers Promotions
eCommerce Application
(WebSphere/Tomcat/PHP/Django) Shopping Cart Orders Catalog

Database
(DB2, MySQL, PostgreSQL) Relational Store NoSQL

• Tightly coupled • Loosely coupled

• Mixed Concerns • Minimal responsibility per service
• Large Deployment units • Small Deployment units
• Hard to Scale • Easy to Scale
• Long release cycles • Short release cycles
• Slow on-boarding for new developers • Fast on-boarding for new developers
• Slower feedback loop • Develop quickly with fast feedback
9
 Conway’s Law

• Any organization that designs a system (defined

broadly) will produce a design whose structure is
a copy of the organization's communication
structure.



— Melvin Conway, Datamation, 1968

• e.g., if you ask an organization with 4 teams to

write a compiler… you will get a 4-pass compiler!
http://www.melconway.com/Home/Conways_Law.html

10
 Monolithic Organization
Organized around technology
Organization Structure Application Structure

Web Tier

User Interface Team

App Tier

Application Logic Team

Database

Database (DBA) Team

11
 Microservices Organization
Organized around Business Domains

Login Inventory
Registration Personalization Shipping
Users Receiving

Account Team Personalization Team Warehouse Team

12
How do you deploy all these Microservices?

13
 The Three Pillars of Software Agility
Cultural Change Loose Coupling/Binding
Automated Pipeline RESTful APIs
Everything as Code Designed to resist failures
Immutable Infrastructure DevOps Microservices Test by break / fail fast
AGILITY

Containers
(Docker)
Portability
Developer Centric
Ecosystem enabler
Fast startup

14
15
 What is Docker?
• Docker is a light-weight container service that runs on Linux
• File system overlay
• One Process Space
• One Network Interface
• Shares the Linux kernel
• Containers encapsulate a run-time environment
• Your code, libraries, etc.
• Almost no overhead
• Containers spin up in milliseconds
• Native performance because there is no emulation
• Package only what you need

16
 Benefits of Containers
• Great isolation

• Great manageability

• Container encapsulates implementation technology

• Efficient resource utilization

• Fast deployment
17
How is it different from Virtual Machines?
• Virtual Machines are heavy-weight • Containers are light-weight like
emulations of real hardware a process
• The app looks like it’s running
on the Host OS
APP 1 APP 2 APP 3

BINS/LIBS BINS/LIBS BINS/LIBS

APP 1 APP 2 APP 3
GUEST OS GUEST OS GUEST OS
BINS/LIBS BINS/LIBS BINS/LIBS

HYPERVISOR DOCKER ENGINE

HOST OPERATING SYSTEM HOST OPERATING SYSTEM

INFRASTRUCTURE INFRASTRUCTURE

VIRTUAL MACHINES DOCKER CONTAINERS

18
How is it different from Virtual Machines?
• Virtual Machines are heavy-weight • Containers are light-weight like
emulations of real hardware a process
• The app looks like it’s running
VM VM VM
on the Host OS
APP 1 APP 2 APP 3

BINS/LIBS BINS/LIBS BINS/LIBS

Container Container Container
APP 1 APP 2 APP 3
GUEST OS GUEST OS GUEST OS
BINS/LIBS BINS/LIBS BINS/LIBS

HYPERVISOR DOCKER ENGINE

HOST OPERATING SYSTEM HOST OPERATING SYSTEM

INFRASTRUCTURE INFRASTRUCTURE

VIRTUAL MACHINES DOCKER CONTAINERS

18
Docker @ 20,000 feet

19
 Images, Layers, and Copy on Write
• Each Docker image references a list of read-
only layers that represent filesystem differences

• Layers are stacked on top of each other to

form a base for a container’s root filesystem

• When you create a new container, you add a

new, thin, writable layer on top of the
underlying stack

• All changes made to the running container -

such as writing new files, modifying existing files,
and deleting files - are written to this thin
writable container layer
20
 Creating an Image from a Dockerfile
• We can create a Dockefile to add our own content to the image

• Create a file called Dockerfile and add the following two lines:





FROM nginx:alpine







 COPY content /usr/share/nginx/html

• Build it with: docker built -t my_image .

21
 Creating an Image from a Dockerfile
• We can create a Dockefile to add our own content to the image

• Create a file called Dockerfile and add the following two lines:



Start FROM the nginx image that’s in Docker Hub


FROM nginx:alpine







 COPY content /usr/share/nginx/html

• Build it with: docker built -t my_image .

21
 Creating an Image from a Dockerfile
• We can create a Dockefile to add our own content to the image

• Create a file called Dockerfile and add the following two lines:



Start FROM the nginx image that’s in Docker Hub


FROM nginx:alpine







 COPY content /usr/share/nginx/html

COPY the folder called 'content' to '/usr/share/nginx/html' inside the container

• Build it with: docker built -t my_image .

21
 More Advanced Dockerfile
FROM alpine:3.3

# Install just the Python runtime (no dev)

RUN apk add --update \
python \
py-pip \
&& rm -rf /var/cache/apk/*

ENV PORT 5000

EXPOSE $PORT

# Set up a working folder and install the pre-reqs

WORKDIR /app
ADD requirements.txt /app
RUN pip install -r requirements.txt

# Add the code as the last Docker layer because it changes the most
ADD static /app/static
ADD service.py /app

# Run the service

CMD [ "python", "service.py" ]
22
 Containers Should Be…
• Stateless
• All state should be maintained in a DB or Object Store
• Light Weight
• Only one process per container
• Immutable
• Do not install an ssh daemon or any other means of entering the container!
• Run from Docker Registry Images or Built from Dockerfiles
• Treated like code, versioned, and reconstituted when needed… not built by hand!
23
Docker Adoption Has Increased 75% in One Year

https://www.datadoghq.com/docker-adoption/ 24
What does all of this have to do with building
loosely coupled microservices?

25
Managing Containers in the Cloud
a.k.a. Container Vulnerability Remediation Services
…the solution that I built using loosely coupled microservices

26
Vulnerabilities Happen!

27
 …and they happen every day

• A container or VM that wasn’t vulnerable yesterday is suddenly

vulnerable today

• This requires a management practice of continuous patching of

vulnerabilities

• Unfortunately, patching practices for VM’s don’t work for Containers

28
 Think Cloud Native
• The Twelve-Factor App describes patterns for cloud-
native architectures which leverage microservices

• Applications are design as a collection of stateless

microservices

• State is maintained in separate databases and persistent

object stores

• Resilience and horizontal scaling is achieved through

deploying multiple instances

• Failing instances are killed and re-spawned, not debugged

and patched (cattle not pets)

• DevOps pipelines help manage continuous delivery of

services
https://www.nginx.com/blog/introduction-to-microservices/
29
Is My Application Vulnerable?

30
 Is My Application Vulnerable?
• What is "the" application?
• It’s a loose collection of microservices ¯\_(ツ)_/¯

30
 Is My Application Vulnerable?
• What is "the" application?
• It’s a loose collection of microservices ¯\_(ツ)_/¯
• Where do I install my Agents?
• Nowhere! Containers are immutable and single process (via best practice)

30
 Is My Application Vulnerable?
• What is "the" application?
• It’s a loose collection of microservices ¯\_(ツ)_/¯
• Where do I install my Agents?
• Nowhere! Containers are immutable and single process (via best practice)
• How do I login to make changes?
• You don’t! Did I mention that Containers are immutable?
• All changes made via DevOps Pipeline
• If you are not involved in the DevOps pipeline, you are not involved in Change Management

30
 Is My Application Vulnerable?
• What is "the" application?
• It’s a loose collection of microservices ¯\_(ツ)_/¯
• Where do I install my Agents?
• Nowhere! Containers are immutable and single process (via best practice)
• How do I login to make changes?
• You don’t! Did I mention that Containers are immutable?
• All changes made via DevOps Pipeline
• If you are not involved in the DevOps pipeline, you are not involved in Change Management
• When is my change window?
• Never! Changes are made by application teams using blue/green deployments for continuous up-time
30
 ?
How do you manage vulnerabilities with this
Explosion of Container Growth?

31
 Center for Internet Security Docker Benchmark
Recommendation
• Scan and rebuild the images to include security patches
• Images should be scanned "frequently" for any vulnerabilities. Rebuild the images to include patches and then instantiate new
containers from it.
• Rationale:
• Vulnerabilities are loopholes/bugs that can be exploited and security patches are updates to resolve these vulnerabilities. We
can use image vulnerability scanning tools to find any kind of vulnerabilities within the images and then check for available
patches to mitigate these vulnerabilities.
• Remediation:
• Step 1: 'docker pull' all the base images (i.e., given your set of Dockerfiles, extract all images declared in 'FROM' instructions,
and re-pull them to check for an updated/patched versions). Patch the packages within the images too.
• Step 2: Force a rebuild of each image with 'docker build --no-cache'.
• Step 3: Restart all containers with the updated images.

32
 User Story: Container Vulnerability Remediation
• User Story: Container Vulnerabilities
• As an Application Owner
• I need an automated way to patch containers
• So that they won't be vulnerable to exploits
 e r
U s
S t o r y
• Assumptions:
• There will be long running containers ( > 24 hrs)
• There will be new vulnerabilities discovered every day
• Manually patching images and redeploying containers is too labor intensive


• Acceptance Criteria:
• Given a Docker image with deployed containers
• When a vulnerability has been found in the Docker image
• Then a remediation of that image will be performed
• And a new image will be created and push to the registry
• And any containers from the old image will be redeployed using the new image

33
The Solution

34
 Container Vulnerability Remediation Services
Architectural Overview
Vulnerability Remediation Services Compliance Remediation Services
IBM Cloud
USN-xxxx.1
Vulnerability Advisor

Cloud Native:
Containers that conform Hybrid Cloud:
to 12-factor Integration with
Compliance Service Customer’s Compliance
Scanning

Hybrid Cloud:
Hybrid Cloud:
Integration with
Deployable on Local of
Customer’s existing
Public Kubernetes
systems

Redeploy Container Workload

Notify Other Systems of Record

Kubernetes

35
 How Does It Work?
• Compliance Service
• Analyzes the input from Vulnerability Advisor and publishes alerts for other services
• Vulnerability Remediation Services
• Maintains a knowledge base of fix procedures and compliance remediation actions
• Composes new Docker file containing the remediation actions based on the knowledge base
• Forwards the new Docker file to the build service which in turn produces new version of the image
• Redeploy Container Workload
• Redeploys the container on Kubernetes by modifying the existing deployment parameters
• Other Systems of Record
• Monitor the other service messages keeping ticking system and development up to date on activities
36
{
What’s in a CVE?
"id": "78de8449-313e-44c9-90b1-ae8277b12f95",
"scan_time": 1510759001,
"status": "WARN",
"vulnerabilities": [
{
"type": "vulnerable_package",
"description": "busybox 1.24.1-r7 has vulnerabilities",
"corrective_action": "Upgrade to busybox 1.24.2-r1",
"fixes": [
{
"cve_ids": "CVE-2016-6301",
"summary": "The recv_and_process_client_pkt function in networking/ntpd.c in busybox allows remote attackers to
cause a denial of service (CPU and bandwidth consumption) via a forged NTP packet, which triggers a communication loop.",
"notice": "",
"meta": {
"usn_id": ""
}
}
],
"meta": {
"package_name": "busybox"
"fix_version": "1.24.2-r1",
"installed_version": "1.24.1-r7",
}
},
37
{
What’s in a CVE?
"id": "78de8449-313e-44c9-90b1-ae8277b12f95",
"scan_time": 1510759001,
"status": "WARN",
"vulnerabilities": [
{
"type": "vulnerable_package",
"description": "busybox 1.24.1-r7 has vulnerabilities",
"corrective_action": "Upgrade to busybox 1.24.2-r1",
"fixes": [
{
"cve_ids": "CVE-2016-6301",
"summary": "The recv_and_process_client_pkt function in networking/ntpd.c in busybox allows remote attackers to
cause a denial of service (CPU and bandwidth consumption) via a forged NTP packet, which triggers a communication loop.",
"notice": "",
"meta": {
"usn_id": ""
}
}
], This is how we know what to patch
"meta": {
"package_name": "busybox"
"fix_version": "1.24.2-r1",
"installed_version": "1.24.1-r7",
}
},
37
 Generated Dockerfile for Alpine
#
# Remediation Service
#

FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1

#
# Patching Vulnerabilities
#

RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi;
RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi;

#
# Fixing Compliance Issues
#

RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYS\t90/g' /etc/login.defs; fi;

# NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.

38
 Generated Dockerfile for Alpine
#
# Remediation Service
# References the original image
FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1

#
# Patching Vulnerabilities
#

RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi;
RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi;

#
# Fixing Compliance Issues
#

RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYS\t90/g' /etc/login.defs; fi;

# NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.

38
 Generated Dockerfile for Alpine
#
# Remediation Service
# References the original image
FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1

#
# Patching Vulnerabilities Updates the libraries that are vulnerable
#

RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi;
RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi;

#
# Fixing Compliance Issues
#

RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYS\t90/g' /etc/login.defs; fi;

# NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.

38
 Generated Dockerfile for Alpine
#
# Remediation Service
# References the original image
FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1

#
# Patching Vulnerabilities Updates the libraries that are vulnerable
#

RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi;
RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi;

#
# Fixing Compliance Issues Fixes known compliance issues
#

RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYS\t90/g' /etc/login.defs; fi;

# NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.

38
 Generated Dockerfile for Ubuntu
#
# Remediation Service
#

FROM registry.ng.bluemix.net/rofrano/counter:latest
MAINTAINER Vulnerability Remediator 1.0

#
# Patching Vulnerabilities
#

RUN apt-get install -y bash=4.3-7ubuntu1.7;if [ $? -ne 0 ]; then apt-get install -y bash; fi;
RUN apt-get install -y libexpat1=2.1.0-4ubuntu1.4;if [ $? -ne 0 ]; then apt-get install -y libexpat1; fi;
RUN apt-get install -y libffi6=3.1~rc1+r3.0.13-12ubuntu0.2;if [ $? -ne 0 ]; then apt-get install -y libffi6; fi;
RUN apt-get install -y libgcrypt11=1.5.3-2ubuntu4.5;if [ $? -ne 0 ]; then apt-get install -y libgcrypt11; fi;
RUN apt-get install -y libgnutls26=2.12.23-12ubuntu2.8;if [ $? -ne 0 ]; then apt-get install -y libgnutls26; fi;
RUN apt-get install -y libssl1.0.0=1.0.1f-1ubuntu2.23;if [ $? -ne 0 ]; then apt-get install -y libssl1.0.0; fi;
RUN apt-get install -y libtasn1-6=3.4-3ubuntu0.5;if [ $? -ne 0 ]; then apt-get install -y libtasn1-6; fi;
RUN apt-get install -y login=1:4.1.5.1-1ubuntu9.5;if [ $? -ne 0 ]; then apt-get install -y login; fi;
RUN apt-get install -y passwd=1:4.1.5.1-1ubuntu9.5;if [ $? -ne 0 ]; then apt-get install -y passwd; fi;
RUN apt-get install -y sudo=1.8.9p5-1ubuntu1.4;if [ $? -ne 0 ]; then apt-get install -y sudo; fi;

39
Possible Architecture Using Fixed Workflow
subscription Container Instance Remediation
to advisories data Knowledge Base

USN-xxxx.1
Detect non-compliant Generate
running containers and Dockerfile
Flag non- initiate remediation of to build
compliant relevant images new image
images
Vulnerability Image
Compliance Container Image Container
Remediation
Advisor Service Service Build Service ReDeploy Service
Service

vulnerabilities and Creates new

compliance Containers existing
violations deployment specs
Policies
Container Image
repository
update
open
close Based on policy, we may
redeploy existing
DevOps Pipeline ServiceNow containers from
Feedback Integration remediated
image

40
Possible Architecture Using Fixed Workflow
subscription Container Instance Remediation
to advisories data
DO Knowledge Base

USN-xxxx.1

Flag non-
N’
Detect non-compliant
running containers and
initiate remediation of
Generate
Dockerfile
to build
compliant
images T
relevant images
Image
new image

Vulnerability
Advisor Service
Compliance
Service DO Remediation
Service
Container Image
Build Service
Container
ReDeploy Service

vulnerabilities and
compliance TH Creates new
Containers existing
violations
Policies IS Container Image
deployment specs

repository
update
open
close Based on policy, we may
redeploy existing
DevOps Pipeline ServiceNow containers from
Feedback Integration remediated
image

40
 Problems with Fixed Workflow
• Microservices are tightly coupled

• Each service knows who to send the next request to

• Ticketing service must be known by all services

• No way to integrate future services without modifying several services to give them
knowledge

• Remediating images could take 30 - 40 minutes and polling for a response is not
desirable
41
 A Microservice Should Have
• High Cohesion (Bounded Context around a Business Domain)

• Does stuff that needs to change together occur together?

• Low Coupling (Shared Nothing with Technology Agnostic API)

• Do you avoid making otherwise independent concerns dependent?

• Low Time to Comprehension (Small and Single Responsibility)

• Small enough for one person to understand quickly

42
 Architecture Using Messaging
Remediation
Knowledge Base
subscription Container Instance
to advisories data

USN-xxxx.1

Flag non- Detect non-compliant

compliant running containers and Image
initiate remediation of
images Remediation
relevant images
Vulnerability Compliance Pub/Sub Service
Advisor Service Service
Generate
Dockerfile
vulnerabilities and
to build
compliance new image
violations
Policies DevOps Pipeline
Creates new
Feedback Containers from Container Image
existing deployment Build Service
specs

Container
ReDeploy
ServiceNow
Integration
Based on policy, we Container Image
may redeploy existing repository
containers from
remediated
image

43
Container Vulnerability Remediation Services Technology
Part of IBM Cloud
VA Policy Manager
New Vulnerability Remediator Vulnerability Advisor
New Compliance Remediator

User Experience Compliance Service Cloud Function

Periodic Trigger
New Vulnerability
Remediation
Knowledge Base

DevOps Issue Service*

Remediation Service

Pub/Sub

Docker Build Service ServiceNow Callout*

Checklist Builder
Runlist Builder
Container Redeploy Service
Compliance Scanner
Compliance Remediator
* Future Services

!44
 Cloud Functions with OpenWhisk
• OpenWhisk is a cloud-first distributed
event-based programming service

• It represents an event-action platform that

allows you to execute code in response to
an event

• Provides a serverless deployment and

operations model hiding infrastructural
complexity

• Simply provide the code you want to

execute
45
Cloud Functions High Level Architecture

46
 Example Python Cloud Function
def main(params):
""" Container Redeploy Function """

# Get the API key and use it in the headers

headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')

value = params['messages'][0]['value'] # process messages

namespace = value.get('namespace', 'default') # check parameters

# create the message body

body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}

logging.info('Redeploying: %s', value['old_image'])

result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
47
 Example Python Cloud Function
def main(params):
""" Container Redeploy Function """

# Get the API key and use it in the headers

Get the endpoint to be called
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')

value = params['messages'][0]['value'] # process messages

namespace = value.get('namespace', 'default') # check parameters

# create the message body

body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}

logging.info('Redeploying: %s', value['old_image'])

result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
47
 Example Python Cloud Function
def main(params):
""" Container Redeploy Function """

# Get the API key and use it in the headers

Get the endpoint to be called
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
Get parameters passed in
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters

# create the message body

body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}

logging.info('Redeploying: %s', value['old_image'])

result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
47
 Example Python Cloud Function
def main(params):
""" Container Redeploy Function """

# Get the API key and use it in the headers

Get the endpoint to be called
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
Get parameters passed in
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters

# create the message body

body = {
'namespace': namespace,
Construct body of request
'old_image': value['old_image'], for microservice to call
'new_image': value['new_image']
}

logging.info('Redeploying: %s', value['old_image'])

result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
47
 Example Python Cloud Function
def main(params):
""" Container Redeploy Function """

# Get the API key and use it in the headers

Get the endpoint to be called
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
Get parameters passed in
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters

# create the message body

body = {
'namespace': namespace,
Construct body of request
'old_image': value['old_image'], for microservice to call
'new_image': value['new_image']
}

logging.info('Redeploying: %s', value['old_image'])

result = requests.post(api_endpoint, Make the call to fire off the microservice
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
47
 Example Python Cloud Function
def main(params):
""" Container Redeploy Function """

# Get the API key and use it in the headers

Get the endpoint to be called
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
Get parameters passed in
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters

# create the message body

body = {
'namespace': namespace,
Construct body of request
'old_image': value['old_image'], for microservice to call
'new_image': value['new_image']
}

logging.info('Redeploying: %s', value['old_image'])

result = requests.post(api_endpoint, Make the call to fire off the microservice
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message} Return the microservice output
47
 Message Pub/Sub Interactions
Each microservice is independent being called asynchronously as events are published

Compliance Service

Remediation Service

Pub/Sub
Image Build Service

Container Redeploy
Service

48
 Message Pub/Sub Interactions
Each microservice is independent being called asynchronously as events are published

Vulnerable Image
Compliance Service

Vulnerable Image
Remediation Service

Pub/Sub
Image Build Service

Container Redeploy
Service

48
 Message Pub/Sub Interactions
Each microservice is independent being called asynchronously as events are published

Vulnerable Image
Compliance Service

Vulnerable Image
Remediation Service Image Remediated

Image Remediated Pub/Sub

Image Build Service

Container Redeploy
Service

48
 Message Pub/Sub Interactions
Each microservice is independent being called asynchronously as events are published

Vulnerable Image
Compliance Service

Vulnerable Image
Remediation Service Image Remediated

Image Remediated Pub/Sub

Image Build Service Image Build Complete

Image Build Complete

Container Redeploy
Service

48
 Message Pub/Sub Interactions
Each microservice is independent being called asynchronously as events are published

Vulnerable Image
Compliance Service

Vulnerable Image
Remediation Service Image Remediated

Image Remediated Pub/Sub

Image Build Service Image Build Complete

Image Build Complete

Container Redeploy
Service Containers Redeployed

48
 How to Add a New Service
New services can be added by subscribing to events on the message bus

Compliance Service

Ticket Notification Pub/Sub

Service

49
 How to Add a New Service
New services can be added by subscribing to events on the message bus

Vulnerable Image
Compliance Service

Vulnerable Image
Create
Ticket

Ticket Notification Pub/Sub

Service

49
 How to Add a New Service
New services can be added by subscribing to events on the message bus

Vulnerable Image
Compliance Service

Vulnerable Image
Create
Ticket

Update Ticket Notification Image Remediated Pub/Sub

Ticket Service

49
 How to Add a New Service
New services can be added by subscribing to events on the message bus

Vulnerable Image
Compliance Service

Vulnerable Image
Create
Ticket

Update Ticket Notification Image Remediated Pub/Sub

Ticket Service

Close Image Redeploy Complete

Ticket

49
 Messaging Allows For:
• Services that are agnostic to other downstream services (a.k.a loosely
coupled)

• Long running services can simply publish when they are done instead
of polling for completion

• New services can be created to participate in the workflow without

any change to existing services
50
 Summary
• Using a messaging pub/sub design keeps microservices loosely coupled

• This also allows other services to participate in the workflow without

changing or knowledge from existing services

• Docker containers make encapsulation and deployment of services easy

• Using Cloud Functions (a.k.a. Serverless) complements the event driven

workflow and simplifies message flows
51