SPRINGONE2GX

WASHINGTON, DC

Reactive Web Applications

Stephane Maldini
Rossen Stoyanchev

Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
About the Speakers
• Stephane Maldini
• Reactor Project Lead
• Reactive Streams Spec contributor

• Rossen Stoyanchev
• Spring Framework committer
• Spring MVC
• WebSocket messaging (STOMP, SockJS)

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 2
 Why are you here ?

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 3
 What is Reactive ?

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 4
 Why Non Blocking Matters ?

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 5
 Non Blocking = Faster ?

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 6
 Non Blocking = Internet Scale ?

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 7
Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 8
 Reactive Web App Stack

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 9
Servlet 3.0 Async Requests
• Async HTTP requests have been possible since Servlet 3.0

• Essentially: the ability to get off the Servlet container thread

• originated from Jetty continuations, CometD, etc.

• “Live” data in web apps

• e.g. HTTP streaming, long polling

• Reduce burden on server thread pool

• scatter-gather, external REST APIs, etc

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 10
Servlet 3.0 Async Requests: the bottom line
• A bit quirky, Servlet API wasn’t built for async

• Complex to grasp in its entirety + comes with some caveats

• e.g. lack of notification when client goes away

• Spring MVC provides full support for Servlet 3 async requests

• alleviates most concerns

• Effectively async can be introduced into existing applications

• … and ecosystem of libraries !

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 11
Servlet 3.1 Non-Blocking I/O
• Read and write without blocking

• Register ReadListener and WriteListener callbacks

• switches to non-blocking mode

• ServletInputStream and ServletOutputStream change behavior

• Must call isReady first before read() or write()

• if connection not ready (slow client), callback is scheduled

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 12
Servlet 3.1 Non-Blocking: the bottom line
• Carefully designed + it does provide non-blocking capability

• However not easy to use correctly, a number of pitfalls

• Many “innocent” looking parts of Servlet API may lead to blocking

• e.g. request.getParameters()

• For a good overview and perspective see Greg Wilkins’ talk

• ”Into the Wild with Servlet 3.1 Async I/O”

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 13
Reactive Streams Spec for the JVM
• Specification for non-blocking library interop
• governs exchange of data across async boundaries

• Collaborative effort
• Kaazing, Netflix, Pivotal, RedHat, Twiiter, Typesafe

• Momentum and growing ecosystem of libraries

• network, database, etc.

• On path to adoption in JDK 9 java.util.concurrent

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 14
Reactive Streams API

onSubscribe

request(n) Subscriber
Publisher Subscription cancel()

onNext*
(onError|onComplete)?

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 15
Reactive Back-Pressure
• A key part of Reactive Streams is back-pressure

• It’s a type of flow control

• Subscribers make requests for additional data

• Publishers abide and produce what has been requested

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 16
Reactive Streams for Network I/O

public interface ServerHttpRequest extends HttpRequest {

Publisher<byte[]> getBody();

public interface ServerHttpResponse extends HttpMessage {

Publisher<Void> writeWith(Publisher<byte[]> publisher);

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 17
Instead of...

public interface ServerHttpRequest extends HttpRequest {

InputStream getBody();

public interface ServerHttpResponse extends HttpMessage {

OutputStream getBody();

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 18
Reactive Streams over Servlet Runtime ?
• Servlet 3.1 provides the non-blocking capability

• Reactive Streams provides a non-blocking API with broad industry support

• A single Reactive Streams pipeline

• back-pressure through the layers (from DB to Web server)

• Jetty is actively investigating this very option

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 19
Reactive Streams and Servlet 4.0 ???
• It is under discussion in the expert group

• It would be great if it did happen

• Potential obstacle is the Java EE 8 - JDK 8 association

• Hopefully not a show-stopper since Reactive Streams is here today

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 20
Reactive Streams Over Netty
• Netty was designed from the ground up to be non-blocking

• Easy to translate to Reactive Streams semantics

• Two ongoing experiments

• RxNetty
• Reactor Net

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 21
 Reactive Streams Network I/O
Akka HTTP, Vert.x, Ratpack, Reactor Net, RxNetty*

*(via RxJava-ReactiveStreams bridge)

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 22
 Reactive
Web App Stack
Reactive Streams

Netty, Servlet 3.1

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 23
Web Framework
Reactive
Web App Stack
Reactive Streams

Netty, Servlet 3.1

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 24
Application

Web Framework
Reactive
Web App Stack
Reactive Streams

Netty, Servlet 3.1

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 25
 HTTP
Reactive
Application Data
Streams
Broker

Web Framework
Reactive
Web App Stack
Reactive Streams

Netty, Servlet 3.1

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 26
 HTTP
Application
? Reactive
Streams Data
Broker
?
Web Framework
API to compose
asynchronous
Reactive Streams
streams?
Netty, Servlet 3.1

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 27
Working with Streams
• Non-blocking services return Publisher<T> instead of <T>

• How do you attach further processing?

• Reactive Streams is a callback-based API

• becomes very nested quickly

• Need something more declarative

• it’s beyond the scope of Reactive Streams

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 28
Stream Operations
• Publisher represents a stream of data

• It’s natural to apply operations functional-style

• like the Java 8 Stream

• Need API for composing async logic

• rise above callbacks

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 29
Reactor Stream
• Project Reactor provides a Stream API

• Reactive Streams Publisher + composable operations

Streams.just('a' 'b' 'c')

.take(2)
.map(Character::toUpperCase)
.consume(System.out::println);

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 30
Reactive Extensions (Rx)
• Stream composition API based on Observer pattern
• originated at Microsoft, work by Erik Meijer

• Implemented for different languages -- RxJava, RxJS, Rx.NET, …

Observable.just('a', 'b', 'c')

.take(2)
.map(Character::toUpperCase)
.subscribe(System.out::println);

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 31
RxJava and Reactive Streams
• RxJava 1.x predates Reactive Streams and doesn’t implement it directly

• Very similar concepts, different names

• Observable-Observer vs Publisher-Subscriber

• RxJava supports “reactive pull” back-pressure

• RxJava 1.x - Reactive Streams bridge

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 32
 Demo:

HeadFirst

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 33
RxNetty
• RxJava as the reactive bridge to Netty IO operations

• Immediate and embedded client/server for TCP and HTTP

• Decorating IO Read with rx.Observable and applying dynamic pull/push

• Decorating IO Write with rx.Subscriber and requesting more on flush

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 34
RxNetty … what’s up
• RxNetty has introduced Backpressure support in 0.5

• Powered by the backpressure protocol from RxJava

• Overall the API has been heavily lifted too

https://github.com/ReactiveX/RxNetty/wiki/0.5.x-FAQs

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 35
Reactor Net
• Reactive Streams as the reactive bridge to Netty and ZeroMQ IO operations

• Immediate and embedded client/server for TCP, UDP and HTTP

• Decorating IO Read with Publisher and applying dynamic pull/push

• Decorating IO Write with Subscriber and requesting more on flush

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 36
Member of the Reactor initiative
• Reactor is a maturing Reactive Streams suite of libraries

• As with a lot of the reactive libraries, observing a rapidly evolving set of API

• Precluded Reactive Streams specification work

• Now evolving and getting ready for Spring 5 Reactive story

projectreactor.io , github.com/reactor

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 37
 Demo:

Back-Pressure

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 38
Async I/O with Reactive Streams
• Never block on Applications Queues :
Max(request) == Queue.capacity()

• Keep Reading : Any asynchronous handoff is non-blocking

• Make Writes Quicker : IO operation signals demand on availability

• Optimize hardware resources : Claim bounded memory and cpu

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 39
Reactive Read
One read implementation : Publisher stop-read

• Consume NIO Buffers only when requested

• request size is != bytes read

• Application stopping reads propagates back-pressure

• server buffers fill
• client buffers fill
• eventually throttles client

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 40
Reactive Write
One write implementation : Subscriber request after flush

• Request after flush indicates availability in write buffer

• Request size is != bytes written

• Write buffers filling naturally propagates back-pressure

• buffers fill
• no requests
• eventually throttles application

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 41
Serialization Concerns
• Subscription.request(n)is not absolute ( != bytes number )

• Decoding operations might produce one to many objects

• must match subscriber capacity

• Encoding operations might consume one to many byte chunks

• must match buffering capacity

• Available serialization libraries not built for chunked streams

• Jackson, Kryo, Protobuf, etc.

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 42
 Demo:

Scatter-gather

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 43
 Spring Reactive

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 44
Spring Reactive
• Experimental work for Spring Framework 5

• Non-blocking runtimes -- Netty, Jetty, Tomcat

• Reactive Streams over network I/O -- RxNetty, Reactor Net

• Spring web processing

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 45
Reactive Streams Publisher

@RequestMapping("/capitalize")
@ResponseBody
public Publisher<Person> capitalize(@RequestBody Publisher<Person> persons) {
// …
}

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 46
Reactor Stream

@RequestMapping("/capitalize")
@ResponseBody
public Stream<Person> capitalize(@RequestBody Stream<Person> persons) {
return persons.map(person -> {
person.setName(person.getName().toUpperCase());
return person;
});
}

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 47
RxJava Observable

@RequestMapping("/capitalize")
@ResponseBody
public Observable<Person> capitalize(@RequestBody Observable<Person> persons) {
return persons.map(person -> {
person.setName(person.getName().toUpperCase());
return person;
});
}

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 48
 ReactiveX On the Client Side

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 49
RxJS
function() {
StreamService.connect("stream")
.flatMap(unpackCsv)
.subscribe(updateUI, logError, logComplete);
}

var unpackCsv = function (ev) {

return Rx.Observable.from(ev.data ? ev.data.split("\n") : [])
.filter(discardEmpty)
.map(csvToJson)
}

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 50
 Demo:

RxJS and Ratpack

Unless otherwise indicated these slides are © 2013-2014 Pivotal Software Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 51
 Learn More. Stay Connected.

@springcentral Spring.io/video

Thank You for Listening!

@smaldini / @rstoya05
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 52