The document is a presentation by John Scattergood on building reactive microservices with JRuby and Docker. Some key points: - Scattergood is a senior principal engineer at Autodesk who enjoys coding in languages like Java, Ruby, Groovy and JavaScript - He will discuss running JRuby applications in production that serve millions of requests per day and building reactive systems using tools like Ratpack, Reactive Streams and Docker Swarm - Examples are given of building reactive REST APIs, handling uneven load through backpressure, buffering and caching, and auto-scaling services using Docker and tools like Orbiter

1. © 2017 Autodesk
John Scattergood
Senior Principal Engineer
JavaOne 2017
Reactive Microservices
with JRuby and Docker

2. § Engineer at Autodesk
§ Enjoy coding in Java, Ruby,
Groovy and JavaScript
§ Run JRuby apps in production
that serve millions of requests/day
§ First time JavaOne Speaker
@johnscattergood
About Me

3. Reactive Systems

4. Our System
Market Updates
Stock Updates
Alerts/Rules
Notifications
SMS
DB Autoscaler
Cluster
Scale

5. § Price Alerts (eCommerce)
§ APM and Infrastructure Monitoring (IT)
§ Fraud Detection (Credit)
§ Devices, Transponders, etc. (IOT)
Other Domains

6. Tools

7. § Dynamic
§ Object-oriented w/ first class functions
§ Expressive
§ Flexible
§ Easy to learn
Ruby
Ruby is simple in appearance, but is very complex inside just like our human body
- Yukihiro “Matz” Matsumoto
Ruby Creator

8. @memo = { 1 => 1, 2 => 2 }
def fib(n)
@memo[n] ||= fib(n - 1) + fib(n - 2)
end
fib(100)
=> 573147844013817084101
Fibonnaci

9. § Fully compatible implementation of Ruby
§ Includes benefits of JVM
§ High Performance
§ True multi-threading
§ Java libraries and interoperability
§ Standard monitoring tools
JRuby

10. require 'java'
java_import 'java.util.concurrent.ConcurrentHashMap'
map = ConcurrentHashMap.new
map['A'] = 123
map.put_if_absent('B', 456)
puts map
=> {"A"=>123, "B"=>456}
Java to Ruby

11. § Asynchronous HTTP via Netty
§ Reactive Streams Support
§ Promises and Streams library
§ Works great with JRuby
§ Plugin Modules
§ Guice
§ Dropwizard Metrics
§ Etc.
Ratpack

12. RatpackServer.start do |server|
server.handlers do |chain|
chain.get do |ctx|
ctx.render(’Hello World!')
end
end
end
Starting a Server

13. § Push-Pull based “stream” processing:
§ Non-blocking
§ Asynchronous
§ Potentially Unbounded
§ Non-blocking backpressure
§ JDK 9: java.util.concurrent.Flow
Reactive Streams

14. Promise
.value(@metric_registry)
.map(MetricRegistryJsonMapper.new(@alloc, MetricFilter::ALL))
.map { |buf| buf.to_string(Charset.default_charset) }
.map { |string| JSON.parse(string) }
.flat_map { |metric| send_event(metric) }
.then { |resp| Common::Log.debug "#{resp.status.code}" }
Example

15. § Cluster management with Docker Engine
§ Task Scheduling and Scaling
§ Service Discovery
§ Load Balancing
§ Automatic Restart/Recovery
§ Rolling updates
Docker Swarm Mode

16. § Cluster service that offers a REST API for scaling up and
down tasks
§ Automatically detects managed services in swarm
§ Supports
§ Preset scale increments/decrements
§ Cooldowns to avoid mistakes
Orbiter

17. Making a reactive system

18. Making a reactive system

19. Making a reactive system

20. Making a reactive system

21. Making a reactive system

22. Our System
HTTP POST
Market Updates
Stock Updates
Alerts/Rules
Notifications
SMS
DB Autoscaler
Cluster
Orbiter
HTTP POST

23. REST = Synchronous
Synchronous ≠ Reactive
How is this reactive?

24. § Message Passing via HTTP POST
§ Service should limit blocking as much as possible
§ Return a Receipt for long processes
§ Communicate Backpressure via Status Codes (429)
§ Client decides how to handle messages if service is
unavailable
§ Buffering
§ Dropping
Reactive REST

25. § HTTP is really nice because…
§ DNS and/or Service Discovery
§ Load Balancing
§ Transparent Scaling
§ Dynamic Routing and Proxies
Nothing wrong with Queues, but…

26. Challenge: Handling Uneven Load

27. § Backpressure
§ Make the caller wait
§ Buffering
§ Accept the requests and send batches downstream
§ Caching
§ Accept the requests and send most recent downstream
Strategies

28. chain.post(stock') do |ctx|
http_client = ctx.get(HttpClient.java_class)
alert_service_client =
AlertServiceClient.new(ENV[’SA_ALERT_SERVICE_URL'],
http_client)
ctx.request.body
.map { |b| JSON.parse(b.text) }
.flat_map { |event| alert_service_client.send_event(event) }
.then { ctx.render('OK') }
end
Backpressure

29. Backpressure Demo

30. Buffering
chain.post(’stock') do |ctx|
buffer = ctx.get(EventBuffer.java_class)
ctx.request.body
.map { |b| JSON.parse(b.text) }
.flat_map { |event| buffer.add(event) }
.then do |buffered|
if buffered
ctx.render('OK')
else
Common::Log.debug 'backpressure!!!'
ctx.response.status(429).send
end
end
end

31. def add(event)
Promise.async do |d|
d.success(@buffer.offer(event, 1, TimeUnit::SECONDS))
end
end
def run
Execution.fork.start do |_|
get_events
.map { |events| reduce_events(events) }
.flat_map { |events| send_events(events) }
.defer(Duration.of_seconds(@backoff_duration))
.then do |responses|
responses.each { |response| handle_response(response) }
end
end
end
Buffering (contd)

32. Buffer Demo

33. Caching
chain.post(’stock') do |ctx|
cache = ctx.get(EventCache.java_class)
ctx.request.body
.map { |b| JSON.parse(b.text) }
.flat_map { |event| cache.add(event) }
.then do |cached|
if cached
ctx.render('OK')
else
Common::Log.debug 'backpressure!!!'
ctx.response.status(429).send
end
end
end

34. def add(event)
Promise.async do |d|
@cache.put(event['ticker'], event['price'])
@updates.put(event['ticker'], true)
d.success(true)
end
end
def run
Execution.fork.start do |_|
start = Time.now
get_events
.map { |keys| hydrate_events(keys) }
.flat_map { |events| send_events(events) }
.defer(Duration.of_seconds(@backoff_duration))
.then do |responses|
responses.each { |response| handle_response(response) }
end
end
end
Caching (contd)

35. Cache Demo

36. def backpressure(backoff)
if backoff
@metric_registry.meter('backpressure.service.alert').mark
@backoff_duration = [30, 1 + @backoff_duration * 2].min
puts "increasing back off to #{@backoff_duration} secs"
else
@backoff_duration = [@backoff_duration / 2, 1].max
if @backoff_duration > 1
puts "decreasing back off to #{@backoff_duration} secs"
else
@backoff_duration = 0
end
end
end
Backpressure

37. Cluster
Machine Machine Machine
Swarm Manager
orbiter
stock alert alertalert notif notif
scale db

38. Cluster
Machine Machine Machine
Swarm Manager
orbiter
stock alert alertalert notif notif
scale
Ingress traffic
Routed to Service
db

39. Cluster
Machine Machine Machine
Swarm Manager
orbiter
stock alert alertalert notif notif
scale
Ingress traffic
Routed to Service
db

40. Cluster
Machine Machine Machine
Swarm Manager
orbiter
stock
alert alertalert notif notif
scale
alert
Ingress traffic
Routed to Service
db

41. Monitoring Our Services

42. Scaling

43. def adjust_scale(metric_name, metric_value)
service = metric_name.split('.').last
Promise
.value(metric_value)
.next_op_if(
->(v) { v['m1_rate'] < 0.01 && v['m5_rate'] < 0.01 },
->(_) { scale_down(service) }
)
.next_op_if(
->(v) { v['m1_rate'] > 1 },
->(_) { scale_up(service) }
)
end
Scaling Up and Down

44. alert:
image: stockalert
command: bundle exec jruby ./alert_service/run.rb
networks:
- stockalert
deploy:
mode: replicated
replicas: 1
labels: [orbiter=true,orbiter.up=2,orbiter.cooldown=300]
…
Docker Scaling Configuration

45. Watch it in Action!
Scaling Up

46. Monitoring Effects of Scaling

47. Watch it in Action!
Scaling Down

48. Demo
HTTP POST
Market Updates
Stock Updates
Alerts/Rules
Notifications
SMS
DB Autoscaler
Cluster
Orbiter
HTTP POST

49. § Joe Kutner, Deploying JRuby 9k
§ Joe Kutner, “Reactive Ruby: Building Real-time Apps with
JRuby and Ratpack”, http://blog.heroku.com
§ Dan Woods, Learning Ratpack
§ Jonas Bonér, Reactive Microservices Architecture
§ GitHub Project:
https://github.com/jscattergood/JavaOne_2017_CON3214
Further Reading

50. Autodesk and the Autodesk logo are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. All other brand names, product names, or trademarks belong to their respective holders.
Autodesk reserves the right to alter product and services offerings, and specifications and pricing at any time without notice, and is not responsible for typographical or graphical errors that may appear in this document.
© 2017 Autodesk. All rights reserved.