Short presentation on AMQP, RabbitMQ and Nanite.

1. RabbitMQ and Nanite
Scalable Messaging
Mathias Meyer, Peritor GmbH

3. The Problem
• A scalable backend to offload asynchronous
tasks
• Self-configuring
• Self-healing
• Self-assembling
• But not a lot of people have it

4. The Problem
• I’m still learning about the nitty-gritty details

5. Protocols
• JMS
• MQ
• XMPP
• SMTP (yep, for real)
• STOMP
• RestMS
• Wtf?

7. AMQP!

8. AMQP
• Open standard for messaging middleware
• Vendor neutral
• Wire level protocol
• Mandates interoperatibility
• Abundant number of libraries that speak it

9. AMQP
• Supports publish/subscribe and point-to-point
messaging, among others

10. AMQP Components
• Broker
• Combination of Exchange and a Queue
• Exchange routes messages (stateless) based
on patterns
• Queue buffers messages (stateful)
• Bound by Bindings

11. AMQP Components
• Consumer
• Producer
• Messages (duh!)

13. RabbitMQ

14. RabbitMQ
• Erlang-based, independent AMQP
implementation
• Some 6k lines of code
• Apache Qpid (Java): 142k lines of code
• Also speaks STOMP, SMTP, XMPP and HTTP
• Message persistence uses a journal
• Wicked fast

18. RabbitMQ
• port install rabbitmq-server
• It just works™
• rabbitmqctl
• Can be clustered through Erlang’s distributed
nodes

19. Nanite
• A self-assembling fabric of Ruby daemons
• Uses mappers, agents and actors
• Mappers route message requests
• Agents handle messages
• Actors offer specific services bundled into
agents

20. Nanite
• Messages can be simple pushes
• Or requests waiting for a response

21. Nanite
Agent
Agent Mapper
RabbitMQ
Agent Mapper
Agent

22. Nanite Agents

23. Nanite Agents
class Reactor
include Nanite::Actor
expose :react
def react(payload)
"reacting to message with payload: #{payload}"
end
end

24. Nanite
Nanite.request('/reactor/react', 'good acting is reacting') do |r|
pr
end

25. Nanite
• Agents register themselves and advertise their
services, constantly pinging the mappers
• Mappers remove timed-out agents
• Work distributed based on agent load

26. Nanite
• Agents can interoperate with each other
during one single message processing
• Mapper state can (and should) be stored in
redis
• Mappers live inside your application or
standalone

27. Intermediate Messages
class Reactor
include Nanite::Actor
expose :react
def react(payload)
request("/worker/work", "kill all humans",
:intermediate_handler => lambda {|res| puts res})
# working, working...
yield "I'm almost done"
# cleanup the mess
"done reacting to message with payload: #{payload}"
end
end

28. Intermediate Messages
class Worker
include Nanite::Actor
expose :work
def work(payload)
Human.all.each do |human|
human.destroy
yield "Destroyed human #{human.name}"
end
"Killed all the humans"
end
end

29. Nanite Internals
• Based on EventMachine and Ruby amqp library
(also EventMachine-based)
• Everything runs in the EventMachine loop

30. Nanite Internals
• Uses a combination of the following queues to
work
• Registration queue
• Heartbeat queue
• Advertise queue
• Agent queues

31. Nanite Internals
• Messages can be encrypted and signed
between agents and mappers
• Sends marshaled Ruby (default),YAML or
JSON
• The code internals are actually mind-boggling

32. Nanite Boos
• Poor documentation
• A not too great test suite

33. So how do we use it?
That really is a topic for another talk, isn’t it?

34. So how do we use it?
That really is a topic for another talk, isn’t it?

35. Resources
• http://skillsmatter.com/podcast/erlang/alexis-richardson-
introduction-to-rabbitmq
• http://skillsmatter.com/podcast/erlang/rabbitmq-internal-
architecture-tony-garnock-jones
• http://brainspl.at/articles/2008/10/11/merbcamp-keynote-and-
introducing-nanite
• http://github.com/ezmobius/nanite
• http://www.rabbitmq.com
• http://www.amqp.org
• http://www.slideshare.net/somic/introduction-to-amqp-messaging-
with-rabbitmq