Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (9)
Recently uploaded
Recently uploaded (20)
Background Processing with Nanite
- 1. Background Processing with Nanite By Jennifer Davis
- 2. Overview
- 3. Overview
- 6. Overview • Nanite • RabbitMQ/AMQP • Code Example
- 7. Overview • Nanite • RabbitMQ/AMQP • Code Example • Questions
- 8. Overview
- 11. Background Processing Q. Don’t we already do background processing with BackgrounDRb?
- 12. Background Processing Q. Don’t we already do background processing with BackgrounDRb? A. Yes, but it is not scalable for our needs.
- 15. Background Processing Q. Why Nanite?
- 16. Background Processing Q. Why Nanite? A. Although other methods of background processing exist, Nanite is more suited to our needs.
- 18. Nanite
- 19. Nanite
- 20. Nanite “A Self Assembling Cluster of Ruby Daemons” - Ezra Zygmuntowicz
- 21. Nanite
- 22. Nanite
- 23. Nanite Composed of Mappers and Agents
- 24. Nanite Composed of Mappers and Agents Thin Mapper
- 25. Nanite Composed of Mappers and Agents Thin Mapper Shell Mapper
- 26. Nanite Composed of Mappers and Agents Thin A Mapper M Message Queue Q (RabbitMQ) P Shell Mapper
- 27. Nanite Composed of Mappers and Agents Agent Thin Actor A Mapper Actor M Message Actor Queue Q (RabbitMQ) P Shell Mapper
- 28. Nanite Composed of Mappers and Agents Agent Thin Actor A Mapper Actor M Message Actor Queue Q (RabbitMQ) P Agent Shell Actor Mapper Actor Actor
- 29. Nanite Composed of Mappers and Agents Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
- 30. Nanite Composed of Mappers and Agents Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
- 31. Nanite
- 32. Nanite Mappers
- 33. Nanite Mappers • Can have one or many
- 34. Nanite Mappers • Can have one or many • Run within Rails or the shell
- 35. Nanite Mappers • Can have one or many • Run within Rails or the shell • Subscribed to the Mapper Exchange
- 36. Nanite Mappers • Can have one or many • Run within Rails or the shell • Subscribed to the Mapper Exchange • Adds/Removes agents based on agent availability
- 37. Nanite
- 38. Nanite
- 39. Nanite Agents
- 40. Nanite Agents • Can have one or many
- 41. Nanite Agents • Can have one or many • Ping the Mapper Exchange every @ping_time seconds
- 42. Nanite Agents • Can have one or many • Ping the Mapper Exchange every @ping_time seconds • A single agent can have many actor classes
- 43. Nanite
- 44. Nanite
- 45. Nanite Selectors for load balancing
- 46. Nanite Selectors for load balancing • :least_loaded
- 47. Nanite Selectors for load balancing • :least_loaded • :rr
- 48. Nanite Selectors for load balancing • :least_loaded • :rr • :random
- 49. Nanite Selectors for load balancing • :least_loaded • :rr • :random • :all
- 50. Nanite
- 51. Nanite Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
- 52. Nanite Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
- 53. Nanite Ping back status Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
- 54. Nanite Agent Thin Actor A A Mapper Actor M M Message Actor Queue Q Q (RabbitMQ) P P Agent Shell Actor Mapper Actor Actor
- 55. Nanite
- 58. Nanite Nanite requires • Erlang • Nanite ruby gem
- 59. Nanite Nanite requires • Erlang • Nanite ruby gem • EventMachine
- 60. Nanite Nanite requires • Erlang • Nanite ruby gem • EventMachine • AMQP
- 61. Nanite Nanite requires • Erlang • Nanite ruby gem • EventMachine • AMQP • RabbitMQ
- 62. Nanite
- 63. AMQP
- 64. AMQP
- 65. AMQP Advanced Messaging Queue Protocol
- 66. AMQP Advanced Messaging Queue Protocol • An open Internet Protocol for Business Messaging
- 67. AMQP Advanced Messaging Queue Protocol • An open Internet Protocol for Business Messaging • Provides a standard for messaging middleware
- 68. AMQP
- 69. RabbitMQ
- 70. RabbitMQ
- 71. RabbitMQ • An AMQP Messaging Broker
- 72. RabbitMQ • An AMQP Messaging Broker • Written in Erlang
- 73. RabbitMQ • An AMQP Messaging Broker • Written in Erlang • Uses the OTP (Open Telecom Platform) system
- 74. RabbitMQ • An AMQP Messaging Broker • Written in Erlang • Uses the OTP (Open Telecom Platform) system • Is stable - Achieved 99.9999999% availability
- 75. RabbitMQ
- 76. Code Example
- 77. Review
- 78. Review • Why background processing is important to us • What Nanite is and how to use it • A little background on RabbitMQ
- 79. Review
- 80. Review • Why background processing is important to us • What Nanite is and how to use it • A little background on RabbitMQ
- 81. Resources
- 82. Resources • Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/ nanite/tree/master/
- 83. Resources • Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/ nanite/tree/master/ • George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/ videos/nanite/
- 84. Resources • Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/ nanite/tree/master/ • George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/ videos/nanite/ • RabbitMQ - Open Source Enterprise Messaging at http:// www.rabbitmq.com/
- 85. Resources • Ezra Zygmuntowicz’s Nanite on Github at http://github.com/ezmobius/ nanite/tree/master/ • George Palmer’s RubyManor Nanite presentation at http://rubymanor.org/ videos/nanite/ • RabbitMQ - Open Source Enterprise Messaging at http:// www.rabbitmq.com/ • AMQP - Advanced Message Queuing Protocol at http://jira.amqp.org/ confluence/display/AMQP/Advanced+Message+Queuing+Protocol
- 86. Resources
- 87. Questions?
Editor's Notes
- BackgrounDRb BackgroundJob Starling with Workling
- BackgrounDRb BackgroundJob Starling with Workling
- BackgrounDRb BackgroundJob Starling with Workling
- It’s highly scalable and provides failover. Workers can easily be removed and added therefore adding to the reliability and availability of the application. It relies on message queueing and provides load balancing.
- It’s highly scalable and provides failover. Workers can easily be removed and added therefore adding to the reliability and availability of the application. It relies on message queueing and provides load balancing.
- The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
- The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
- The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
- The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
- The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
- The control nodes for Nanite Right now we only have one All mappers are copied the ping times from the agents
- The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
- The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
- The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
- The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
- The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
- The default status advertised is the load average as a float. Used for determining the default request dispatching based on the least loaded server. The status can be changed as long as it’s something that’s comparable Selectors include :rr, :random, :all, :least_loaded (default) Return state - default is the load of the server they’re currently running on
- By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner. The \"exchange\" receives messages from publisher applications and routes these to \"message queues\", based on arbitrary criteria - usually message properties or content. The \"message queue\" stores messages until they can be safely processed by a consuming client application (or multiple applications). The \"binding\" defines the relationship between a message queue and an exchange and provides the message routing criteria. Standard proposed by industrial banks High performance Nanite implements AMQP
- By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner. The \"exchange\" receives messages from publisher applications and routes these to \"message queues\", based on arbitrary criteria - usually message properties or content. The \"message queue\" stores messages until they can be safely processed by a consuming client application (or multiple applications). The \"binding\" defines the relationship between a message queue and an exchange and provides the message routing criteria. Standard proposed by industrial banks High performance Nanite implements AMQP
- By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner. The \"exchange\" receives messages from publisher applications and routes these to \"message queues\", based on arbitrary criteria - usually message properties or content. The \"message queue\" stores messages until they can be safely processed by a consuming client application (or multiple applications). The \"binding\" defines the relationship between a message queue and an exchange and provides the message routing criteria. Standard proposed by industrial banks High performance Nanite implements AMQP
- By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner. The \"exchange\" receives messages from publisher applications and routes these to \"message queues\", based on arbitrary criteria - usually message properties or content. The \"message queue\" stores messages until they can be safely processed by a consuming client application (or multiple applications). The \"binding\" defines the relationship between a message queue and an exchange and provides the message routing criteria. Standard proposed by industrial banks High performance Nanite implements AMQP