Restate: Event-driven Asynchronous Services, Easy as Synchronous RPC
"Event-driven services are a popular pattern in microservice architectures. Compared to RPC services, they are typically more resilient (especially when using durable logs, like Kafka), efficient, and scalable. Plus they lend themselves to advanced techniques like bootstrapping/reinstating the service from an existing event log. But writing event-driven services is much harder than writing RPC-based services: Instead of writing simple request/reply handlers, developers must express logic as asynchronous event-handling logic, introduce complex state machines (and handle concerns of their consistency and persistence). Logic gets spread logic across different event handlers. Applications need to be aware of subscriptions, hot partitions, slow re-balancing, head-of-the-line waiting issues, etc. Many companies that adopt event-driven services find themselves building extensive tools to support dead letter queues, delayed events, and handle retries/recovery/idempotency. We are developing Restate, to make event-driven applications as easy as RPC-based application. Developers write RPC-based services and Restate adds lightweight durable execution to the handlers (making them suspendable and allowing them to recover partial progress), persistent communication, and request ordering. The result is that code written as simple request/response RPCs executes with the same resilience and efficiency as an async state machine driven by events from a persistent queue, but remains easy to write, reason about, debug, and observe. Restate natively integrates with Kafka, allowing users to directly process events of Kafka topics with Restate durable async/await programming model. We believe that Restate complements the current landscape of event-processing systems, by radically simplifying transactional event-driven applications that require complex control flow or interact with many external systems."
Recommended
Recommended
More Related Content
Similar to Restate: Event-driven Asynchronous Services, Easy as Synchronous RPC
Similar to Restate: Event-driven Asynchronous Services, Easy as Synchronous RPC (20)
More from HostedbyConfluent
More from HostedbyConfluent (20)
Recently uploaded
Recently uploaded (20)
Restate: Event-driven Asynchronous Services, Easy as Synchronous RPC
- 1. 1 Event-driven apps… … the easy way Stephan Ewen https://restate.dev/
- 2. A.k.a. Durable Execution (Durable Event Handlers) - Stream Processing’s curious twin with a special taste for event-driven apps & microservices 2
- 3. Event-driven apps - aren’t they great? Event-driven is totally 1337 h3x0r all the way. Until you need to… ● enrich events with slow API calls. ○ Head of the line waiting. ● Delay an event. ○ Custom state, time management ● You need to join events. ○ Database, or go stream processing. ● You call multiple downstream APIs ○ Workflows? 3
- 4. 4 Code sample 1 Updating systems that respond slowly
- 5. Kafka Consumers poll (pull) events 5
- 7. 7 Code sample 2 Updating multiple systems per event
- 8. A flow of dependent actions 8 Create Empty Role Entry Assign to User User-id Role UUID Apply permissions Auth Service User Service
- 9. The Stream processing approach 9 Create Role Apply permissions Assign to User
- 10. The Stream processing approach 10 Create Role Apply permissions Assign to User Knuth Durability between steps Knuth has completed step 2 and not yet step 3
- 11. Durable Execution / Journaling Approach 11 Create Role Apply permissions Assign to User Invocation Journal Invocation Journal Stored as nested event stream
- 12. 12 Code sample 3 Putting back an unprocessable event
- 13. Suspending and replaying the journal 13 First attempt Sleep Stop Execution (return partial journal)
- 14. Suspending and replaying the journal 14 First attempt Sleep Second attempt Replay Finish
- 15. 15 Event consumers are a very low level abstraction to program. Durable Execution is a higher level tool (like Stream Processing), and very convenient for transactional use cases and complex control flow.
- 16. Request / Response with events 16 Start Send Event Awaiti Resp- onse End Process response Reload / validate state State Store Receiver
- 17. 17 Code sample 4 Event-based req/resp in Restate
- 18. 18 Durable execution (Restate-flavored) helps to build applications that execute event-driven, but without actually working with events /consumers/state-machines directly.
- 19. A quick into to Restate 19
- 20. 20 Distributed durable versions of your everyday building blocks. Functions/Services that handle retries, recovery, asynchrony, idempotency. Durable execution Virtual Objects Persistent state directly in your objects with a simple concurrency model. Fault-tolerant communication across services, processes, and time. Durable promises/rpc A distributed programming model…
- 21. … for a variety of use cases. 21 Durable Execution & Workflows-as-code Durable Promises and Signals Event-driven Applications MicroService Orchestration Persistent State Machines Async Tasks, Timers, Schedulers
- 22. … or as Event Broker 22 Service Service Service Restate is your event broker and reverse proxy turns HTTP- or gRPC services into durable handlers, workflows, objects. …
- 23. 23 The realm of stream processing: Flink, Kafka Streams, Materialize, and friends
- 25. The dangers of not getting this right? 25
- 26. 26 Code sample 5 Durable Promises / Slots
- 27. How far can you go? 27
- 29. Order status Restaurant PoS System Payment Provider driver customer Delivery Driver pool order API GW driver-updates Driver digital twin Order workflow https://kafka2024.restate.dev/ Demo: order processing
- 30. 30 https://restate.dev/ @restatedev /company/restatedev Resilient, consistent, and scalable applications. As functions & services, in your existing infra. Just code - no DSLs, queues, workflow managers, etc. On FaaS, K8s, servers, containers. On-prem or in the cloud.