SlideShare a Scribd company logo
1 of 32
Download to read offline
Event Driven Architectures
Avinash Ramineni
Agenda
•
•
•
•
•
•
•

What is Event Driven Architecture
How is EDA related to SOA
Event Capture/Generation
Event Processing
Event Delivery Guarantees
Complex Event Processing
Events, Internet of Things and Big Data
Event
• Significant change in the state of an Object
–
–
–
–
–

Bank Transaction
Students grade gets posted
New Order gets created
Customer Profile gets updated
Change in Heart Beat

• Characteristics of an Event
– Events are broadcast
– Communications are asynchronous
– Events are fine grained
Event Driven Architecture (EDA)
• Handling of the Events as they occur and the transfer of
events among the systems in a loosely coupled way
• Enables Situation Awareness and enables sense-and-respond
behavior
Event Driven Architecture (EDA)
• Characteristics of EDA
–
–
–
–
–
–
–

Producer/Consumers are not aware of each other
Little or no statefullness
Distributed
Loosely Coupled
Platform and Language Independent
Reliable
Fast and Highly Scalable
EDA extension of SOA
Component
software done
right

RPC-style SOA

Computing
with event
objects

SOA
Web-oriented
architecture
(WOA)

Web Architecture

REST, URIs and
HTTP

Simple Web
sites

Event-driven
SOA

Event Processing
Non-SOA EDA
and CEP
EDA extension to SOA
• EDA similar to SOA facilitates Agility by using Modular Design
and Separation of Concerns
• EDA fits well into organizations that have some SOA
infrastructure in place
• EDA adds another dimension / Possibilities to SOA
– Events are generally stateless and do not have much content
• While processing the events that do not have much data in them, the consuming
applications call the services exposed to get the actual data

• Publish – Subscribe Model
– Similar to Asynchronous Messaging Architectures
How to generate Events
• Application Generated
– Application generates an event when a business transaction
completes
– Events generated by parsing the access logs/ server logs
– Polling web pages
– capturing the events from the services

• Database Generated
– Database Trigger generated Events
• On insert/update/delete
– During database replication
Fallacies of Distributed Systems
• The network is reliable
• Latency is zero

• Bandwidth is infinite
• The network is secure
• Topology does not change

• Transport cost is zero
• Network is homogeneous
Usecase: 1
• Send an email when a customers profile gets updated
Customer Profile
– Assumptions:
• There is an Application A that provides a updateProfileService
• There is an Application B that provides a sendEmailService
• All the Events are delivered with Once and Only Once SLA and the Events
are processed
Profile Updated Event
Event
A
Channel

Update DB
B
DB
Considerations while generating Events
• The overhead involved in capturing events must be as low as
possible to avoid impairing the performance of a business
transaction.
• Business Transaction and the Event generation needs to be in
the same transaction
– 1,2 needs to be part of a same transaction
• Both 1 and 2 should succeed or both should fail

2. Profile Updated Event

A

Event
Channel

1. Update DB
B
DB
Usecase: 1 – Web service
• Successful Update of Profile -- Call a service to publish a
Profile update Event
• Can a database transaction and Web service call be made
atomic?
try{

try{

1. Update Profile on DB
2. Call web service to publish profile
updated event
} catch(Exception e)
{
rollback();
}
commit;

1. Update Profile on DB
} catch(Exception e)
{
rollback();
}
commit();
2. Call web service to publish profile
updated event

• Can Compensating Transaction work ?
Usecase: 1 – JMS
• Successful Update of Profile -- Put a message on to Message
Broker
A

2. Profile Updated Event
(JMS message)

Event Channel

1. Update DB

DB

B

• Possible to make 1 and 2 into a single transaction but it
requires – Distributed Transaction.
Distributed/XA Transactions (1)
• Follows 2 Phase Commit protocol
• Very Chatty protocol and does a lot of logging to be able to
recover from any failure scenario.
• Too much overhead to 99.9% of the cases to handle less than
0.1% of cases
• Increases the chances for deadlocks on the database.
• Lowers the overall performance of the system.
• Bane of scalability. Grinds the entire system to halt by adding
overhead to each and every transaction.
Distributed/XA Transactions (2)
•
•
•
•

Availability of the Systems goes down.
XA Configuration is complicated
Difficult to test.
Many People tend to believe that using JTA implementation of
transaction manager will take care of a XA transactions
• Many a time people end up using JTA Manager even while
dealing with single resource.
Usecase: 1 – Local Database
• Successful Update of Profile -- Put a message on to Message
Broker
A

1. Update DB

Event Channel

2. Store Event
3. Batch process to send
events to Event Channel

DB

B

• 1 and 2 are in same transaction. 3 can be retried multiple
times till they succeed.
Usecase: 1 – Local JMS
• Successful Update of Profile -- Put a message on to Message
Broker
queue

2
A

3. Publish to Event
Channel(JMS message)

Event Channel

Send Event to Local Queue

1. Update DB

DB

B

• Possible to make 1 and 2 into a single transaction without a
Distributed Transaction as long as Queue is backed by a same
database
Usecase: 1 – Other Ways
• Successful Update of Profile -- Put a message on to Message
Broker / Call a web service to publish an Event
• Have a Reconciliation Process in place to verify that there is
an event generated for every Business Transaction
– Have a unique Id (may be stored in DB) along with update and use that
Id as Correlation Id with the Events Archive
– If it events don’t tally up, recreate those events

• Use the DB triggers to generate events
– Write to a different local table
– Call a web service to send the event to event channel (have seen
people do that)
Event Delivery
• Event Delivery Guarantees
– Reliability
• At least Once
– Duplicate events can be delivered

• At most Once
– Some events can be lost

• Once and only Once
– Each is delivered Exactly once

– Order of Delivery
• In Order Guaranteed
• In Order not Guaranteed
Considerations while Processing
Events
• Processing an Event and associated Business Transaction
needs to be in the same transaction
• 3,2 needs to be part of a same transaction
• Both 3 and 2 should succeed or both should fail

1. Profile Updated Event
Event
Channel

2. Send Email
B

3. Acknowledge Event
Process success
Usecase: 1 – Consumer (1)
• On Profile Update Event -- Call a service to send Email and on
success acknowledge process successful
onMessage
try {
sendEmail();
jms.commit()
}
catch (Exception e) {
jms.rollback()
}

• What if jms.commit() fails ??
Usecase: 1 – Consumer (2)
• If Jms.commit() fails , message gets delivered again
onMessage
try {
if I have not processed this message successfully before {
do some stuff in the database / Idempotency Logic /JMSRelivered flag
jdbc.commit;
}
jms.commit()
}
catch (Exception e) {
jms.rollback()
}
Work Around for At-Least-Once
• Idempotent Operation
– An idempotent operations means that the result of a successful
performed request is independent of the number of times it is
executed.

• Design consumer to be an Idempotent Consumer
– By Understanding the Business rules it can be achieved relatively easy

• Good Practice to make all the write operations Idempotent !!
• Idempotency logic can be built on Event Ids, Business Ids –
orderId, customerId, etc based on the Business scenario
What If In-order Delivery is required
• In-order delivery of events
– Limit the number of event publisher/consumer instance
for these specific events to 1
• Brings up questions like if one of the event is having an issue – all
the events of that event type can come to a stand still

– AVOID the need for In-order Delivery of Events
• By limiting the content in the Event , we can avoid a lot of cases
where In-order delivery is required
• Using Message Selectors to select and aggregate the events can
solve a few of the cases
Usecase: 2
• Student Information System (SIS) maintain a list of students in
a course roster. The students can be added or dropped from
the course roster. SIS sends an event to Learning Management
System (LMS) when the roster gets updated.
• Lets us assume that SIS is sending the course roster as part of
the event.
• Processing of this Event requires the Roster Change events to
be delivered in order to maintain the data from getting
corrupted.
Usecase: 2 – Requires In-Order

1. Roster Change Event
SIS

Event
Channel
2. Deliver Roster
Change Event
LMS

DB

LMS DB
Usecase: 2 - Solution
• Remove the roster data from the event and just pass a roster
Id
• On Delivery of the Event , the LMS system would use the
rosterId from the event, makes a service call to the source
system (SIS) to get the latest Roster data and builds the
classes in LMS
• Even if the events are delivered out of order , the roster data
on LMS will be always be the latest.
• By making the processor on LMS idempotent, we can even
avoid the database update
Usecase: 2

1. Roster Change Event
SIS

Event
Channel
2. Deliver Roster
Change Event

3. Get the latest roster
for the Id
LMS
DB

LMS DB

4. Update the LMS
database
Best Practices
• Make Event Publishers responsible for making sure that
events are generated for every Business transaction and are
published to the event channel
• Make Event Consumers responsible for making sure that they
react to the Stimuli from the event
• Dashboards in place to make sure failed Events are detected
and acted upon
• Collects Stats to see if the events need to be made finer or
coarser
• Maintain Event Catalogs and self service capabilities to
subscribe to events
Complex Event Processing (CEP)
• CEP tap the events happening with in an organization and
provide a situation awareness and enable better and faster
decisions
– Identify complicated events that are inferred from event pattern
detection, event pattern interpretors and so on
– Ex: Monitoring – capture events from ticket system, network load,
performance and send out notification bring up new servers into the
pool

Traditional BI finds "needles
in haystacks."

Event-driven CEP finds
"needles" in continuously
arriving streams of "hay."
Events ,Big Data and Internet of Things
• With Big Data hype and with the “capture-it-all“ approach and
IoT , EDA and Event Driven programming will get a huge boost
in the coming years.
• Events and EDA due to their characteristics will be very well
suited for Ubiquitous Computing
• We might start seeing an advent of Complex Event
Applications by being used by various gadgets
– The concepts of EDA and Events will be a perfect Match for gadgets like Basis ,FIT
Questions ?
Industry Experience

avinash@clairvoyantsoft.com
Twitter:@avinashramineni

More Related Content

What's hot

Microservices, Kubernetes, and Application Modernization Done Right
Microservices, Kubernetes, and Application Modernization Done RightMicroservices, Kubernetes, and Application Modernization Done Right
Microservices, Kubernetes, and Application Modernization Done RightLightbend
 
CQRS + Event Sourcing
CQRS + Event SourcingCQRS + Event Sourcing
CQRS + Event SourcingMike Bild
 
Practical introduction to DDD, CQRS and Event Sourcing
Practical introduction to DDD, CQRS and Event SourcingPractical introduction to DDD, CQRS and Event Sourcing
Practical introduction to DDD, CQRS and Event SourcingDennis Doomen
 
Moving Legacy Apps to Cloud: How to Avoid Risk
Moving Legacy Apps to Cloud: How to Avoid RiskMoving Legacy Apps to Cloud: How to Avoid Risk
Moving Legacy Apps to Cloud: How to Avoid RiskCloverDX
 
A practical introduction to Event Sourcing and CQRS
A practical introduction to Event Sourcing and CQRSA practical introduction to Event Sourcing and CQRS
A practical introduction to Event Sourcing and CQRSRobert Lemke
 
Kafka and event driven architecture -apacoug20
Kafka and event driven architecture -apacoug20Kafka and event driven architecture -apacoug20
Kafka and event driven architecture -apacoug20Vinay Kumar
 
Event Sourcing & CQRS, Kafka, Rabbit MQ
Event Sourcing & CQRS, Kafka, Rabbit MQEvent Sourcing & CQRS, Kafka, Rabbit MQ
Event Sourcing & CQRS, Kafka, Rabbit MQAraf Karsh Hamid
 
Dynamic Infrastructure and The Cloud
Dynamic Infrastructure and The CloudDynamic Infrastructure and The Cloud
Dynamic Infrastructure and The CloudNew Relic
 
Migration to Microsoft Online Services from Exchange and Non-Microsoft Platforms
Migration to Microsoft Online Services from Exchange and Non-Microsoft PlatformsMigration to Microsoft Online Services from Exchange and Non-Microsoft Platforms
Migration to Microsoft Online Services from Exchange and Non-Microsoft PlatformsBitTitan
 
Transforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming DataTransforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming Dataconfluent
 
Customer Driven DevOps at Work: Operating the Digital Turnstile [FutureStack1...
Customer Driven DevOps at Work: Operating the Digital Turnstile [FutureStack1...Customer Driven DevOps at Work: Operating the Digital Turnstile [FutureStack1...
Customer Driven DevOps at Work: Operating the Digital Turnstile [FutureStack1...New Relic
 
Cloud computing 101 with amazon web service
Cloud computing 101 with amazon web serviceCloud computing 101 with amazon web service
Cloud computing 101 with amazon web serviceDr. Ketan Parmar
 
Migration Tools: The True Cost of Free
Migration Tools: The True Cost of FreeMigration Tools: The True Cost of Free
Migration Tools: The True Cost of FreeOpenText Portfolio
 
Simplify Cloud Migration to AWS with RISC Network’s Complete App Analysis
Simplify Cloud Migration  to  AWS with RISC Network’s Complete App AnalysisSimplify Cloud Migration  to  AWS with RISC Network’s Complete App Analysis
Simplify Cloud Migration to AWS with RISC Network’s Complete App AnalysisRISC Networks
 
AWS Customer Presentation - Thomson Reuters - Delivering on the Promise of Di...
AWS Customer Presentation - Thomson Reuters - Delivering on the Promise of Di...AWS Customer Presentation - Thomson Reuters - Delivering on the Promise of Di...
AWS Customer Presentation - Thomson Reuters - Delivering on the Promise of Di...Amazon Web Services
 
Migrating Legacy Applications to AWS Cloud: Strategies and Challenges
Migrating Legacy Applications to AWS Cloud: Strategies and ChallengesMigrating Legacy Applications to AWS Cloud: Strategies and Challenges
Migrating Legacy Applications to AWS Cloud: Strategies and ChallengesOSSCube
 
Event Sourcing from the Trenches (with examples from .NET)
Event Sourcing from the Trenches (with examples from .NET)Event Sourcing from the Trenches (with examples from .NET)
Event Sourcing from the Trenches (with examples from .NET)Dennis Doomen
 
How to Get Cloud Architecture and Design Right the First Time
How to Get Cloud Architecture and Design Right the First TimeHow to Get Cloud Architecture and Design Right the First Time
How to Get Cloud Architecture and Design Right the First TimeDavid Linthicum
 
Microservices pattern language (microxchg microxchg2016)
Microservices pattern language (microxchg microxchg2016)Microservices pattern language (microxchg microxchg2016)
Microservices pattern language (microxchg microxchg2016)Chris Richardson
 
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...Skelton Thatcher Consulting Ltd
 

What's hot (20)

Microservices, Kubernetes, and Application Modernization Done Right
Microservices, Kubernetes, and Application Modernization Done RightMicroservices, Kubernetes, and Application Modernization Done Right
Microservices, Kubernetes, and Application Modernization Done Right
 
CQRS + Event Sourcing
CQRS + Event SourcingCQRS + Event Sourcing
CQRS + Event Sourcing
 
Practical introduction to DDD, CQRS and Event Sourcing
Practical introduction to DDD, CQRS and Event SourcingPractical introduction to DDD, CQRS and Event Sourcing
Practical introduction to DDD, CQRS and Event Sourcing
 
Moving Legacy Apps to Cloud: How to Avoid Risk
Moving Legacy Apps to Cloud: How to Avoid RiskMoving Legacy Apps to Cloud: How to Avoid Risk
Moving Legacy Apps to Cloud: How to Avoid Risk
 
A practical introduction to Event Sourcing and CQRS
A practical introduction to Event Sourcing and CQRSA practical introduction to Event Sourcing and CQRS
A practical introduction to Event Sourcing and CQRS
 
Kafka and event driven architecture -apacoug20
Kafka and event driven architecture -apacoug20Kafka and event driven architecture -apacoug20
Kafka and event driven architecture -apacoug20
 
Event Sourcing & CQRS, Kafka, Rabbit MQ
Event Sourcing & CQRS, Kafka, Rabbit MQEvent Sourcing & CQRS, Kafka, Rabbit MQ
Event Sourcing & CQRS, Kafka, Rabbit MQ
 
Dynamic Infrastructure and The Cloud
Dynamic Infrastructure and The CloudDynamic Infrastructure and The Cloud
Dynamic Infrastructure and The Cloud
 
Migration to Microsoft Online Services from Exchange and Non-Microsoft Platforms
Migration to Microsoft Online Services from Exchange and Non-Microsoft PlatformsMigration to Microsoft Online Services from Exchange and Non-Microsoft Platforms
Migration to Microsoft Online Services from Exchange and Non-Microsoft Platforms
 
Transforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming DataTransforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming Data
 
Customer Driven DevOps at Work: Operating the Digital Turnstile [FutureStack1...
Customer Driven DevOps at Work: Operating the Digital Turnstile [FutureStack1...Customer Driven DevOps at Work: Operating the Digital Turnstile [FutureStack1...
Customer Driven DevOps at Work: Operating the Digital Turnstile [FutureStack1...
 
Cloud computing 101 with amazon web service
Cloud computing 101 with amazon web serviceCloud computing 101 with amazon web service
Cloud computing 101 with amazon web service
 
Migration Tools: The True Cost of Free
Migration Tools: The True Cost of FreeMigration Tools: The True Cost of Free
Migration Tools: The True Cost of Free
 
Simplify Cloud Migration to AWS with RISC Network’s Complete App Analysis
Simplify Cloud Migration  to  AWS with RISC Network’s Complete App AnalysisSimplify Cloud Migration  to  AWS with RISC Network’s Complete App Analysis
Simplify Cloud Migration to AWS with RISC Network’s Complete App Analysis
 
AWS Customer Presentation - Thomson Reuters - Delivering on the Promise of Di...
AWS Customer Presentation - Thomson Reuters - Delivering on the Promise of Di...AWS Customer Presentation - Thomson Reuters - Delivering on the Promise of Di...
AWS Customer Presentation - Thomson Reuters - Delivering on the Promise of Di...
 
Migrating Legacy Applications to AWS Cloud: Strategies and Challenges
Migrating Legacy Applications to AWS Cloud: Strategies and ChallengesMigrating Legacy Applications to AWS Cloud: Strategies and Challenges
Migrating Legacy Applications to AWS Cloud: Strategies and Challenges
 
Event Sourcing from the Trenches (with examples from .NET)
Event Sourcing from the Trenches (with examples from .NET)Event Sourcing from the Trenches (with examples from .NET)
Event Sourcing from the Trenches (with examples from .NET)
 
How to Get Cloud Architecture and Design Right the First Time
How to Get Cloud Architecture and Design Right the First TimeHow to Get Cloud Architecture and Design Right the First Time
How to Get Cloud Architecture and Design Right the First Time
 
Microservices pattern language (microxchg microxchg2016)
Microservices pattern language (microxchg microxchg2016)Microservices pattern language (microxchg microxchg2016)
Microservices pattern language (microxchg microxchg2016)
 
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
 

Similar to Event Driven Architectures

Event Driven Architectures - Phoenix Java Users Group 2013
Event Driven Architectures - Phoenix Java Users Group 2013Event Driven Architectures - Phoenix Java Users Group 2013
Event Driven Architectures - Phoenix Java Users Group 2013clairvoyantllc
 
Event-Driven Architecture (EDA)
Event-Driven Architecture (EDA)Event-Driven Architecture (EDA)
Event-Driven Architecture (EDA)WSO2
 
Data Microservices with Spring Cloud
Data Microservices with Spring CloudData Microservices with Spring Cloud
Data Microservices with Spring CloudOrkhan Gasimov
 
Event Driven Architecture – Enabling Microservices
Event Driven Architecture – Enabling MicroservicesEvent Driven Architecture – Enabling Microservices
Event Driven Architecture – Enabling MicroservicesBradley Irby
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Patterns of Distributed Application Design
Patterns of Distributed Application DesignPatterns of Distributed Application Design
Patterns of Distributed Application DesignOrkhan Gasimov
 
Event driven architecure
Event driven architecureEvent driven architecure
Event driven architecureTouraj Ebrahimi
 
Event Driven Architectures - Net Conf UY 2018
Event Driven Architectures - Net Conf UY 2018Event Driven Architectures - Net Conf UY 2018
Event Driven Architectures - Net Conf UY 2018Bradley Irby
 
apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...
apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...
apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...apidays
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Patterns of Distributed Application Design
Patterns of Distributed Application DesignPatterns of Distributed Application Design
Patterns of Distributed Application DesignGlobalLogic Ukraine
 
Introducing Ironstream Support for ServiceNow Event Management
Introducing Ironstream Support for ServiceNow Event Management Introducing Ironstream Support for ServiceNow Event Management
Introducing Ironstream Support for ServiceNow Event Management Precisely
 
8 application servers_v2
8 application servers_v28 application servers_v2
8 application servers_v2ashish61_scs
 
apidays New York 2023 - Why Finance needs Asychronous APIs, Nicholas Goodman,...
apidays New York 2023 - Why Finance needs Asychronous APIs, Nicholas Goodman,...apidays New York 2023 - Why Finance needs Asychronous APIs, Nicholas Goodman,...
apidays New York 2023 - Why Finance needs Asychronous APIs, Nicholas Goodman,...apidays
 
Kafka and event driven architecture -og yatra20
Kafka and event driven architecture -og yatra20Kafka and event driven architecture -og yatra20
Kafka and event driven architecture -og yatra20Vinay Kumar
 
Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates Uncovered
Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates UncoveredRuslan Belkin And Sean Dawson on LinkedIn's Network Updates Uncovered
Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates UncoveredLinkedIn
 
Actors or Not: Async Event Architectures
Actors or Not: Async Event ArchitecturesActors or Not: Async Event Architectures
Actors or Not: Async Event ArchitecturesYaroslav Tkachenko
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 

Similar to Event Driven Architectures (20)

Event Driven Architectures - Phoenix Java Users Group 2013
Event Driven Architectures - Phoenix Java Users Group 2013Event Driven Architectures - Phoenix Java Users Group 2013
Event Driven Architectures - Phoenix Java Users Group 2013
 
Event-Driven Architecture (EDA)
Event-Driven Architecture (EDA)Event-Driven Architecture (EDA)
Event-Driven Architecture (EDA)
 
Data Microservices with Spring Cloud
Data Microservices with Spring CloudData Microservices with Spring Cloud
Data Microservices with Spring Cloud
 
Event Driven Architecture – Enabling Microservices
Event Driven Architecture – Enabling MicroservicesEvent Driven Architecture – Enabling Microservices
Event Driven Architecture – Enabling Microservices
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream Processing
 
Micro service architecture
Micro service architecture  Micro service architecture
Micro service architecture
 
Patterns of Distributed Application Design
Patterns of Distributed Application DesignPatterns of Distributed Application Design
Patterns of Distributed Application Design
 
Event driven architecure
Event driven architecureEvent driven architecure
Event driven architecure
 
Event Driven Architectures - Net Conf UY 2018
Event Driven Architectures - Net Conf UY 2018Event Driven Architectures - Net Conf UY 2018
Event Driven Architectures - Net Conf UY 2018
 
apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...
apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...
apidays LIVE Jakarta - Building an Event-Driven Architecture by Harin Honesty...
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream Processing
 
Patterns of Distributed Application Design
Patterns of Distributed Application DesignPatterns of Distributed Application Design
Patterns of Distributed Application Design
 
Introducing Ironstream Support for ServiceNow Event Management
Introducing Ironstream Support for ServiceNow Event Management Introducing Ironstream Support for ServiceNow Event Management
Introducing Ironstream Support for ServiceNow Event Management
 
8 application servers_v2
8 application servers_v28 application servers_v2
8 application servers_v2
 
apidays New York 2023 - Why Finance needs Asychronous APIs, Nicholas Goodman,...
apidays New York 2023 - Why Finance needs Asychronous APIs, Nicholas Goodman,...apidays New York 2023 - Why Finance needs Asychronous APIs, Nicholas Goodman,...
apidays New York 2023 - Why Finance needs Asychronous APIs, Nicholas Goodman,...
 
L21 scalability
L21 scalabilityL21 scalability
L21 scalability
 
Kafka and event driven architecture -og yatra20
Kafka and event driven architecture -og yatra20Kafka and event driven architecture -og yatra20
Kafka and event driven architecture -og yatra20
 
Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates Uncovered
Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates UncoveredRuslan Belkin And Sean Dawson on LinkedIn's Network Updates Uncovered
Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates Uncovered
 
Actors or Not: Async Event Architectures
Actors or Not: Async Event ArchitecturesActors or Not: Async Event Architectures
Actors or Not: Async Event Architectures
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream Processing
 

More from Avinash Ramineni

Simplifying the data privacy governance quagmire building automated privacy ...
Simplifying the data privacy governance quagmire  building automated privacy ...Simplifying the data privacy governance quagmire  building automated privacy ...
Simplifying the data privacy governance quagmire building automated privacy ...Avinash Ramineni
 
Winning the war on data breaches in a changing data landscape
Winning the war on data breaches in a changing data landscapeWinning the war on data breaches in a changing data landscape
Winning the war on data breaches in a changing data landscapeAvinash Ramineni
 
Autonomous Security: Using Big Data, Machine Learning and AI to Fix Today's S...
Autonomous Security: Using Big Data, Machine Learning and AI to Fix Today's S...Autonomous Security: Using Big Data, Machine Learning and AI to Fix Today's S...
Autonomous Security: Using Big Data, Machine Learning and AI to Fix Today's S...Avinash Ramineni
 
Building zero data loss pipelines with apache kafka
Building zero data loss pipelines with apache kafkaBuilding zero data loss pipelines with apache kafka
Building zero data loss pipelines with apache kafkaAvinash Ramineni
 
Effectively deploying hadoop to the cloud
Effectively  deploying hadoop to the cloudEffectively  deploying hadoop to the cloud
Effectively deploying hadoop to the cloudAvinash Ramineni
 
Practical guide to architecting data lakes - Avinash Ramineni - Phoenix Data...
Practical guide to architecting data lakes -  Avinash Ramineni - Phoenix Data...Practical guide to architecting data lakes -  Avinash Ramineni - Phoenix Data...
Practical guide to architecting data lakes - Avinash Ramineni - Phoenix Data...Avinash Ramineni
 
MongoDB Replication fundamentals - Desert Code Camp - October 2014
MongoDB Replication fundamentals - Desert Code Camp - October 2014MongoDB Replication fundamentals - Desert Code Camp - October 2014
MongoDB Replication fundamentals - Desert Code Camp - October 2014Avinash Ramineni
 
HBase from the Trenches - Phoenix Data Conference 2015
HBase from the Trenches - Phoenix Data Conference 2015HBase from the Trenches - Phoenix Data Conference 2015
HBase from the Trenches - Phoenix Data Conference 2015Avinash Ramineni
 
Strata+Hadoop World NY 2016 - Avinash Ramineni
Strata+Hadoop World NY 2016 - Avinash RamineniStrata+Hadoop World NY 2016 - Avinash Ramineni
Strata+Hadoop World NY 2016 - Avinash RamineniAvinash Ramineni
 
Log analysis using Logstash,ElasticSearch and Kibana
Log analysis using Logstash,ElasticSearch and KibanaLog analysis using Logstash,ElasticSearch and Kibana
Log analysis using Logstash,ElasticSearch and KibanaAvinash Ramineni
 

More from Avinash Ramineni (10)

Simplifying the data privacy governance quagmire building automated privacy ...
Simplifying the data privacy governance quagmire  building automated privacy ...Simplifying the data privacy governance quagmire  building automated privacy ...
Simplifying the data privacy governance quagmire building automated privacy ...
 
Winning the war on data breaches in a changing data landscape
Winning the war on data breaches in a changing data landscapeWinning the war on data breaches in a changing data landscape
Winning the war on data breaches in a changing data landscape
 
Autonomous Security: Using Big Data, Machine Learning and AI to Fix Today's S...
Autonomous Security: Using Big Data, Machine Learning and AI to Fix Today's S...Autonomous Security: Using Big Data, Machine Learning and AI to Fix Today's S...
Autonomous Security: Using Big Data, Machine Learning and AI to Fix Today's S...
 
Building zero data loss pipelines with apache kafka
Building zero data loss pipelines with apache kafkaBuilding zero data loss pipelines with apache kafka
Building zero data loss pipelines with apache kafka
 
Effectively deploying hadoop to the cloud
Effectively  deploying hadoop to the cloudEffectively  deploying hadoop to the cloud
Effectively deploying hadoop to the cloud
 
Practical guide to architecting data lakes - Avinash Ramineni - Phoenix Data...
Practical guide to architecting data lakes -  Avinash Ramineni - Phoenix Data...Practical guide to architecting data lakes -  Avinash Ramineni - Phoenix Data...
Practical guide to architecting data lakes - Avinash Ramineni - Phoenix Data...
 
MongoDB Replication fundamentals - Desert Code Camp - October 2014
MongoDB Replication fundamentals - Desert Code Camp - October 2014MongoDB Replication fundamentals - Desert Code Camp - October 2014
MongoDB Replication fundamentals - Desert Code Camp - October 2014
 
HBase from the Trenches - Phoenix Data Conference 2015
HBase from the Trenches - Phoenix Data Conference 2015HBase from the Trenches - Phoenix Data Conference 2015
HBase from the Trenches - Phoenix Data Conference 2015
 
Strata+Hadoop World NY 2016 - Avinash Ramineni
Strata+Hadoop World NY 2016 - Avinash RamineniStrata+Hadoop World NY 2016 - Avinash Ramineni
Strata+Hadoop World NY 2016 - Avinash Ramineni
 
Log analysis using Logstash,ElasticSearch and Kibana
Log analysis using Logstash,ElasticSearch and KibanaLog analysis using Logstash,ElasticSearch and Kibana
Log analysis using Logstash,ElasticSearch and Kibana
 

Recently uploaded

Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServiceRenan Moreira de Oliveira
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxYounusS2
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Introduction to Quantum Computing
Introduction to Quantum ComputingIntroduction to Quantum Computing
Introduction to Quantum ComputingGDSC PJATK
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIUdaiappa Ramachandran
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 

Recently uploaded (20)

Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptx
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Introduction to Quantum Computing
Introduction to Quantum ComputingIntroduction to Quantum Computing
Introduction to Quantum Computing
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 

Event Driven Architectures

  • 2. Agenda • • • • • • • What is Event Driven Architecture How is EDA related to SOA Event Capture/Generation Event Processing Event Delivery Guarantees Complex Event Processing Events, Internet of Things and Big Data
  • 3. Event • Significant change in the state of an Object – – – – – Bank Transaction Students grade gets posted New Order gets created Customer Profile gets updated Change in Heart Beat • Characteristics of an Event – Events are broadcast – Communications are asynchronous – Events are fine grained
  • 4. Event Driven Architecture (EDA) • Handling of the Events as they occur and the transfer of events among the systems in a loosely coupled way • Enables Situation Awareness and enables sense-and-respond behavior
  • 5. Event Driven Architecture (EDA) • Characteristics of EDA – – – – – – – Producer/Consumers are not aware of each other Little or no statefullness Distributed Loosely Coupled Platform and Language Independent Reliable Fast and Highly Scalable
  • 6. EDA extension of SOA Component software done right RPC-style SOA Computing with event objects SOA Web-oriented architecture (WOA) Web Architecture REST, URIs and HTTP Simple Web sites Event-driven SOA Event Processing Non-SOA EDA and CEP
  • 7. EDA extension to SOA • EDA similar to SOA facilitates Agility by using Modular Design and Separation of Concerns • EDA fits well into organizations that have some SOA infrastructure in place • EDA adds another dimension / Possibilities to SOA – Events are generally stateless and do not have much content • While processing the events that do not have much data in them, the consuming applications call the services exposed to get the actual data • Publish – Subscribe Model – Similar to Asynchronous Messaging Architectures
  • 8. How to generate Events • Application Generated – Application generates an event when a business transaction completes – Events generated by parsing the access logs/ server logs – Polling web pages – capturing the events from the services • Database Generated – Database Trigger generated Events • On insert/update/delete – During database replication
  • 9. Fallacies of Distributed Systems • The network is reliable • Latency is zero • Bandwidth is infinite • The network is secure • Topology does not change • Transport cost is zero • Network is homogeneous
  • 10. Usecase: 1 • Send an email when a customers profile gets updated Customer Profile – Assumptions: • There is an Application A that provides a updateProfileService • There is an Application B that provides a sendEmailService • All the Events are delivered with Once and Only Once SLA and the Events are processed Profile Updated Event Event A Channel Update DB B DB
  • 11. Considerations while generating Events • The overhead involved in capturing events must be as low as possible to avoid impairing the performance of a business transaction. • Business Transaction and the Event generation needs to be in the same transaction – 1,2 needs to be part of a same transaction • Both 1 and 2 should succeed or both should fail 2. Profile Updated Event A Event Channel 1. Update DB B DB
  • 12. Usecase: 1 – Web service • Successful Update of Profile -- Call a service to publish a Profile update Event • Can a database transaction and Web service call be made atomic? try{ try{ 1. Update Profile on DB 2. Call web service to publish profile updated event } catch(Exception e) { rollback(); } commit; 1. Update Profile on DB } catch(Exception e) { rollback(); } commit(); 2. Call web service to publish profile updated event • Can Compensating Transaction work ?
  • 13. Usecase: 1 – JMS • Successful Update of Profile -- Put a message on to Message Broker A 2. Profile Updated Event (JMS message) Event Channel 1. Update DB DB B • Possible to make 1 and 2 into a single transaction but it requires – Distributed Transaction.
  • 14. Distributed/XA Transactions (1) • Follows 2 Phase Commit protocol • Very Chatty protocol and does a lot of logging to be able to recover from any failure scenario. • Too much overhead to 99.9% of the cases to handle less than 0.1% of cases • Increases the chances for deadlocks on the database. • Lowers the overall performance of the system. • Bane of scalability. Grinds the entire system to halt by adding overhead to each and every transaction.
  • 15. Distributed/XA Transactions (2) • • • • Availability of the Systems goes down. XA Configuration is complicated Difficult to test. Many People tend to believe that using JTA implementation of transaction manager will take care of a XA transactions • Many a time people end up using JTA Manager even while dealing with single resource.
  • 16. Usecase: 1 – Local Database • Successful Update of Profile -- Put a message on to Message Broker A 1. Update DB Event Channel 2. Store Event 3. Batch process to send events to Event Channel DB B • 1 and 2 are in same transaction. 3 can be retried multiple times till they succeed.
  • 17. Usecase: 1 – Local JMS • Successful Update of Profile -- Put a message on to Message Broker queue 2 A 3. Publish to Event Channel(JMS message) Event Channel Send Event to Local Queue 1. Update DB DB B • Possible to make 1 and 2 into a single transaction without a Distributed Transaction as long as Queue is backed by a same database
  • 18. Usecase: 1 – Other Ways • Successful Update of Profile -- Put a message on to Message Broker / Call a web service to publish an Event • Have a Reconciliation Process in place to verify that there is an event generated for every Business Transaction – Have a unique Id (may be stored in DB) along with update and use that Id as Correlation Id with the Events Archive – If it events don’t tally up, recreate those events • Use the DB triggers to generate events – Write to a different local table – Call a web service to send the event to event channel (have seen people do that)
  • 19. Event Delivery • Event Delivery Guarantees – Reliability • At least Once – Duplicate events can be delivered • At most Once – Some events can be lost • Once and only Once – Each is delivered Exactly once – Order of Delivery • In Order Guaranteed • In Order not Guaranteed
  • 20. Considerations while Processing Events • Processing an Event and associated Business Transaction needs to be in the same transaction • 3,2 needs to be part of a same transaction • Both 3 and 2 should succeed or both should fail 1. Profile Updated Event Event Channel 2. Send Email B 3. Acknowledge Event Process success
  • 21. Usecase: 1 – Consumer (1) • On Profile Update Event -- Call a service to send Email and on success acknowledge process successful onMessage try { sendEmail(); jms.commit() } catch (Exception e) { jms.rollback() } • What if jms.commit() fails ??
  • 22. Usecase: 1 – Consumer (2) • If Jms.commit() fails , message gets delivered again onMessage try { if I have not processed this message successfully before { do some stuff in the database / Idempotency Logic /JMSRelivered flag jdbc.commit; } jms.commit() } catch (Exception e) { jms.rollback() }
  • 23. Work Around for At-Least-Once • Idempotent Operation – An idempotent operations means that the result of a successful performed request is independent of the number of times it is executed. • Design consumer to be an Idempotent Consumer – By Understanding the Business rules it can be achieved relatively easy • Good Practice to make all the write operations Idempotent !! • Idempotency logic can be built on Event Ids, Business Ids – orderId, customerId, etc based on the Business scenario
  • 24. What If In-order Delivery is required • In-order delivery of events – Limit the number of event publisher/consumer instance for these specific events to 1 • Brings up questions like if one of the event is having an issue – all the events of that event type can come to a stand still – AVOID the need for In-order Delivery of Events • By limiting the content in the Event , we can avoid a lot of cases where In-order delivery is required • Using Message Selectors to select and aggregate the events can solve a few of the cases
  • 25. Usecase: 2 • Student Information System (SIS) maintain a list of students in a course roster. The students can be added or dropped from the course roster. SIS sends an event to Learning Management System (LMS) when the roster gets updated. • Lets us assume that SIS is sending the course roster as part of the event. • Processing of this Event requires the Roster Change events to be delivered in order to maintain the data from getting corrupted.
  • 26. Usecase: 2 – Requires In-Order 1. Roster Change Event SIS Event Channel 2. Deliver Roster Change Event LMS DB LMS DB
  • 27. Usecase: 2 - Solution • Remove the roster data from the event and just pass a roster Id • On Delivery of the Event , the LMS system would use the rosterId from the event, makes a service call to the source system (SIS) to get the latest Roster data and builds the classes in LMS • Even if the events are delivered out of order , the roster data on LMS will be always be the latest. • By making the processor on LMS idempotent, we can even avoid the database update
  • 28. Usecase: 2 1. Roster Change Event SIS Event Channel 2. Deliver Roster Change Event 3. Get the latest roster for the Id LMS DB LMS DB 4. Update the LMS database
  • 29. Best Practices • Make Event Publishers responsible for making sure that events are generated for every Business transaction and are published to the event channel • Make Event Consumers responsible for making sure that they react to the Stimuli from the event • Dashboards in place to make sure failed Events are detected and acted upon • Collects Stats to see if the events need to be made finer or coarser • Maintain Event Catalogs and self service capabilities to subscribe to events
  • 30. Complex Event Processing (CEP) • CEP tap the events happening with in an organization and provide a situation awareness and enable better and faster decisions – Identify complicated events that are inferred from event pattern detection, event pattern interpretors and so on – Ex: Monitoring – capture events from ticket system, network load, performance and send out notification bring up new servers into the pool Traditional BI finds "needles in haystacks." Event-driven CEP finds "needles" in continuously arriving streams of "hay."
  • 31. Events ,Big Data and Internet of Things • With Big Data hype and with the “capture-it-all“ approach and IoT , EDA and Event Driven programming will get a huge boost in the coming years. • Events and EDA due to their characteristics will be very well suited for Ubiquitous Computing • We might start seeing an advent of Complex Event Applications by being used by various gadgets – The concepts of EDA and Events will be a perfect Match for gadgets like Basis ,FIT

Editor's Notes

  1. As defined by Gartner SOA is classified into multiple styles RPC-Style , WOA-style , Event Driven SOA
  2. Application generated events are generally Coarse grained. Database generated events are generally grained
  3. kind of ID and version, you can often detect duplicates yourself and so not require to pay the performance cost of XA