Mobile Devices come in many forms and shapes. Most of them work with data - either collecting, presenting or editing data. This data is usually retrieved from or eventually uploaded to a back end server. However, in order to allow users to work smoothly and quickly - even with limited or sometimes even no bandwidth - it becomes imperative to offer local storage on the device. To act as a cache for pre-fetched data as well as for modified and newly gathered data. And to negotiate with the back-end systems in a way that is transparent for users and apps. This presentation introduces the need for a local, on device data store. It discusses the requirements for such a mobile database. Then it introduces the key concepts in Service Oriented Architecture and finally it marries the mobile database and the SOA with its Enterprise Service Bus together - explaining how SOA concepts as well as a typical SOA implementation using an ESB is a perfect fit with and foundation for the Mobile Database.

1. Lucas Jellema (AMIS, The Netherlands)
MOBILE DATABASES AND SOA
Workshop “Big Data and Its Impact” – 16 & 17 March 2013
for the Military College of Telecommunication Engineering
1

2. THE PRESENTER:
LUCAS JELLEMA
• Lives in The Netherlands
(close to Amsterdam)
• 1994-2002 Oracle Consulting Global Center of
Excellence for Internet Development
• Joined AMIS in 2002 – now working as
CTO, Consultant (Architect & Technical Lead) &
Trainer
• Oracle ACE (2005) & ACE Director (2006)
• Author of „Oracle SOA Suite 11g Handbook‟
(Oracle Press, 2010)
• Presenter at Oracle OpenWorld, JavaOne, OTN
Yathra & more User Group Conferences
• Frequent blogger at http://technology.amis.nl
• Active with SQL & PL/SQL, Java EE &
2 ADF, SOA, BPM & more Fusion Middleware

3. OVERVIEW
• Overview of mobile devices
• Data is presented, collected and edited in devices
• Handle „challenged server data connection‟
• Introduce the Mobile Database
• Mobile Database challenges
– Big Data, Fast Data, Fresh Data, Distributed Database
System, Security, Notifications
• Overview of core SOA concepts and objectives
• How SOA helps to enable the Mobile Database
3

4. MY CAR – IS A DATA COLLECTOR
• Collecting metrics
– Some for big data analysis for all cars (of this type)
– Some for car analysis of my car when serviced
– Some for real time (traffic jam, local temperatures)
4

5. FORMULA ONE TELE METRICS
5

6. CAR NAVIGATION SYSTEM
6

7. LIVE TRAFFIC INFORMATION
7

8. LIVE? TRAFFIC INFORMATION
8

9. ZOOMING TRAFFIC INFORMATION
9

10. LIVE ON-ROAD TRAFFIC INFORMATION
SYSTEM == MOBILE DEVICE
10

11. ON SITE CRASH INSPECTION
11

12. PLETHORA OF MOBILE DEVICES & USAGES
12

13. MOBILE AGENTS
ATM
Security Badge Scanners
Sensors
Audio Guide
Traffic Information
Barcode Scanners & Satellites
Handheld Devices
Cars
Security Gates
Navigation Systems
Push Buttons
Cameras
Mobile Phones Microphones
& Speakers
Tablets
Report Read Read &
& Edit Refresh
(Alert)
13

14. DATA AND MOBILE DEVICES
• Some data requirements depend on the actual location
of the device
• Some data has to be on the device even when no data
connection is available
• Some data to be presented has to be fetched in (near)
real time from a central service
• Some (big) data is collected for analysis at a much
later point in time
• Some data is unstructured
• Some data needs to be sent from the device to a
central service in real time
• Most data will need to be refreshed at some point in
time – although the refresh rate differs
• Sometimes the user needs to be (very) aware when
data is not recently refreshed
14

15. DATA USED ON MOBILE DEVICES
• Originates from/to be sent to a back end (“the server”)
• Through limited bandwidth
• Sometimes entirely disconnected
– No reception
– Not permitted to use device
– Server side unavailable
15

16. NEED FOR ON-DEVICE-DATABASE
• Provide quick response for user
• Work when off-line
• Handle bursts of fast-data and piles of big data
• Take load off back end servers
16

17. REQUIREMENTS FOR MOBILE DATABASE
• Able to handle right types of data – including
unstructured data
• Easily accessible to apps
– Support for SQL
• Right scalability, right performance
• Right security
– Encrypted, Recoverable, Remote Wipe
• Maintainable/upgradable
– Database software and database model
– Pre-populated data (“firmware”)
• Able to fetch and process data from back-end
– Compact, secure, smart (incremental data updates)
• Smart upload to server, also Fast and Big Data
– pre-aggregated, pre-filtered, compacted
17

18. TRANSPARENT TO USER AND APPS
• The local database should be „invisible‟ to parties:
they interact with data service to query or report data
– However: it should be clear when data cannot
guaranteed to be fresh or complete
18

19. SECURITY
• Confidentiality – no unauthorized access
– Encryption of data in storage and transport
– Authorization on server access
• Integrity – no tampering, constraint compliant
• Availability – no data loss, right-time-access and
freshness
19

20. HOW DOES DATA ENTER THE MOBILE
DATABASE?
• Initial load when app is first installed
• Fetched from back end
• Entered by mobile app(s)
• When
– During install or upgrade of app
– During use of the application (by the user or device
features such as GPS, Bar Code
scanner, Camera or Recorder)
– When cache freshness is in doubt
– When cache completeness
is in doubt
– Synch upon re-connect:
submit collected new and
changed data
20

21. SOME GENERIC CHARACTERISTICS
• Pre loading
• Lazy loading (background)
• Write behind (lazy posting)
• Data Freshness and Completeness management
• Local fallback (next best thing)
21

22. DATA FRESHNESS AND COMPLETENESS
MANAGEMENT
• Based on actual time and geo-location…
– And possibly user, running app(s) and other context
• … assess if locally available data is fresh & complete
• When data is no longer fresh
– Try to negotiate a smart refresh with the server
– Depending on type of data: hide stale data, show stale
data with warning, continue to show stale data
• Incomplete data should be complemented
– Different location/area, zoom level, …
22

23. PUSH NOTIFICATIONS
• The server side can process messages, updates and
events from many directions
• Sometimes a mobile device should be informed of this
new information
• The server needs to know about the relevance of
information to a device (subscription)
• The server needs to be able to
push information (or poll-based
semi-push)
• The mobile database & app has to
process the pushed information
push
23

24. MOBILE DATABASE(S) & BACK END
SYSTEMS IS A …
24

25. DISTRIBUTED DATABASE SYSTEM
25

26. THE 12 OBJECTIVES FOR DISTRIBUTED
DATABASE SYSTEMS
26

27. MOBILE DATABASES DO NOT SUPPORT
DISTRIBUTED TRANSACTIONS
• Server side updates happen independent from clients
– Refresh of Mobile Database may take place at any
point in the future
• Data manipulation in Mobile Database is committed
locally on the device
– Upload to server happens later: „write (far) behind‟
• This means that at some point after the transaction is
complete, the transaction
is „replicated‟ to other
databases
• During this replication,
conflicts may occur
27

28. DATA REPLICATION – STAGE ONE
• Updates on the client are made through local apps an
stored in the mobile database
• Local validations apply – based on the information
available in the mobile device at the time of the
transaction
– Without consulting the back end
28

29. DATA REPLICATION („WRITE BEHIND‟)
CONFLICTS – STAGE TWO
• Updates on client violate server side integrity rules
– Invalid data is not tolerated by the back end
– The mobile client has previously accepted the
transaction
– How is the app/user engaged to correct/resolve?
29

30. DATA REPLICATION („WRITE BEHIND‟)
CONFLICTS
• Updates on client collide with changes in the server
– The client attempts to update a record that has also
been updated in some other way
– How are conflicts detected?
– How should conflicts be resolved?
– How should parties be notified
about „lost conflicts‟?
30

31. SOA = BAD
31

32. Business
SOA = Agility through
Decoupling
32

33. DECOUPLING
≈
MANAGING DEPENDENCIES
minimize impact of change while maximizing
reusability
33

34. COUPLING
A B
34

35. DECOUPLING
A B B
35

36. DECOUPLING
A B B
C
36

37. ADDITIONAL DECOUPLING THROUGH
INTERMEDIARY
A B
ESB B
C
37

38. COMPOSITE SERVICES
D
A B
ESB
E
38

39. TRANSFORMATION/ADAPTION OF
PROTOCOL AND FORMAT
XML/SOAP
HTTP
WS-Security EJB over RMI
B
D
B
ESB
A
E
SQL
JSON
JDBC over SQL*Net
HTTP
No encryption
SSL XML according to
canonical model
39

40. ORCHESTRATION, POTENTIALLY LONG
RUNNING, STATEFUL, ASYNCHRONOUS
D
A Process
B
(BPM/BPEL)
E
40

41. IMPLEMENTATION OF SERVICES
A B
ESB
PL/SQL
Mail
Server
41

42. EXTREME DECOUPLING USING EVENTS
subscription
D
Event
A B
Handler E
F
B
42

43. SOA AND MOBILE DATABASE
43 Service Bus

44. SOA AND MOBILE DATABASE
• Services that hide underlying complexity
– Technology, formats, protocols, data structure of back
end systems
• Services that handle two-way data synchronization
– Possibly conflicting changes on both ends
– Violation of server side data constraints
– Smart refresh of data on mobile device
• ESB that supports subscriptions and push to send
alerts/notifications/updates to mobile devices
• ESB that exposes services in appropriate protocol and
format (e.g. http/json)
• ESB that handles
security in transport
and message contents
44

45. SOA AND MOBILE DATABASE
45 Service Bus

46. SOA AND MOBILE DATABASE
• ESB ensuring the right availability – possibly
exceeding the service window of back-end systems
• ESB dealing with potentially large number of devices
– Scalability, Throttling (DoS protection)
• (Simulated) Device-to-Device-Push
• Upgrade of meta-data, apps and data-structure
• Bulk upload of Big Data from mobile device
• Initialization/Update of
pre-populated data on device
46

47. SUMMARY
• Mobile Database is a necessity to ensure apps can run
– With acceptable performance
– At all (in off-line situations)
• Mobile Database is a local cache that contains a
snapshot of the „back end data‟ at a certain timestamp
– Cache is not complete and is not current
– Cache contains data collected and edited on device
• Mobile Database is part of distributed database system
– Without real-time distributed transactions
– Periodic synchronization between Mobile Database and
Server is required (to really complete the transaction)
– Data conflicts need to be resolved
• SOA can help to hide complexity of back end and offer
available services with fitting protocol and data format
– Providing security, handling peak loads, supporting
push, understanding delta-based updates, …
47

48. RESOURCES
• Blog: technology.amis.nl
– On Oracle, SQL, Java, SOA, BPM & more
• Email: lucas.jellema@amis.nl
• : lucasjellema
• Slideshare: www.slideshare.net/lucasjellema
• : www.amis.nl, info@amis.nl
+31 306016000
Edisonbaan 15, Nieuwegein
The Netherlands
48