This document discusses Messente's scaling of its SMS API to handle international growth in SMS traffic over time. It describes how Messente built its initial SMS sending and delivery report APIs, then scaled them by moving processing to backend services and pipelines, adding more API nodes, implementing database archiving, and ensuring backups to handle increased traffic and data volume. The scaling approach emphasized building a working product first before scaling later as needed.

1. What we have learned
about
API scaling
throughout Messente's
international
SMS traffic growth

2. Build Working Product
• SMS Sending API
• Delivery Report API
• Backend services
• Public website
• Admin interface

3. 22,5
45
67,5
90
2011 2012 2013 2014 2015 2016

4. SMS Sending API
$ curl 'https://messente.com/api/send_sms/'
-d 'user=hnoac48nta938ynvas039n04972'
-d 'api_key=023fjhwlekhf023ihf2poi3hf23'
-d 'from=Test'
-d 'to=%2B37251916062'
-d 'text=Hi+There!'
-d 'dlr_url=http%3A%2F%2Fmyshop.com%2Fmessente-dlr%2F'
OK slkdlskndflkdnflksdnflksdnl 0.056
ERROR 101
ERROR 102
....

5. Delivery Report Request
http://myshop.com/messente-dlr/
?message_unique_id=slkdlskndflkdnflksdnflksdnl
&status=DELIVERED

6. SMS Sending API
• connect to database
• validate account credentials
• validate sender name
• determine country and operator
• calculate the message price
• account crediting
• unique message ID generation
• save all this data to database

7. SMS Sending API
• connect to database
• validate account credentials
• validate sender name
• determine country and operator of the number
• determine the message price
• account crediting
• unique message ID generation
• save all this data to database

8. SMS Sending API
$ curl 'https://messente.com/api/send_sms/'
-d 'user=hnoac48nta938ynvas039n04972'
-d 'api_key=023fjhwlekhf023ihf2poi3hf23'
-d 'from=Test'
-d 'to=%2B37251916062'
-d 'text=Hi+There!'
-d 'dlr_url=http%3A%2F%2Fmyshop.com%2Fmessente-dlr%2F'
OK slkdlskndflkdnflksdnflksdnl 0.056
bottleneck

9. “Build now. Scale later.”

10. SMS Sending API - v2
$ curl 'https://api2.messente.com/send_sms/'
-d 'username=hnoac48nta938ynvas039n04972'
-d 'password=023fjhwlekhf023ihf2poi3hf23'
-d 'from=Test'
-d 'to=%2B37251916062'
-d 'text=Hi+There!'
-d 'dlr_url=http%3A%2F%2Fmyshop.com%2Fmessente-dlr%2F'
OK slkdlskndflkdnflksdnflksdnl
bottleneck

11. Delivery Report Request
http://myshop.com/messente-dlr/
?message_unique_id=slkdlskndflkdnflksdnflksdnl
&stat=DELIVRD
&err=000
error parameter added

12. SMS Sending API - v2
• extremely fast API response
• handle hundreds of req/sec
• return error codes asynchronously
• offloaded processing to backend service

13. SMS Processing Flow
Client
Messente HTTP API
Operator
Messages Queue
Backend Service

14. Backend Service - v2
• determine the country &
operator
• check the roaming and
portability information from
HLR service
• determine the route
• calculate the price of the
message
• credit account
• check for number in
blacklist
• verify sender name
• auto-convert text if
required
• update statistics
• save message to
database

15. Backend Service - v2
• determine the country &
operator
• check the roaming and
portability information
from HLR service
• determine the route
• calculate the price of the
message
• credit account
• check for number in
blacklist
• verify sender name
• auto-convert text if
required
• update statistics
• save message to
database

16. SMS Processing Flow
Client
Messente HTTP API
Operator
Messages Queue
Backend Daemon

17. SMS Processing Flow
Client
Messente HTTP API
Backend Daemon
Operator
Messages Queue
Backend Daemon
Backend Daemon

18. SMS Processing Flow
Client
Messente HTTP API
Backend Daemon
Operator
Messages Queue
Backend Daemon
Backend Daemon
Backend Daemon
Backend Daemon
Backend Daemon
Backend Daemon
Backend Daemon
Backend Daemon

19. SMS Processing Flow
Client
Messente HTTP API
Operator
Messages Queue
SMS Processing Pipeline

20. SMS Processing Pipeline
SMS
Determine Operator & MNP
Pricing & Account Crediting
Validator
Send SMS to Operator
Statistics

21. SMS Processing Pipeline
SMS
Determine Operator & MNP
(Multi-threaded HLR service)
Pricing & Account Crediting
Validator
Send SMS to Operator
Statistics

22. Handling Database
Size Growth
Multiple GB of data added every month

23. Handling Database
Size Growth
SMS DLR

24. Handling Database
Size Growth
SMS_2016_01 DLR_2016_01
SMS_2015_12 DLR_2015_12
SMS_2015_11 DLR_2015_11

25. Handling Database
Size Growth
SMS DLR
ARCHIVE_2016_01
ARCHIVE_2015_12
ARCHIVE_2015_11

26. Backup Servers
• Multiple API nodes
• api2.messente.com
• api3.messente.com
• Database Replication
• Server monitoring

27. Quick Takeaways
• Don't be afraid to scale later
• API will evolve - be prepared to handle different
versions
• Always have a backup