Show real time transactions on a map. Talk presented to Melbourne DevOPS Meetup in September, 2016.

1. REAL TIME EVENT DASHBOARDS
— MUJTABA HUSSAIN —

2. Who am I?
Infrastructure engineer and developer @ Fillr
Reluctant javascript code maintainer.
Blatant code copier.

3. Whats happening now?
Dashboard?
Real time? Is there an un-real time one? Can I get
that one?
What?

4. At the end …
Its not that hard! Its much harder muhahahahah
Know where to start looking!
For multiple users!

5. Why did I do this?
Investors.
Looks cool.
I learnt new stuff.

6. How I started?
Google “How do I build a google map?”
Google “How do I get norse map source code?”
Google “Please help!”

7. Screw it, I will build it
myself.
Highmaps
AWS JS SDK
Amazingly horrible JavaScript.

8. Done, but … meh
Looked OK.
Updates were basically bubbles.
My Javascript.

9. Enter DataMaps
Built on d3js
Solely for maps.
Bit more functionality compared to HighMaps.

10. Start Simple.

11. Data on the map.

12. Data from queue to map.

13. Did it work?

14. init: function(){
setTimeout(
jQuery.proxy(this.getData, this),
this.interval
);
}
var sqs = new AWS.SQS({
apiVersion: '2012-11-05',
accessKeyId: 'GREEDOSHOTFIRST',
secretAccessKey: 'TWILIGHTISCOOL',
region: 'ap-southeast-2'
});

15. Authentication?
I do not want to make this complicated.
Simplest procedure.
AWS::Cognito

16. Multi-users
Polling the queue within the client would have
issues.
Needed to set it up for a lot of clients.

17. WebSockets
em-websocket (ruby) as server
WebSocket (JavaScript) for client

18. EM.run {
EM::WebSocket.run(:host => WS_HOST, :port => WS_PORT) do |ws|
ws.onopen { |handshake|
queue.poll do |msg|
logger.debug 'Websocket connection open'
ws.send MESSAGE_FROM_QUEUE
end
}
ws.onclose { |msg| logger.info "Connection closed" }
ws.onerror { |msg| logger.error "Something went wrong" }
}
end
}
end

19. {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "STATEMENT_ID",
"Effect": "Allow",
"Action": [
"sqs:DeleteMessage",
"sqs:GetQueueAttributes",
"sqs:GetQueueUrl",
"sqs:ListDeadLetterSourceQueues",
"sqs:ListQueues",
"sqs:ReceiveMessage"
],
"Resource": [
"QUEUE_ARN"
]
}
]
}

20. Timeout
Client and server kept timing out.
Messages kept getting lost.
I kept losing hair.

21. @channel = EM::Channel.new
threads << Thread.new do
queue.poll do |msg|
logger.debug "Message received #{msg.body}"
@channel.push msg.body
end
end
threads << Thread.new do
EM.run {
EM::WebSocket.run(:host => WS_HOST, :port => WS_PORT) do |ws|
EventMachine.add_periodic_timer(3) { ws.send HEARTBEAT }
ws.onopen { |handshake|
sid = @channel.subscribe {|msg| ws.send msg }
logger.debug 'Websocket connection open'
ws.onmessage {|msg| ws.send “—heartbeat—" }
ws.onclose { |msg| }
ws.onerror { |msg| }
}
end
}
end
ws = new WebSocket("SERVER");
var heartbeat_msg = '--heartbeat--';
ws.onopen = function(evt) {
if (heartbeat_interval == null) {
missed_heartbeats = 0;
heartbeat_interval = setInterval(function() {
SEND_HEARTBEAT }, X_TIME_UNITS
);
}
};
ws.onclose = function(evt) { console.log("CLOSE" + evt); };
ws.onerror = function(evt) { console.log("ERROR" + evt); };
ws.onmessage = function(evt) {
if (evt.data === heartbeat_msg) {
ACK_HEARTBEAT_AND_RESET;
}
PROCESS_DATA;
};

22. But wait, there is more.
Deployment intended to AWS.
Where to set up the WebSocket server?
Surely binding to 0.0.0.0

23. Cue NGinX awesomeness
server {
listen HTTP_PORT;
server_name MAP_ENDPOINT;
location /socketcluster {
proxy_pass http://0.0.0.0:WS_PORT;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
location / {
rewrite ^/(.*) /$1 break;
proxy_ignore_client_abort on;
proxy_pass http://localhost:LOCAL_HTTP_PORT;
proxy_redirect http://localhost:LOCAL_HTTP_PORT MAP_ENDPOINT;
proxy_set_header HEADERS;
}
}