7 September 2017 - At ION Conference Durban, South Africa, Andrew Alston on how Liquid Telecom deployed IPv6 and how other organizations can do the same.

1. Building Africa’sdigital future
IPv6 -A Case Study
1st September2016

2. First things first….
• Roll out our own IPv6 on our network
• We run IS-IS – and we enforce single topology!
• Mistakes were made - point to points out of a single /64 was a major one –
and we’re still fixing that even to this day
• We had to run full V6 table on the P layer – LDP6 didn’t exist when we
started this and nor did segment routing – again – something still be fixed!
• V6 in large part still runs unlabeled across BGP-LU transitions – this is
changing as we move towards an SR paradigm – but it forced divergence
in design.
• Basically – getting V6 into the core of a large network – isn’t as simple as it
can seem.

3. The next steps
• IP Transit customers were the first to get IPv6
• It was the simplest point – just turn up the BGP
• Transit customers typically handle their own networks – if they are
announcing V6 all I have to do is accept it
• A decision had to be made after that – where next
• The mass consumer market (home users) was the easiest target for
quick wins
• The enterprise market is far more difficult – because it often requires
the customer to do something
• We still had a lot of work to do on our own internal services
• We chose the quick win – the consumer market – in parallel with
continuing to work on our own internal services

4. Internal services (1)….
• During the early part of 2016 a decision was made – no more v4
management on our wireless gear.
• Our wireless deployments are generally Ruckus based
• We no longer have any v4 on the management of the end point AP’s –
this is pure IPv6 (a few thousand devices).
• Provisioning is done automatically – the device gets a V6 DNS server
(via RA) and a search domain and then uses DNS to find the
controller and provision itself.
• DNS Via RA was easy – the search domain was more problematic –
we had end devices that didn’t support handing it out
• Simple answer – cross connect them back to something that did!

5. Internal Services (2)….
• Web services took a long time – mainly convincing the IT department
things weren’t going to break!
• Mail services still to this day for inbound email are on IPv4 – mimecast
does not support IPv6 – and does not seem to have any plans to do so
• IPv6 on the office lans is pretty much everywhere – and it allows us to see
breakage fast
• (Note for those running IPv6: Microsoft Outlook does not seem to do
happy eyeballs – if you have V6 and its broken – the chances are –
Outlook 365 will not be connecting and synching email!)

6. Home Users – The Goals
• Move the V6 to the consumer – go beyond the edge of the network
• Ensure what we were doing was transparent
• The customer does not need to know if they are using V4 or V6!!!!
• Ensure a zero service impact during the rollout
• Downtime is not an option, and we were rolling out to live customers!

7. The Home user rollout….
• Move the V6 to the consumer – go beyond the edge of the network
• Ensure what we were doing was transparent
• The customer does not need to know if they are using V4 or V6!!!!
• Ensure a zero service impact during the rollout
• Downtime is not an option, and we were rolling out to live customers!

8. Starting with the basics…
• Addressing plans came first – make sure we had a proper plan to avoid
chaos later!
• After some testing – abandon the concept of dynamic addresses – go
static everywhere – its far less problematic
• Adjust the provisioning systems to allocate static addresses when
customers are provisioned
• Ensure the provisioning systems are talking back to the IPAM system so
we have a record of what is assigned where
• Enable the BRAS – get it actually allocating V6 addresses
• Ensure that CPE’s are getting the addresses and handling them properly
• Monitor and watch the traffic – is everything working

9. Our challenges along the
way…

10. The Challenges we faced
• We attempted dynamic V6 allocations and switched to static allocations
• Going with static allocations meant reworking the backend provisioning
system.
• Next step was BRAS configuration – this was relatively simple and without
issue.
• Modifications of the provisioning system was simpler than anticipated –
and our developers did an amazing job getting this done fast!
• Then we got to the CPE layer – and the wheels fell off – but more on this in
a second.
• Bottom line – V6 is pretty easy until you get to the CPE layer
• Once you get all that working though – you still have to deal with happy
eyeballs when you’re testing and monitoring

11. The CPE Issue
(1)….
• CPE’s had to be cheap – this is a requirement for a mass market
product, any CPE that cost to much wasn’t going to work
• CPE’s had to support TR-069 – the initial work we did was done on
Mikrotik’s for the metro home customers, which didn’t support this –
so an alternative had to be found.
• CPE’s on the GPON Network (ONT’s), are locked to OLT’s, so if they
didn’t support IPv6, it was time to talk to the vendor

12. The CPE Issue
(2)….
• Almost every CPE we tested had its issues….
• ALU ONT’s did not initially support IPv6 – getting the firmware
that did was challenging (and deploying it even more so!)
• Mikrotik had good V6 – but no TR-069, so its a non starter
• DLINK’s v6 support was fantastic but they have VERY
problematic firewall settings
• TP-Link requires loading OpenWRT – Not realistic for mass
deployment
• AVM Fritz!Box has a half English half German GUI and certain
severe limitations with its firewalls.
• Huawei ONT’s just worked!!!

13. Current status…
• We’re still deploying Mikrotik CPE’s on Metro Broadband – but we’re
still searching for good alternatives
• We got V6 on the ONT’s! Both Huawei and ALU ONT’s are now V6
tested.
• Our BRAS’s are both Cisco and Huawei – no problems here – and
we’ll be introducing Juniper BNG’s in the next few weeks
• We now have over a thousand /48’s allocated and active in Kenya
• We have over 20 thousand allocated /48s in Zimbabwe and the
majority of our V6 traffic is here
• vCPE option is in testing – give the customer a basic bridge device
on the metro and bring them back to a vCPE that actually has the
functions they really need!

14. Next Steps (1)…
• We’re about to start rolling out another network – and because its
green fields – it will be done differently
• We will be rolling segment routing – that means labelled V6 node
SID’s and full V6 (Testing is completed doing this).
• We are attempting to push what V4 we need on this network over V6
SR based LSP’s. Some testing has been done here – but all
indications are it’s going to require BGP-LS created LSP’s. Juniper’s
latest 17.3R1 though does support V4 prefix’s over V6 BGP
sessions!
• We are working extremely closely with vendors to fix what’s missing
• Cisco is currently the only vendor that supports V6 binding of
Martini Cross connects – we’re hoping we’ll see similar from
Juniper within 6 months
• LDP/SR Mapping is critical to making this work properly – and
testing on this is being done actively

15. Next Steps (2)…
• Announcement of V6 node SID’s into BGP is still not quite where it
needs to be – this means we’ve still got to make a plan for crossing
IGP boundaries (Testing is based on next-hop-self)
• We are working to start moving away from V4 traffic engineering –
build the TE tunnels over SR V6 LSP’s and push the V4 over that.
• We really are hoping to avoid V4 addressing on point to points – this
saves a LOT of address space in a network with thousands of links!
• We’re actively testing NAT64 and 464XLAT – we want to eliminate V4
to the customer other than via translation where possible – but we’re
probably still a way from getting this quite right.

16. Final Thoughts…
• Getting rid of V4 is going to require innovation – it’s already
happening in the mobile world – but taking it further is not as easy as
it sounds
• MPLS is a part of our lives – and V6 and MPLS until recently have
not played well together – this is changing with Segment Routing
(and LDPv6)
• The vendors are still lagging and have huge feature lag between
them – Features that exist in one vendor may not show up in another
for 2 to 3 years.
• A close working relationship with the vendors is critical – often you’re
pushing the boundaries on alpha and beta code and feeding back to
the vendor so they can get it right before production!
• Never, EVER, accept no from a vendor – there are options – and the
day you go elsewhere and explain why you did – will be the day the
previous vendor wakes up!

17. The sun is setting on IPV4
Liquid is ready for IPv6, is your
ISP?