Nuvola: a tale of migration to AWS

Nuvola: a tale of migration to AWS
Ansible + AWS: victory is mine!

Who I am?
CTO @
website: madisoft.it
tech blog: labs.madisoft.it

It’s a story
●It’s our story
●It’s about a migration
●We did it!
●We’ve learnt a lot
●We want to share it with you

Nuvola
●~ 2M users
●~ 1000 databases
●~ 350GB of mysql data
●~ 25M of media files
●~ 4.50TB of media files
●~ 60 servers

Why a migration?
No automation, flexibility and autoscaling

Need of change
●Flexibility
●Horizontal scaling
●Infrastructure as code
●Multiple environments
●All services by one provider
●Cost optimization

Obstacles
●It’s a distributed app
●Learn an entire new ecosystem
●Causing no troubles for users
●Few weeks of time

AWS
●100% automation
●Tons of services
●Very well integrated with Ansible
●IaaS services
●Autoscaling / Reserved instances / Spot instances

AWS
Autoscaling + reserverd instances + spot instances
Optimizing services while reducing costs

Ansible
●IT automation tool
●Easy lo learn
●No coding skills. It uses YAML
●No agents on target machines
●Ready-made AWS modules
●Can be easily idempotent

Ansible & AWS
AWS Azure Cloudstack Digital Ocean Google
86 18 31 5 10
Openstack Ovh Rackspace Softlayer Vmware
52 1 26 1 26
http://docs.ansible.com/ansible/list_of_cloud_modules.html

Ansible & AWS
ec2 - Create, terminate, start or stop an instance in ec2
http://docs.ansible.com/ansible/ec2_module.html
ec2_asg - Create or delete AWS Autoscaling Groups
http://docs.ansible.com/ansible/ec2_asg_module.html
ec2_elb_lb - Creates or destroys Amazon ELB.
http://docs.ansible.com/ansible/ec2_elb_lb_module.html
ec2_snapshot - creates a snapshot from an existing volume
http://docs.ansible.com/ansible/ec2_snapshot_module.html
ec2_tag - create and remove tag(s) to ec2 resources.
http://docs.ansible.com/ansible/ec2_tag_module.html
s3 - manage objects in S3.
http://docs.ansible.com/ansible/s3_module.html
route53 - add or delete entries in Amazons Route53 DNS service
http://docs.ansible.com/ansible/route53_module.html

Multiple environments
● ./infrastructure_nuvola_env_aws.sh --env prod
● ./infrastructure_nuvola_env_aws.sh --env dev
● ./infrastructure_nuvola_env_aws.sh --env lavorazione

Infrastructure
./infrastructure_nuvola_env_aws.sh --env prod
#!/bin/bash
. libs/limit_option_parser.sh
….
ansible-playbook
--vault-password-file secrets/infrastructure_nuvola_env.secret
ansible/infrastucture_nuvola_env.yml
-e"$EXTRA_OPTIONS"
….

Infrastructure playbook
infrastructure_nuvola_env.yml
tasks:
- include: .../infrastructure_nuvola_vpc.yml
tags: vpc
- include: .../infrastructure_nuvola_ec2.yml
tags: ec2
- include: .../infrastructure_nuvola_elb.yml
tags: elb
- include: .../infrastructure_nuvola_destroy.yml
when: destroy == "true" and nuvola_env != "prod"

VPC
tasks/infrastructure_nuvola_vpc.yml
- name: INFRASTRUCTURE NUVOLA VPC | setting up vpc
ec2_vpc:
state: present
cidr_block: 10.0.0.0/16
resource_tags: {
Name: "nuvola_{{ nuvola_env }}_vpc",
nuvola_env: '{{ nuvola_env }}',
nuvola_role: "vpc",
billing: "{{ billing_tag_value }}"
}
az: eu-west-1a
internet_gateway: True
register: vpc

VPC
infrastructure_nuvola_vpc.yml
- name: INFRASTRUCTURE NUVOLA VPC | vpc peering route
ec2_vpc_route_table:
vpc_id: "{{ vpc['vpc']['id'] }}"
tags:
Name: "nuvola_{{ nuvola_env }}_to_nuvola_default"
subnets:
- "10.0.{{ ec2_vpc_subnet }}.0/24"
routes:
- dest: 0.0.0.0/0
gateway_id: "{{ vpc.igw_id }}"
http://docs.ansible.com/ansible/ec2_vpc_route_table_module.html

ELB
infrastructure_nuvola_elb.yml
- name: INFRASTRUCTURE NUVOLA ELB | Setup ELB
ec2_elb_lb:
state: present
name: 'nuvola-{{ nuvola_env }}-elb'
listeners:
- protocol: http
load_balancer_port: 80
instance_port: 80
- protocol: https
load_balancer_port: 443
instance_protocol: http
instance_port: 80
ssl_certificate_id: '{{ output.stdout }}'
http://docs.ansible.com/ansible/ec2_elb_lb_module.html

EC2 backend
infrastructure_nuvola_ec2.yml
- name: INFRASTRUCTURE NUVOLA EC2 | Init backend instances
ec2:
key_name: '{{ ec2_key_name }}'
instance_type: '{{ backend_instance_type }}'
instance_tags:
nuvola_type: "{{ nuvola_env }}_backend"
nuvola_env: '{{ nuvola_env }}'
nuvola_role: "backend"
billing: "{{ billing_tag_value }}"
image: "{{ ec2_ami_id }}"
zone: "{{ ec2_zone }}"
wait: yes
wait_timeout: 600
group: "nuvola_{{ nuvola_env }}_backend_sg"

EC2 backend
count_tag:
nuvola_type: "{{ nuvola_env }}_backend"
exact_count: '{{ nuvola_backend_ec2_instances }}'
vpc_subnet_id: "{{ vpc['subnets'][0]['id'] }}"
assign_public_ip: yes
termination_protection: "{{ delete_lock }}"
volumes:
- device_name: /dev/sda1
volume_type: gp2
volume_size: "{{ ec2_volume_size_backend }}"
delete_on_termination: true
instance_profile_name: "{{ ec2_instance_role }}"
register: ec2_backend

Provisioning
./provision_nuvola_backend.sh --limit "tag_nuvola_type_${ENV}_backend"
./provision_nuvola_dbserver.sh --limit "tag_nuvola_type_${ENV}_database"
./provision_nuvola_routine.sh --limit "tag_nuvola_type_${ENV}_routine"
if [ "$ENV" != "prod" ]; then
./nuvola-init-not-prod-env.sh --env ${ENV}
./deploy_nuvola.sh --limit "tag_nuvola_type_${ENV}_backend" --env $
{ENV}
fi

Provisioning PHP7
roles/php7/tasks/php7_prod.yml
- name: PHP7 ALL | install php packages
apt: pkg={{ item }} state=latest update_cache=yes
with_items: '{{ php7_packages }}'
- name: PHP7 ALL | Set php.ini CLI
template:
src=roles/php7/templates/nuvola/php.ini.cli.j2
dest=/etc/php/7.0/cli/php.ini
- name: PHP7 ALL | Set php.ini php-fpm
template:
src=roles/php7/templates/nuvola/php.ini.web.j2
dest=/etc/php/7.0/fpm/php.ini

Multiple env: how do I find it?
Route 53
● Public DNS
○ nuvola-prod-backend-3.ops.madisoft.it
○ nuvola-prod-database-24.ops.madisoft.it
○ nuvola-dev-database-34.ops.madisoft.it
○ nuvola-issue8978-database-34.ops.madisoft.it
● Private DNS
○ local-prod-backend-0.ops.madisoft.it
○ local-prod-cache-sessioni-0.ops.madisoft.it
○ local-dev-database-14.ops.madisoft.it
○ local-issue8978-backend-0.ops.madisoft.it
pixabay.com

DNS
- name: INFRASTRUCTURE NUVOLA EC2 | Assign backend dns
route53:
command: create
zone: "{{ domain_tld }}"
record: "nuvola-{{ nuvola_env }}-backend-{{ item.0 }}.
{{ domain_tld }}"
type: A
value: '{{ item.1.public_ip }}'
overwrite: yes
ttl: "{{ ttl_expire }}"
with_indexed_items: '{{ ec2_backend.instances }}'
nuvola-prod-backend-3.ops.madisoft.it

DNS: local
- name: INFRASTRUCTURE NUVOLA EC2 | Assign database local
dns
route53:
command: create
zone: "{{ domain_tld }}"
record: "local-{{ nuvola_env }}-database-{{ item.0 }}.
{{ domain_tld }}"
type: A
value: '{{ item.1.private_ip }}'
overwrite: yes
with_indexed_items: '{{ ec2_database.instances }}'
local-dev-database-14.ops.madisoft.it

Warm up
Moving:
- static files from a shared NAS to S3
- external standalone services to ec2
- Jenkins CI to AWS
- ELK stack to AWS
- (and testing) Nuvola stage environment

Switch of
- Stop current app
- Create prod env infrastructure
- App deployment
- Copy db data

Db data migration
Many small databases on diferent machines
Use of parallelization
Mysql_migrate_dbserver.sh
….
ansible-playbook -l $LIMIT
ansible/mysql_migrate_dbserver.yml -e "nuvola_env=$ENV"
--vault-password-file ./secrets/provision_nuvola_dbserver.secret
….

App deploy
deploy_nuvola.sh
ansible-playbook
ansible/deploy_nuvola.yml
--extra-vars="nuvola_env=$ENV"

Achievement
Amazing
migration!

WE ARE
HIRING!(wanna join? ask us at the end of the talk or visit our website)

@mat_teo8
matteo.moretti@madisoft.it

Nuvola: a tale of migration to AWS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Nuvola: a tale of migration to AWS

Similar to Nuvola: a tale of migration to AWS (20)

Recently uploaded

Recently uploaded (20)

Nuvola: a tale of migration to AWS