Pimp My Cloud.pdf

Pimp My Cloud
Nova Configuration Hints and Tricks

Joe Gordon
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 1

Wednesday, October 17, 12

About Me
• Engineer at Cloudscaling
• Contributor
• Deployer
• Folsom Contributions
o Top 10 developer
(by commits)
o Mostly in Nova

http://bitergia.com/public/reports/openstack/2012_09_folsom/



By the Numbers: Nova Folsom
• 190+ Contributors
• Release Python Lines Other Lines Python Files Other Files

Folsom 186,738 242,721 666 788

Essex 150,894 221,109 593 302

Diablo 110,581 110,393 427 389

• Code churn:
Release Lines Insertions Lines Deletions Insertions/LoC %

Folsom 110,308 71,911 59.0%

Essex 182,298 138,346 120.8%

Code churn generated with git log --numstat --pretty="%H" $A..$B| grep .py$ | awk 'NF==3 {plus+=$1; minus+=$2} END {printf("+%d, -%dn", plus,
minus)}'



Who Wrote Nova Folsom?
1 138 Russell Bryant <rbryant@redhat.com>
2 112 Johannes Erdfelt <johannes.erdfelt@rackspace.com>
3 97 Dan Prince <dprince@redhat.com>
4 88 Vishvananda Ishaya <vishvananda@gmail.com>
5 81 Joe Gordon <jogo@cloudscaling.com>
6 63 Michael Still <mikal@stillhq.com>
7 59 Mark McLoughlin <markmc@redhat.com>
8 58 Rick Harris <rconradharris@gmail.com>
9 50 Yun Mao <yunmao@gmail.com>
10 45 Daniel P. Berrange <berrange@redhat.com>
11 36 Chris Behrens <cbehrens@codestud.com>
12 31 Eoghan Glynn <eglynn@redhat.com>
13 29 Brian Waldon <brian.waldon@rackspace.com>
14 26 Pádraig Brady <pbrady@redhat.com>
15 25 Chuck Short <zulcss@ubuntu.com>
16 23 Sean Dague <sdague@linux.vnet.ibm.com>
17 21 Alex Meade <alex.meade@rackspace.com>
18 18 Kevin L. Mitchell <kevin.mitchell@rackspace.com>
19 17 Brian Elliott <brian.elliott@rackspace.com>
20 17 Zhongyue Luo <zhongyue.nah@intel.com>
21 16 John Griffith <john.griffith@solidfire.com>
22 13 Dan Smith <danms@us.ibm.com>
23 13 Andrew Bogott <abogott@wikimedia.org>
24 12 Renuka Apte <renuka.apte@citrix.com>
25 12 Thierry Carrez <thierry@openstack.org>
git shortlog -sne --since="Tue Mar 20
26 12 Monty Taylor <mordred@inaugust.com> 08:17:40 2012 +0100" --no-merges | cat -n
27 10 MotoKen <motokentsai@gmail.com>



OpenStack Architecture



Nova Architecture



Vanilla Nova
• Devstack
• All in One
• RabbitMQ
• KVM
• MySQL
• Default configuration options



Customize Nova
• Choose a
o Hypervisor
o Database
o Message Queue
o Network Model
o Scheduler
• Configuration
o Any of the 500+ options
• Custom Services
o Any service can be
swapped out for one
you write



What Makes Openstack Open
• 3 RPC backends
• 3+ DB backends
• 6+ Virtualization backends
• 500+ configuration options



New Options for Folsom
• 1 RPC backends
o ZeroMQ
• 2 virt backends
o Hyper-V
o PowerVM
• 100+ configuration options



New Features for Folsom by Blueprint
• General host aggregates (operator)
• Disable API extensions
• Rootwrap pluggable filters (operator)
• Multi-scheduler support (operator)
• Flavor extra-specs
• LVM ephemeral disk images (operator)
• Project specific flavors
• Multi-process API services (operator)



Backend Choices



RPC Comparisons

RPC
Protocol Company Affiliations Brokerless Language
Backend

kombu
AMQP 0-9-1 Erlang
(rabbitMQ)

qpid AMQP 0-10 Java, C++

ZeroMQ Python, C++



Database Comparisons

Database Simultaneous writes High Availability Native python client

?



Virtualization Comparisons
Virtualization
Backend maintained by Fast Open Source
type

Baremetal N/A N/A

UML paravirt only

LXC container

Xen full/paravirt

KVM full

Qemu qemu.org full

Hyper-V full

PowerVM full

VMware ESX full



Configuration Options



Configuration Options by Type
Boolean

68
Floats

6
Ints

119 13%

Lists

18 1%

MultiString

6
String

290
23%
57%

4%
1%



New Config Options in Folsom
allowed_rpc_exception_modules max_age
api_url netapp_storage_service_prefix
auth_blob nfs_mount_point_base
backdoor_port non_inheritable_image_properties
base_dir_name notify_on_any_change
cinder_* notify_on_state_change
claim_timeout_seconds osapi_max_request_body_size
config_drive_format osapi_volume_workers
config_drive_skip_versions periodic_fuzzy_delay
config_drive_tempdir port
default_scheduler_driver powervm_*
defer_iptables_apply quantum_*
ec2_strict_validation quota_driver
ec2_workers quota_key_pairs
fatal_deprecations quota_security_group_rules
force_config_drive quota_security_groups
glance_api_insecure rbd_*
hyperv_attaching_volume_retry_count reservation_expire
hyperv_wait_between_attach_retry rootwrap_config
image_info_filename_pattern rpc_cast_timeout
instance_build_timeout rpc_zmq_*
instance_uuid_format server
libvirt_cpu_mode server_ca_file
libvirt_cpu_model snapshot_same_host
libvirt_images_type sql_connection_debug
libvirt_images_volume_group sql_connection_trace
libvirt_inject_key storwize_*
libvirt_inject_partition syslog_log_facility
libvirt_snapshots_directory until_refresh
libvirt_sparse_logical_volumes vswitch_name
limit_cpu_features xenapi_check_host
matchmaker_ringfile xenapi_num_vbd_unplug_retries



Virtualization options
1.pick a compute RPC topic
scheduler_topic=compute
2.pick a compute manager
compute_manager=nova.compute.manager.ComputeManager
3.compute manager options
instances_path=$state_path/instances
base_dir_name=_base
compute_driver=libvirt.LibvirtDriver
console_host=socket.getfqdn()
live_migration_retry_count=30
reboot_timeout=0
instance_build_timeout=0
rescue_timeout=0
resize_confirm_window=0
*_interval=
running_deleted_instance_timeout=0
running_deleted_instance_action=log
instance_usage_audit=false
4.compute driver options
rescue_*_id=
libvirt_type=kvm
libvirt_inject_password=false
libvirt_inject_key=true
*_migration_=
libvirt_cpu_mode=
libvirt_cpu_model=
...



compute.manager options
    cfg.IntOpt('live_migration_retry_count',
               default=30,
               help="Number of 1 second retries needed in live_migration"),
    cfg.IntOpt("reboot_timeout",
               default=0,
               help="Automatically hard reboot an instance if it has been "
                    "stuck in a rebooting state longer than N seconds. "
                    "Set to 0 to disable."),
    cfg.IntOpt("instance_build_timeout",
               default=0,
               help="Amount of time in seconds an instance can be in BUILD "
                    "before going into ERROR status."
    cfg.IntOpt("rescue_timeout",
               default=0,
               help="Automatically unrescue an instance after N seconds. "
    cfg.IntOpt("resize_confirm_window",
               default=0,
               help="Automatically confirm resizes after N seconds. "
  ...
    cfg.StrOpt("running_deleted_instance_action",
               default="log",
               help="Action to take if a running deleted instance is detected."
                    "Valid options are 'noop', 'log' and 'reap'. "
                    "Set to 'noop' to disable."),



reap running deleted instances
    cfg.IntOpt("running_deleted_instance_timeout",
               default=0,
               help="Number of seconds after being deleted when a running "
                    "instance should be considered eligible for cleanup."),
    cfg.IntOpt("running_deleted_instance_interval",
               default=30,
               help="Number of periodic scheduler ticks to wait between runs of "
                    "the cleanup task."),
    cfg.StrOpt("running_deleted_instance_action",
               default="noop",
               help="Action to take if a running deleted instance is detected."
                    "Valid options are 'noop', 'log' and 'reap'. "
                    "Set to 'noop' to disable."),

nova.conf settings
running_deleted_instance_timeout=60
running_deleted_instance_action=reap



Scheduler options
1.pick a scheduler RPC topic
scheduler_topic=scheduler
2.pick a scheduler manager
scheduler_manager=nova.scheduler.manager.SchedulerManager
3.pick a scheduler driver
scheduler_driver=nova.scheduler.multi.MultiScheduler
4.set up scheduler.driver options
scheduler_host_manager=nova.scheduler.host_manager.HostManager
scheduler_max_attempts=3
5.multi scheduler options
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler
volume_scheduler_driver=nova.scheduler.chance.ChanceScheduler
default_scheduler_driver=nova.scheduler.chance.ChanceScheduler
6.host_manager options
scheduler_available_filters=nova.scheduler.filters.standard_filters
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeC
apabilitiesFilter,ImagePropertiesFilter
7.filter options
ram_allocation_ratio=1.5
8.resource_tracker options
reserved_host_disk_mb=0
reserved_host_memory_mb=512
claim_timeout_seconds=600
compute_stats_class=nova.compute.stats.Stats



Add custom filter
cfg.MultiStrOpt('scheduler_available_filters',
            default=['nova.scheduler.filters.standard_filters'],
            help='Filter classes available to the scheduler which may '
                    'be specified more than once. An entry of '
                    '"nova.scheduler.filters.standard_filters" '
                    'maps to all filters included with nova.'),
     cfg.ListOpt('scheduler_default_filters',
                default=[
                  'RetryFilter',
                  'AvailabilityZoneFilter',
                  'RamFilter',
                  'ComputeFilter',
                  'ComputeCapabilitiesFilter',
                  'ImagePropertiesFilter'
                  ],
                help='Which filter class names to use for filtering hosts '
                      'when not specified in the request.'),

nova.conf settings
scheduler_available_filters=myfilter.CustomFilter
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,
ComputeCapabilitiesFilter,ImagePropertiesFilter,CustomFilter



*_driver *_manager
• network • compute
• quota • console
• compute • cert
• console • instance_dns
• db • floating_ip_dns
• linuxnet_interface • network
• scheduler • volume
o compute_scheduler
o volume_scheduler • scheduler
o default_scheduler • metadata
• baremetal • image_cache
• firewall • consoleauth
• libvirt_vif • scheduler_host
• volume_driver


Nova Architecture

manager

Driver



Resources
Blueprints: https://blueprints.launchpad.net/nova/folsom
Source: https://github.com/openstack/nova/tree/stable/folsom
Documentation: http://docs.openstack.org
Release notes: http://wiki.openstack.org/ReleaseNotes/Folsom



Questions?



Pimp My Cloud.pdf

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Viewers also liked

Viewers also liked (7)

Similar to Pimp My Cloud.pdf

Similar to Pimp My Cloud.pdf (20)

More from OpenStack Foundation

More from OpenStack Foundation (20)

Pimp My Cloud.pdf