How to Perform an Upgrade from Havana to Icehouse—Red Hat Enterprise Linux and Derivatives

For this section, we assume that you are starting with the architecture provided in the OpenStack OpenStack Installation Guide and upgrading to the same architecture for Icehouse. All nodes should run Red Hat Enterprise Linux 6.5 or compatible derivatives such as CentOS and Scientific Linux with the latest Havana packages installed and operational. This section primarily addresses upgrading core OpenStack services such as Identity (keystone), Image Service (glance), Compute (nova), Networking (neutron), Block Storage (cinder), and the dashboard. The Networking upgrade procedure includes conversion from the Open vSwitch (OVS) plug-in to the Modular Layer 2 (ML2) plug-in.

 Impact on Users

The upgrade process interrupts management of your environment, including the dashboard. If you properly prepare for this upgrade, tenant instances continue to operate normally. However, instances might experience intermittent network interruptions while the Networking service rebuilds virtual networking infrastructure.

 Upgrade Considerations

Always review the release notes before performing an upgrade to learn about newly available features that you might want to enable and deprecated features that you should disable.

 Perform a Backup

Save the configuration files on all nodes, as shown here:

# for i in keystone glance nova cinder neutron openstack-dashboard; \
  do mkdir $i-havana; \
  done
# for i in keystone glance nova cinder neutron openstack-dashboard; \
  do cp -r /etc/$i/* $i-havana/; \
  done
[Note]Note

You can modify this example script on each node to handle different services.

Back up all databases on the controller:

# mysqldump -u root -p --opt --add-drop-database --all-databases > havana-db-backup.sql
[Note]Note

You must update your MySQL server configuration and restart the service as described in the MySQL controller setup section of the OpenStack Installation Guide.

 Manage Repositories

On all nodes, remove the repository for Havana packages and add the repository for Icehouse packages:

# yum erase rdo-release-havana
# yum install http://repos.fedorapeople.org/repos/openstack/openstack-icehouse/ \
  rdo-release-icehouse-3.noarch.rpm
[Warning]Warning

Disable any automatic package updates.

[Note]Note

Consider checking for newer versions of the Icehouse repository.

 Upgrade Notes

  • Disable Compute file injection:

    Icehouse disables file injection by default per the release notes.

    If you plan to deploy Icehouse in stages, you must disable file injection on all compute nodes that will remain on Havana.

    Edit the /etc/nova/nova-compute.conf file:

    [libvirt]
    ...
    libvirt_inject_partition = -2
  • Convert from OVS to ML2 plug-in:

    You must convert the configuration for your environment contained in the /etc/neutron/neutron.conf and /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini files from OVS to ML2. For example, the OpenStack Installation Guide covers ML2 plug-in configuration using GRE tunnels.

    We recommend keeping the OVS plug-in packages and configuration files until you verify the upgrade.

 Upgrade the Controller Node

Upgrade packages on the controller node to Icehouse, as follows:

# yum upgrade
[Note]Note

The package manager appends .rpmnew to the end of newer versions of existing configuration files. You should consider adopting conventions associated with the newer configuration files and merging them with your existing configuration files after completing the upgrade process. You can find newer versions of existing configuration files with the following command:

# find /etc -name *.rpmnew

 Upgrade Each Service

The upgrade procedure for each service typically requires that you stop the service, run the database synchronization command to update the associated database, and start the service to apply the new configuration. You need administrator privileges for these procedures. Some services require additional steps.

  • OpenStack Identity:

    Update the configuration file for compatibility with Icehouse.

    Edit the /etc/keystone/keystone.conf file:

    Add the [database] section.

    Move the connection key from the [sql] section to the [database] section.

    Stop services, upgrade the database, and start services.

    # service openstack-keystone stop
    # keystone-manage token_flush
    # keystone-manage db_sync
    # service openstack-keystone start
  • OpenStack Image Service:

    Before you upgrade the Image Service database, convert the character set for each table to UTF-8.

    Use the MySQL client to run the following commands:

    # mysql -u root -p
    mysql> SET foreign_key_checks = 0;
    mysql> ALTER TABLE glance.image_locations CONVERT TO CHARACTER SET 'utf8';
    mysql> ALTER TABLE glance.image_members CONVERT TO CHARACTER SET 'utf8';
    mysql> ALTER TABLE glance.image_properties CONVERT TO CHARACTER SET 'utf8';
    mysql> ALTER TABLE glance.image_tags CONVERT TO CHARACTER SET 'utf8';
    mysql> ALTER TABLE glance.images CONVERT TO CHARACTER SET 'utf8';
    mysql> ALTER TABLE glance.migrate_version CONVERT TO CHARACTER SET 'utf8';
    mysql> SET foreign_key_checks = 1;
    mysql> exit
    [Note]Note

    Your environment might contain different or additional tables that you must convert to UTF-8 by using similar commands.

    Update the configuration for compatibility with Icehouse.

    Edit the /etc/glance/glance-api.conf and /etc/glance/glance-registry.conf files:

    Add the [database] section.

    Rename the sql_connection key to connection and move it to the [database] section.

    Edit the /etc/glance/glance-api.conf file:

    Add Qpid message broker keys to the [DEFAULT] section.

    [DEFAULT]
    ...
    rpc_backend = qpid
    qpid_hostname = controller

    Stop services, upgrade the database, and start services:

    # service openstack-glance-api stop
    # service openstack-glance-registry stop
    # glance-manage db_sync
    # service openstack-glance-api start
    # service openstack-glance-registry start
  • OpenStack Compute:

    Update the configuration for compatibility with Icehouse.

    Edit the /etc/nova/nova.conf file:

    Change the rpc_backend key from nova.openstack.common.rpc.impl_qpid to qpid.

    Edit the /etc/nova/api-paste.ini file:

    Comment out or remove any keys in the [filter:authtoken] section beneath the paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory statement.

    Stop services, upgrade the database, and start services.

    # service openstacknova-api stop
    # service openstack-nova-scheduler stop
    # service openstack-nova-conductor stop
    # service openstack-nova-cert stop
    # service openstack-nova-consoleauth stop
    # service openstack-nova-novncproxy stop
    # nova-manage db sync
    # service openstack-nova-api start
    # service openstack-nova-scheduler start
    # service openstack-nova-conductor start
    # service openstack-nova-cert start
    # service openstack-nova-consoleauth start
    # service openstack-nova-novncproxy start
  • OpenStack Networking:

    Install the ML2 plug-in package:

    # yum install openstack-neutron-ml2

    Populate the /etc/neutron/plugins/ml2/ml2_conf.ini file with the equivalent configuration for your environment. Do not edit the /etc/neutron/neutron.conf file until after the conversion steps.

    Change the /etc/neutron/plugin.ini symbolic link to reference /etc/neutron/plugins/ml2/ml2_conf.ini.

    Stop services, upgrade the database, and perform the conversion from OVS to ML2.

    Replace NEUTRON_DBPASS with the password you chose for the database.

    [Warning]Warning

    We highly recommend that you perform a database backup prior to executing the following commands as the conversion script cannot roll back.

    # service neutron-server stop
    # neutron-db-manage --config-file /etc/neutron/neutron.conf \
      --config-file /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini stamp havana
    # neutron-db-manage --config-file /etc/neutron/neutron.conf \
      --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade icehouse
    # python -m neutron.db.migration.migrate_to_ml2 openvswitch \
      mysql://neutron:NEUTRON_DBPASS@controller/neutron

    Edit the /etc/neutron/neutron.conf file to use the ML2 plug-in and enable network change notifications:

    Replace SERVICE_TENANT_ID with the service tenant identifier (id) in the Identity service and NOVA_PASS with the password you chose for the nova user in the Identity service.

    [DEFAULT]
    ...
    core_plugin = ml2
    service_plugins = router
    ...
    notify_nova_on_port_status_changes = True
    notify_nova_on_port_data_changes = True
    nova_url = http://controller:8774/v2
    nova_admin_username = nova
    nova_admin_tenant_id = SERVICE_TENANT_ID
    nova_admin_password = NOVA_PASS
    nova_admin_auth_url = http://controller:35357/v2.0
    

    Start Networking services.

    # service neutron-server start
  • OpenStack Block Storage:

    Stop services, upgrade the database, and start services.

    # service openstack-cinder-api stop
    # service openstack-cinder-volume stop
    # service openstack-cinder-scheduler stop
    # cinder-manage db sync
    # service openstack-cinder-api start
    # service openstack-cinder-volume start
    # service openstack-cinder-scheduler start
  • Dashboard:

    Update the configuration for compatibility with Icehouse.

    Edit the /etc/openstack-dashboard/local_settings file:

    Change the OPENSTACK_KEYSTONE_DEFAULT_ROLE key from "Member" to "_member_".

    Restart Dashboard services.

    # service httpd restart

The controller node update is complete. Now you can upgrade the remaining nodes.

 Upgrade the Network Node

Upgrade packages on the network node to Icehouse:

[Note]Note

Make sure you have removed the repository for Havana packages and added the repository for Icehouse packages.

# yum upgrade

Install the ML2 plug-in package:

# yum install openstack-neutron-ml2

Edit the /etc/neutron/neutron.conf file to use the ML2 plug-in:

[DEFAULT]
core_plugin = ml2
service_plugins = router

Populate the /etc/neutron/plugins/ml2/ml2_conf.ini file with the equivalent configuration for your environment.

Change the /etc/neutron/plugin.ini symbolic link to reference /etc/neutron/plugins/ml2/ml2_conf.ini.

Clean the active OVS configuration:

# service neutron-ovs-cleanup restart

Restart Networking services:

# service neutron-dhcp-agent restart
# service neutron-l3-agent restart
# service neutron-metadata-agent restart
# service neutron-openvswitch-agent restart

 Upgrade the Compute Nodes

Upgrade packages on the compute nodes to Icehouse:

[Note]Note

Make sure you have removed the repository for Havana packages and added the repository for Icehouse packages.

# yum upgrade

Install the ML2 plug-in package:

# yum install openstack-neutron-ml2

Edit the /etc/neutron/neutron.conf file to use the ML2 plug-in:

[DEFAULT]
core_plugin = ml2
service_plugins = router

Populate the /etc/neutron/plugins/ml2/ml2_conf.ini file with the equivalent configuration for your environment.

Change the /etc/neutron/plugin.ini symbolic link to reference /etc/neutron/plugins/ml2/ml2_conf.ini.

Clean the active OVS configuration:

# service neutron-ovs-cleanup restart

Restart Networking services:

# service neutron-openvswitch-agent restart

Restart Compute services:

# service openstack-nova-compute restart

 Upgrade the Storage Nodes

Upgrade packages on the storage nodes to Icehouse:

[Note]Note

Make sure you have removed the repository for Havana packages and added the repository for Icehouse packages.

# yum upgrade

Restart Block Storage services.

# service openstack-cinder-volume restart
Questions? Discuss on ask.openstack.org
Found an error? Report a bug against this page


loading table of contents...