Design document for attach/detach functionality =============================================== The goal of this design document is to initiate the discussion regarding the implementation of the attach/detach operations for Synnefo Volumes. We will first document what is the current state of Synnefo (0.16.1 as of writing this) regarding Volumes, explain some Ganeti issues that make the implementation a bit tricky and then provide a design draft. Current state and shortcomings ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since 0.16, Synnefo has provided an API for Volumes that is compatible to the Openstack Cinder API (see `Block Storage API Guide `_). Using this API, the user can create Volumes, update and list them. There is just a minor nitpick, that the user cannot create a Volume without specifying a VM that it will be attached to. Moreover, the user currently has no way to detach a volume from a VM and attach it to another. Ganeti issues ~~~~~~~~~~~~~ Ganeti 2.14 has added support for the attach/detach volume operations. However, these operations still have some limitations and caveats. Limitations ----------- 1. There is no ``gnt-volume`` command. All volume operations happen within the context of a VM modification, and more specifically from ``gnt-instance modify --disk ...``. This means that all volumes must be created with a VM as target and detached volumes can be referenced only for attach operations. 2. The attach/detach operations work only within a Ganeti cluster. This means that there is no **pure** Ganeti way to detach a volume from a VM in **gnt-cluster-1** and attach it to a VM in **gnt-cluster-2**. 3. A volume can be attached to a VM if and only if that VM can access the volume data. This means that e.g. a DRBD volume can only be attached to DRBD VMs whose primary and secondary nodes match that of the volume. Caveats ------- 1. The volume name that is generated by Ganeti and used by Synnefo follows roughly this form: **UUID.disk_template.diskX**. This means that during the disk's lifetime, its name can change. Note that Ganeti can provide two names to the ExtStorage scripts: * ``VOL_NAME``: A unique name that Ganeti generates for the volume. * ``VOL_CNAME``: A unique name that the user (in our case Synnefo) creates for the volume. For more info, you can read the `related Ganeti doc `_. Archipelago currently uses ``VOL_NAME`` as the name for its volumes. 2. Ganeti does not support multiple disk templates for VMs. 3. Ganeti does not pass user-provided parameters for Ext templates during the removal of a disk. Design overview ~~~~~~~~~~~~~~~ In this section, we will propose a way to decouple the **Create** operation from the **Attach** operation and add a **Detach** operation. Also, we will suggest an altered **Delete** operation from the existing one. Before showing the design behind the above operations, we will list a few compromises that are needed to make the attach/detach operations work: 1. Since the user is not aware of Ganeti nodes and clusters, we must provide an **Attach** operation that will work Synnefo-wide. This is achievable only with Archipelago volumes, which are visible from all Ganeti clusters. 2. In order to be able to reference the Archipelago volumes from different clusters, we have to resort in using the ``VOL_CNAME`` instead of ``VOL_NAME``, since the first will remain the same throughout the Volume's lifetime. For more info regarding this issue, you can consult the `Backend names of existing volumes`_ section. 3. We must alter Ganeti in order to pass an option that will keep the disk data after the removal of a disk from the VM. The ``snf-ganeti`` package already has this option (``--keep-disks``), so we only need to fix a minor bug. We will now proceed with the design of each action: Create ------ The **Create** operation currently is tightly coupled with the **Attach** operation, and requires the presence of a ``server_id`` argument in order to create and attach a Volume to a VM at once. In order to decouple these two operations and maintain backwards compatibility, we can make this argument optional. Therefore, a **Create** operation with no ``server_id`` provided will simply do the checks that it does now and stop at the step where it stores a Volume entry in the database. The actual volume will be created once the user attempts to attach it to an instance. Attach ------ The **Attach** operation will have two targets, a VM id and a Volume id. This operation will continue from where **Create** left off, i.e. it will send a disk creation job (``gnt-instance modify --disk -1:add,name=,reuse_data=True``) to Ganeti. The ``reuse_data`` Ext parameter should inform Archipelago to not create a new volume but reuse an existing one. Note that there are two cases here: 1. **The volume has not been attached to an instance before:** In this case, we will inform the ExtStorage script to create the Volume in Archipelago, using the ``VOL_CNAME`` as its name. 2. **The volume has been attached to an instance before:** This means that the Volume has been initialized. In this case, we will inform the ExtStorage script to simply map the existing volume to the instance, using the ``VOL_CNAME`` as identifier. Detach ------ The **Detach** operation will not use the Ganeti **detach** operation as one would expect, but the **remove** operation (``gnt-instance modify --keep-disks --disk :remove``). As mentioned above, the ``--keep-disks`` will keep the disk data intact. In Archipelago terms, the ``detach`` ExtStorage script will be called but not the ``remove`` script. The rationale behind this choice is to avoid having duplicate references to the same volume from different clusters, since the remove operation must operate only in one. Also, a detached Ganeti volume cannot be destroyed from Ganeti (see limitation 1), therefore it must be done from Archipelago. In this case, we do not want any reference of this volume to exist in any Ganeti cluster. Delete ------ The **Delete** operation will have two cases: 1. **The volume is attached to an instance:** In this case, we will issue a remove operation (``gnt-instance modify --disk :remove``), as we are currently doing. 2. **The volume is detached:** Although at this point the volume is not in any Ganeti config and can be safely removed using a ``vlmc`` command, we need a way to receive callbacks for this action and to make sure that the remove has succeeded in order to change quotas etc. We could extend the ``snf-dispatcher`` to support Archipelago as a backend, but after consideration, we decided that it would be best if we reused the existing logic and removed the disk through Ganeti. This means that a detached volume must be attached to a helper server and then be removed. The attachment to the helper server must be transparent to the user. Implementation details ~~~~~~~~~~~~~~~~~~~~~~ The above design has some practical issues which must be tackled in order to have a functional Synnefo installation with detachable volumes. Backend names of existing volumes --------------------------------- The attach/detach feature cannot work out-of-the-box for existing Synnefo installations which have live Archipelago volumes. The reason is that the name of these volumes is the ``VOL_NAME`` ExtStorage parameter which cannot be used as it is not consistent across Ganeti clusters. Preferably, we would like to change the name of the Archipelago volumes to match the one that is stored in the DB (``VOL_CNAME``). However, this is not easy to do, especially for live volumes. Thus, we suggest to do the opposite, i.e. read the Ganeti config of each Ganeti cluster, find all Archipelago volumes and store their Ganeti name in the Cyclades DB. In order to do the above, we need to add a new field in the Volume model, since the ``backend_volume_uuid`` field is not an actual column in the DB, but a Python class property which has the following definition: .. code-block:: python @property def backend_volume_uuid(self): return u"%svol-%d" % (settings.BACKEND_PREFIX_ID, self.id) We propose the following change in the Volume class: .. code-block:: python legacy_backend_volume_uuid = models.CharField("Legacy volume UUID in backend", max_length=128, null=True) @property def backend_volume_uuid(self): return (self.legacy_backend_volume_uuid or u"%svol-%d" % (settings.BACKEND_PREFIX_ID, self.id)) With this change, we can: * Κeep the ``backend_volume_uuid`` interface intact and avoid refactoring the existing code, * Allow any new volume to use the existing naming scheme (``VOL_CNAME``), * ...and update the old ones so that ``backend_volume_uuid`` outputs the legacy name (``VOL_NAME``). .. note:: The above change needs a migration script to run before the new Archipelago version is installed in Synnefo. This migration script will be similar to the ``add_unique_name_to_disks`` script. Helper servers -------------- In order to be able to delete a detached volume, there has to be a helper server instance in an accessible Ganeti cluster. This means that the administrator must create some helper servers, preferably one for every Ganeti cluster, using the following command:: snf-manage server-create ... --helper --backend-id Also, for security reasons, the helper servers should be in stopped state, which means that the administrator must use the following command for each server:: snf-manage server-modify ... --action=stop To make the administrator's life easier, the above can be wrapped in an ``snf-manage`` command. API extensions ~~~~~~~~~~~~~~ Synnefo's current API implementation regarding Volumes is almost fully compatible with the OpenStack **Cinder** and **Nova** (os-volume_attachments) API. The only change that it needs to be marked as fully compatible is to lift the requirement of a server id during the creation of a Volume. The user will still be able to provide a server id, in order to retain the backwards compatibility, however it should no longer be necessary.