The bulk of the upgrade to v0.13 is about user and quota migrations. In summary, the migration process has 3 steps:
Warning
It is strongly suggested that you keep separate database backups for each service after the completion of each of step.
All web services must be brought down so that the database maintains a predictable and consistent state during the migration process:
# service gunicorn stop
# service snf-dispatcher stop
# etc.
Backup databases for recovery to a pre-migration state.
Keep the database servers running during the migration process
It is possible that two or more users have been registered with emails that differ only in the case of its letters. There can only be one of those accounts after the migration, so the rest must be deleted.
Note that even if the users are deleted in Astakos, there still are duplicate entries in Cyclades and Pithos. For each service we need to reduce those multiple accounts into one, either merging them together, or deleting and discarding data from all but one.
(script: find_astakos_users_with_conflicting_emails.py):
astakos-host$ cat << EOF > find_astakos_users_with_conflicting_emails.py
#!/usr/bin/env python
import os
import sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'synnefo.settings'
import astakos
from astakos.im.models import AstakosUser as A
def user_filter(user):
return A.objects.filter(email__iexact=user.email).count() > 1
all_users = list(A.objects.all())
userlist = [(str(u.pk) + ': ' + str(u.email) + ' (' + str(u.is_active) + ', ' +
str(u.date_joined) + ')') for u in filter(user_filter, all_users)]
sys.stderr.write("id email (is_active, creation date)\n")
print "\n".join(userlist)
EOF
astakos-host$ python ./find_astakos_users_with_conflicting_emails.py
(script: delete_astakos_users.py):
astakos-host$ cat << EOF > delete_astakos_users.py
#!/usr/bin/env python
import os
import sys
from time import sleep
os.environ['DJANGO_SETTINGS_MODULE'] = 'synnefo.settings'
import astakos
from astakos.im.models import AstakosUser as A
def user_filter(user):
return A.objects.filter(email__iexact=user.email).count() > 1
argv = sys.argv
argc = len(sys.argv)
if argc < 2:
print "Usage: ./delete_astakos_users.py <id>..."
raise SystemExit()
id_list = [int(x) for x in argv[1:]]
print ""
print "This will permanently delete the following users:\n"
print "id email (is_active, creation date)"
print "-- --------------------------------"
users = A.objects.filter(id__in=id_list)
for user in users:
print "%s: %s (%s, %s)" % (user.id, user.email, user.is_active,
user.date_joined)
print "\nExecute? (yes/no): ",
line = raw_input().rstrip()
if line != 'yes':
print "\nCancelled"
raise SystemExit()
print "\nConfirmed."
sleep(2)
for user in users:
print "deleting %s: %s" % (user.id, user.email)
user.delete()
EOF
astakos-host$ python ./delete_astakos_users.py 30 40
Warning
After deleting users with the delete_astakos_users.py script, check again with find_astakos_users_with_conflicting_emails.py (as in 2.1 Find duplicate email entries in Astakos) to make sure that no duplicate email conflicts remain.
astakos.host$ apt-get install \
kamaki \
snf-common \
snf-webproject \
snf-quotaholder-app \
snf-astakos-app
cyclades.host$ apt-get install \
kamaki \
snf-common \
snf-webproject \
snf-pithos-backend \
snf-cyclades-app
pithos.host$ apt-get install \
kamaki \
snf-common \
snf-webproject \
snf-pithos-backend \
snf-pithos-app \
snf-pithos-webclient
ganeti.node$ apt-get install \
kamaki \
snf-common \
snf-cyclades-gtools \
snf-pithos-backend
Note
Installing the packages will cause services to start. Make sure you bring them down again (at least gunicorn, snf-dispatcher)
Note
If you are asked about stale content types during the migration process, answer ‘no’ and let the migration finish.
astakos-host$ snf-manage syncdb
astakos-host$ snf-manage migrate
cyclades-host$ snf-manage syncdb
cyclades-host$ snf-manage migrate
Note
After the migration, Astakos has created uuids for all users, and has set the uuid as the public identifier of a user. This uuid is to be used both at other services (Cyclades, Pithos) and at the clientside (kamaki client settings).
Duplicate-email users have been deleted earlier in 2.1 Remove duplicate users in Astakos by their id
Generally:
# Service Setting Value
# quotaholder: QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v
# quotaholder: QUOTAHOLDER_TOKEN = <random string>
# astakos: ASTAKOS_QUOTAHOLDER_TOKEN = <the same random string>
# astakos: ASTAKOS_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v
# cyclades: CYCLADES_QUOTAHOLDER_TOKEN = <the same random string>
# cyclades: CYCLADES_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v
# cyclades: CYCLADES_USE_QUOTAHOLDER = True
# pithos: PITHOS_QUOTAHOLDER_TOKEN = <the same random string>
# pithos: PITHOS_QUOTAHOLDER_URL = https://quotaholder.host/quotaholder/v
# pithos: PITHOS_USE_QUOTAHOLDER = True
# All services must match the quotaholder token and url configured for quotaholder.
Specifically:
On the Astakos host, edit /etc/synnefo/20-snf-astakos-app-settings.conf:
QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v'
QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w'
ASTAKOS_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w'
ASTAKOS_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v'
On the Cyclades host, edit /etc/synnefo/20-snf-cyclades-app-quotas.conf:
CYCLADES_USE_QUOTAHOLDER = True
CYCLADES_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v'
CYCLADES_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w'
# Set to False if astakos & cyclades are on the same host
#CYCLADES_PROXY_USER_SERVICES = True
Note
If Cylcades and Astakos are installed on the same server, set CYCLADES_PROXY_USER_SERVICES = False
On the Pithos host, edit /etc/synnefo/20-snf-pithos-app-settings.conf:
PITHOS_QUOTAHOLDER_URL = 'https://accounts.example.synnefo.org/quotaholder/v'
PITHOS_QUOTAHOLDER_TOKEN = 'aExampleTokenJbFm12w'
PITHOS_USE_QUOTAHOLDER = False # will set to True after migration
Note
During the migration it must be set, PITHOS_USE_QUOTAHOLDER = False. Set to True once the migration is over.
Remove this redirection from astakos front-end web server
RewriteRule ^/login(.*) /im/login/redirect$1 [PT,NE]
(see http://www.synnefo.org/docs/synnefo/latest/quick-install-admin-guide.html#apache2-setup)
Enable users to change their contact email. Edit /etc/synnefo/20-snf-astakos-app-settings.conf
ASTAKOS_EMAILCHANGE_ENABLED = True
Rename the following (Astakos-specific) setting:
ASTAKOS_DEFAULT_FROM_EMAIL
to this (Django-specific) name:
SERVER_EMAIL
Instead of using the following (Astakos-specific) setting:
ASTAKOS_DEFAULT_ADMIN_EMAIL
include one or more entries in this (Django-specific) setting:
ADMINS = (
('Joe Doe', 'doe@example.net'),
('Mary Jean', 'mary@example.net'),
)
Note
The SERVER_EMAIL and ADMINS settings are Django-specific. As such they will be shared among any two (or more) services that happen to be collocated within the same application server (e.g. astakos & cyclades within the same gunicorn)
Run on the Astakos host
# snf-manage service-list
Set the Cyclades service token in /etc/synnefo/20-snf-cyclades-app-api.conf
CYCLADES_ASTAKOS_SERVICE_TOKEN = 'asfasdf_CycladesServiceToken_iknl'
Since version 0.13, Synnefo uses VMAPI in order to prevent sensitive data needed by ‘snf-image’ to be stored in Ganeti configuration (e.g. VM password). This is achieved by storing all sensitive information to a CACHE backend and exporting it via VMAPI. The cache entries are invalidated after the first request. Synnefo uses memcached as a django cache backend. To install, run on the Cyclades host:
apt-get install memcached
apt-get install python-memcache
You will also need to configure Cyclades to use the memcached cache backend. Namely, you need to set IP address and port of the memcached daemon, and the default timeout (seconds tha value is stored in the cache). Edit /etc/synnefo/20-snf-cyclades-app-vmapi.conf
VMAPI_CACHE_BACKEND = "memcached://127.0.0.1:11211/?timeout=3600"
Finally, set the BASE_URL for the VMAPI, which is actually the base URL of Cyclades, again in /etc/synnefo/20-snf-cyclades-app-vmapi.conf. Make sure the domain is exaclty the same, so that no re-directs happen
VMAPI_BASE_URL = "https://cyclades.example.synnefo.org"
Note
Pithos forwards user catalog services to Astakos so that web clients may access them for uuid-displayname translations. Edit on the Pithos host /etc/synnefo/20-snf-pithos-app-settings.conf
PITHOS_USER_CATALOG_URL = https://accounts.example.synnefo.org/user_catalogs/
PITHOS_USER_FEEDBACK_URL = https://accounts.example.synnefo.org/feedback/
PITHOS_USER_LOGIN_URL = https://accounts.example.synnefo.org/login/
# Set to False if astakos & pithos are on the same host
#PITHOS_PROXY_USER_SERVICES = True
Note
If Pithos and Astakos are installed on the same server, set PITHOS_PROXY_USER_SERVICES = False
Warning
To ensure consistency, prevent public access to astakos during migrations. This can be done via firewall or webserver access control.
Start (or restart, if running) the webserver and gunicorn on the Astakos host. E.g.:
# service apache2 start
# service gunicorn start
First, set the corresponding values on the following dict in /etc/synnefo/20-snf-astakos-app-settings.conf
# Set the cloud service properties
ASTAKOS_SERVICES = {
'cyclades': {
# # Specifying the key 'url' will overwrite it.
# # Use this to (re)set service URL.
# 'url': 'https://cyclades.example.synnefo.org/ui/',
# # order services in listings, cloudbar, etc.
# 'order' : 1
'resources': [{
'name': 'disk',
'group': 'compute',
'uplimit': 30*1024*1024*1024,
'unit': 'bytes',
'desc': 'Virtual machine disk size'
}, {
'name': 'cpu',
'group': 'compute',
'uplimit': 6,
'desc': 'Number of virtual machine processors'
}, {
'name': 'ram',
'group': 'compute',
'uplimit': 6*1024*1024*1024,
'unit': 'bytes',
'desc': 'Virtual machines'
}, {
'name': 'vm',
'group': 'compute',
'uplimit': 2,
'desc': 'Number of virtual machines'
}, {
'name': 'network.private',
'group': 'network',
'uplimit': 1,
'desc': 'Private networks'
}
]
},
'pithos+': {
# # Use this to (re)set service URL.
# 'url': 'https://pithos.example.synnefo.org/ui/',
# # order services in listings, cloudbar, etc.
# 'order' : 2
'resources':[{
'name': 'diskspace',
'group': 'storage',
'uplimit': 5*1024*1024*1024,
'unit': 'bytes',
'desc': 'Pithos account diskspace'
}]
}
}
Note
The name of the Pithos service is pithos+. If you have named your pithos service pithos, without +, then you must rename it:
$ snf-manage service-list | grep pithos # find service id
$ snf-manage service-update --name='pithos+' <service id>
Then, configure and load the available resources per service and associated default limits into Astakos. On the Astakos host run
# snf-manage astakos-init --load-service-resources
Note
Before v0.13, only cyclades.vm, cyclades.network.private, and pithos+.diskspace existed (not with these names, there were per-service settings). However, limits to the new resources must also be set.
If the intention is to keep a resource unlimited, (counting on that VM creation will be limited by other resources’ limit) it is best to calculate a value that is too large to be reached because of other limits (and available flavours), but not much larger than needed because this might confuse users who do not readily understand that multiple limits apply and flavors are limited.
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --validate
Duplicate user found?
either merge (merge will merge all resources to one user):
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --merge-user=kpap@grnet.gr
or delete
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --delete-user=KPap@grnet.gr
# (only KPap will be deleted not kpap)
pithos.host$ snf-manage pithos-manage-accounts --list-duplicate
Duplicate user found?
If you want to migrate files first:
merge (merge will merge all resources to one user):
pithos.host$ snf-manage pithos-manage-accounts --merge-accounts --src-account=SPapagian@grnet.gr --dest-account=spapagian@grnet.gr
# (SPapagian@grnet.gr's contents will be merged into spapagian@grnet.gr, but SPapagian@grnet.gr account will still exist)
and then delete
pithos.host$ snf-manage pithos-manage-accounts --delete-account=SPapagian@grnet.gr
# (only SPapagian@grnet.gr will be deleted not spapagian@grnet.gr)
If you do NOT want to migrate files just run the second step and delete the duplicate account.
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --migrate-users
if invalid usernames are found, verify that they do not exist in astakos:
astakos.host$ snf-manage user-list
if no user exists:
cyclades.host$ snf-manage cyclades-astakos-migrate-013 --delete-user=<userid>
Finally, if you have set manually quotas for specific users inside /etc/synnefo/20-snf-cyclades-app-api.conf (in VMS_USER_QUOTA, NETWORKS_USER_QUOTA make sure to update them so that:
To find the UUIDs for step 2 run on the Astakos host
# snf-manage user-list
Check if alembic has not been initialized
pithos.host$ pithos-migrate current
If alembic current is None (e.g. okeanos.io)
pithos.host$ pithos-migrate stamp 3dd56e750a3
Then, migrate pithos account name to uuid:
pithos.host$ pithos-migrate upgrade head
Finally, set this setting to True:
PITHOS_USE_QUOTAHOLDER = True
Migrate from pithos native to astakos/quotaholder. This requires a file to be transfered from Cyclades to Astakos:
pithos.host$ snf-manage pithos-export-quota --location=pithos-quota.txt
pithos.host$ scp pithos-quota.txt astakos.host:
astakos.host$ snf-manage user-set-initial-quota pithos-quota.txt
Note
pithos-export-quota will only export quotas that are not equal to the defaults in Pithos. Therefore, it is possible to both change or maintain the default quotas across the migration. To maintain quotas the new default pithos+.diskpace limit in Astakos must be equal to the (old) default quota limit in Pithos. Change either one of them make them equal.
see 5. Load resource definitions into Astakos on how to set the (new) default quotas in Astakos.
cyclades.host$ snf-manage cyclades-export-quota --location=cyclades-quota.txt
cyclades.host$ scp cyclades-quota.txt astakos.host:
astakos.host$ snf-manage user-set-initial-quota cyclades-quota.txt
cyclades-export-quota will only export quotas that are not equal to the defaults. See note above.
The following should report all users not having quota limits set because the effective quota database has not been initialized yet.
astakos.host$ snf-manage astakos-quota --verify
Initialize the effective quota database:
astakos.host$ snf-manage astakos-quota --sync
This procedure may be used to verify and re-synchronize the effective quota database with the quota limits that are derived from policies in Astakos (initial quotas, project memberships, etc.)
The effective quota database (quotaholder) has just been initialized and knows nothing of the current resource usage. Therefore, each service must send it in.
pithos.host$ snf-manage pithos-usage --reset
cyclades.host$ snf-manage cyclades-reset-usage
In order to detect and effect project expiration, a management command has to be run periodically (depending on the required granularity, e.g. once a day/hour):
astakos.host$ snf-manage project-control --terminate-expired
A list of expired projects can be extracted with:
astakos.host$ snf-manage project-control --check-expired
Start (or restart, if running) all Synnefo services on all hosts.
# service gunicorn restart
# service snf-dispatcher restart
# etc.