modoboa/modoboa-pfxadmin-migrate

Domain matching query does not exist

tkald opened this issue · 19 comments

tkald commented

Trying to run migration script on mysql database - after last domain I get error:

Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 353, in execute_from_command_line
utility.execute()
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 288, in handle
self._do_migration(options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 282, in _do_migration
options["_from"]).get(domain='ALL')
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/query.py", line 387, in get
self.model._meta.object_name
modoboa_pfxadmin_migrate.models.DoesNotExist: Domain matching query does not exist.


Latest version of modoboa. Ubuntu 16.04

Hi, have you defined administrators in postfixadmin?

tkald commented

Yes, I have 2 admin users. For one admin (limited access) there was notice that password was set to superadmin password.
No info about superadmin was displayed though.

What is your postfixadmin version?

Both admins are limited? Or do you have a global admin?

tkald commented

One admin is superadmin, otherone is limited.
Postfixadmin 2.3.6.
And superadmin does have domain "ALL" in domain_admins.domain table

But looks like the 'ALL' domain is missing from the domain table... Can you confirm that?

tkald commented

Yes, no "ALL" domain

That's why the migration script fails... Do you know why this domain is missing? I thought it was mandatory...

tkald commented

I did some minor PA code modifications back in a day, can't recall anything related to ALL domain though :)
Ok, I can modify tables fo fit any requirements.

Maybe you can add to the docs expected PA database schema and other requirements. I did also have trouble with some accounts having "0000-00-00" modification date... as it is default value in my db schema.

I can also try a fix to migrate super admins even if the ALL domain is not present. About dates, I don't think I can to something... The migration script is compatible with version 2.3 only.

tkald commented

Maybe you can set date now() if date is 0000-00-00 in PA database?

@tkald could you try the 2 fixes I just pushed please?

tkald commented

Originally I also have null dates in PA domain table - it does fail with latest commits:

Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 353, in execute_from_command_line
utility.execute()
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 292, in handle
self._do_migration(options)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 284, in _do_migration
self._migrate_domain(pf_domain, options, creator)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 202, in _migrate_domain
self._migrate_mailboxes(newdom, options, creator)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_pfxadmin_migrate/management/commands/migrate_from_postfixadmin.py", line 182, in _migrate_mailboxes
new_user.save(creator=creator, using=options["_to"])
File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa/core/models.py", line 366, in save
super(User, self).save(*args, **kwargs)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 708, in save
force_update=force_update, update_fields=update_fields)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 736, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 820, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 859, in _do_insert
using=using, raw=raw)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/manager.py", line 122, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/query.py", line 1039, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1060, in execute_sql
cursor.execute(sql, params)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 117, in execute
six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 112, in execute
return self.cursor.execute(query, args)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/srv/modoboa/env/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.IntegrityError: (1048, "Column 'date_joined' cannot be null")

tkald commented

Yes, works now! Thank you.

However I did get one more error stating that user name was too long:

django.db.utils.DataError: (1406, "Data too long for column 'last_name' at row 1")

I made name shorter in PA mailbox table and migration was success.

I do have next problem with my custom domain transports which are all gone in modoboa database.
I agree that you can't count with every custom installation but will it possibly work if I insert my transports directly into database? I did have some sympa transports there..

Never tried to import transport entries from postfixadmin. Can you paste an example please?

tkald commented

image

With import, all those alternative transports became type: domain
Bigger problem is, there doesn't seems to be relevant data fields in modoboa database where to store this info.

Actually, Modoboa does not provide a generic way to create transport entries... About your configuration, you only need to manually create a transport table for sympa mailing lists. Modoboa provides a vacation plugin, so vacation service in your example will be replaced. The virtual service is not used by modoboa and it is replaced by dovecot but the migration script already creates the needed data so you don't have to care about it.

tkald commented

Yeah, it's the sympa transport I'm concerned about.

Would be nice to have a way to define your own generic/custom transports if needed :)