Starting `docker-compose up` on a fresh install runs Celery beat tasks before migrations causing an error
Closed this issue · 1 comments
mrosendin commented
Description
See the logs from docker-compose up on a fresh start up:
polyledger git/master 33s
❯ docker-compose up
Creating network "polyledger_default" with the default driver
Creating postgres ... done
Creating redis ... done
Creating server ... done
Creating web_client ... done
Attaching to postgres, redis, server, web_client
postgres | The files belonging to this database system will be owned by user "postgres".
postgres | This user must also own the server process.
postgres |
redis | 1:C 11 May 04:36:10.147 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis | 1:C 11 May 04:36:10.147 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
redis | 1:C 11 May 04:36:10.147 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis | 1:M 11 May 04:36:10.155 * Running mode=standalone, port=6379.
redis | 1:M 11 May 04:36:10.155 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis | 1:M 11 May 04:36:10.155 # Server initialized
redis | 1:M 11 May 04:36:10.156 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis | 1:M 11 May 04:36:10.156 * Ready to accept connections
server | /usr/lib/python2.7/dist-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
server | 'Supervisord is running as root and it is searching '
postgres | The database cluster will be initialized with locale "en_US.utf8".
postgres | The default database encoding has accordingly been set to "UTF8".
postgres | The default text search configuration will be set to "english".
postgres |
postgres | Data page checksums are disabled.
postgres |
postgres | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres | creating subdirectories ... ok
server | 2018-05-11 04:36:11,297 CRIT Supervisor running as root (no user in config file)
postgres | selecting default max_connections ... 100
server | 2018-05-11 04:36:11,305 INFO supervisord started with pid 1
postgres | selecting default shared_buffers ... 128MB
postgres | selecting dynamic shared memory implementation ... posix
postgres | creating configuration files ... ok
postgres | running bootstrap script ... ok
postgres | performing post-bootstrap initialization ... ok
postgres | syncing data to disk ...
postgres | WARNING: enabling "trust" authentication for local connections
postgres | You can change this by editing pg_hba.conf or using the option -A, or
postgres | --auth-local and --auth-host, the next time you run initdb.
postgres | ok
postgres |
postgres | Success. You can now start the database server using:
postgres |
postgres | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres |
postgres | ****************************************************
postgres | WARNING: No password has been set for the database.
postgres | This will allow anyone with access to the
postgres | Postgres port to access your database. In
postgres | Docker's default configuration, this is
postgres | effectively any other container on the same
postgres | system.
postgres |
postgres | Use "-e POSTGRES_PASSWORD=password" to set
postgres | it in "docker run".
postgres | ****************************************************
postgres | waiting for server to start....2018-05-11 04:36:12.001 UTC [41] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2018-05-11 04:36:12.037 UTC [42] LOG: database system was shut down at 2018-05-11 04:36:11 UTC
postgres | 2018-05-11 04:36:12.046 UTC [41] LOG: database system is ready to accept connections
postgres | done
postgres | server started
postgres | ALTER ROLE
postgres |
postgres |
postgres | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres |
postgres | 2018-05-11 04:36:12.285 UTC [41] LOG: received fast shutdown request
postgres | waiting for server to shut down....2018-05-11 04:36:12.289 UTC [41] LOG: aborting any active transactions
postgres | 2018-05-11 04:36:12.293 UTC [41] LOG: worker process: logical replication launcher (PID 48) exited with exit code 1
postgres | 2018-05-11 04:36:12.293 UTC [43] LOG: shutting down
server | 2018-05-11 04:36:12,309 INFO spawned: 'celery_beat' with pid 9
server | 2018-05-11 04:36:12,312 INFO spawned: 'celery_worker' with pid 10
server | 2018-05-11 04:36:12,315 INFO spawned: 'django' with pid 11
postgres | 2018-05-11 04:36:12.362 UTC [41] LOG: database system is shut down
postgres | done
postgres | server stopped
postgres |
postgres | PostgreSQL init process complete; ready for start up.
postgres |
postgres | 2018-05-11 04:36:12.414 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres | 2018-05-11 04:36:12.414 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres | 2018-05-11 04:36:12.422 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2018-05-11 04:36:12.451 UTC [59] LOG: database system was shut down at 2018-05-11 04:36:12 UTC
postgres | 2018-05-11 04:36:12.464 UTC [1] LOG: database system is ready to accept connections
server | celery beat v4.1.0 (latentcall) is starting.
web_client | yarn run v1.5.1
web_client | $ react-scripts start
server | 2018-05-11 04:36:14,248 INFO success: celery_beat entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
server | 2018-05-11 04:36:14,248 INFO success: celery_worker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
server | 2018-05-11 04:36:14,249 INFO success: django entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
server | /usr/local/lib/python3.6/site-packages/celery/platforms.py:795: RuntimeWarning: You're running the worker with superuser privileges: this is
server | absolutely not recommended!
server |
server | Please specify a different user using the -u option.
server |
server | User information: uid=0 euid=0 gid=0 egid=0
server |
server | uid=uid, euid=euid, gid=gid, egid=egid,
server |
server | -------------- celery@server v4.1.0 (latentcall)
server | ---- **** -----
server | --- * *** * -- Linux-4.9.87-linuxkit-aufs-x86_64-with-debian-8.10 2018-05-11 04:36:16
server | -- * - **** ---
server | - ** ---------- [config]
server | - ** ---------- .> app: polyledger:0x7eff5a955e10
server | - ** ---------- .> transport: redis://redis:6379//
server | - ** ---------- .> results: redis://redis:6379/
server | - *** --- * --- .> concurrency: 4 (prefork)
server | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
server | --- ***** -----
server | -------------- [queues]
server | .> celery exchange=celery(direct) key=celery
server |
server |
server | [tasks]
server | . api.tasks.allocate_for_user
server | . api.tasks.fill_daily_historical_prices
server | . api.tasks.fit_distributions
server | . api.tasks.get_current_prices
server | . api.tasks.send_confirmation_email
server | . polyledger.celery.debug_task
server |
server | No changes detected
server | Stale pidfile exists - Removing it.
server | __ - ... __ - _
server | LocalTime -> 2018-05-11 04:36:16
server | Configuration ->
server | . broker -> redis://redis:6379//
server | . loader -> celery.loaders.app.AppLoader
server | . scheduler -> celery.beat.PersistentScheduler
server | . db -> celerybeat-schedule.db
server | . logfile -> [stderr]@%INFO
server | . maxinterval -> 5.00 minutes (300s)
server | [2018-05-11 04:36:16,502: INFO/MainProcess] beat: Starting...
server | [2018-05-11 04:36:16,704: INFO/MainProcess] Scheduler: Sending due task get-new-day-prices (api.tasks.get_current_prices)
server | [2018-05-11 04:36:16,813: INFO/MainProcess] Connected to redis://redis:6379//
server | [2018-05-11 04:36:16,838: INFO/MainProcess] mingle: searching for neighbors
server | [2018-05-11 04:36:17,893: INFO/MainProcess] mingle: all alone
server | [2018-05-11 04:36:17,961: WARNING/MainProcess] /usr/local/lib/python3.6/site-packages/celery/fixups/django.py:202: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
server | warnings.warn('Using settings.DEBUG leads to a memory leak, never '
server | [2018-05-11 04:36:17,963: INFO/MainProcess] celery@server ready.
server | [2018-05-11 04:36:17,971: INFO/MainProcess] Received task: api.tasks.get_current_prices[2f800f82-8a1f-41b6-a46d-092a7e83df8f]
server | [2018-05-11 04:36:17,976: INFO/MainProcess] Received task: api.tasks.fill_daily_historical_prices[b69c86f5-d879-4739-a491-2da9669f6792]
server | [2018-05-11 04:36:18,037: ERROR/ForkPoolWorker-1] Task api.tasks.get_current_prices[2f800f82-8a1f-41b6-a46d-092a7e83df8f] raised unexpected: OperationalError('no such table: api_coin',)
server | Traceback (most recent call last):
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
server | return self.cursor.execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
server | return Database.Cursor.execute(self, query, params)
server | sqlite3.OperationalError: no such table: api_coin
server |
server | The above exception was the direct cause of the following exception:
server |
server | Traceback (most recent call last):
server | File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 374, in trace_task
server | R = retval = fun(*args, **kwargs)
server | File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 629, in __protected_call__
server | return self.run(*args, **kwargs)
server | File "/srv/polyledger/api/tasks.py", line 200, in get_current_prices
server | 'fsyms': ','.join(coins.values_list('symbol', flat=True)),
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 250, in __iter__
server | self._fetch_all()
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1118, in _fetch_all
server | self._result_cache = list(self._iterable_class(self))
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 152, in __iter__
server | for row in compiler.results_iter():
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 836, in results_iter
server | results = self.execute_sql(MULTI)
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
server | raise original_exception
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 884, in execute_sql
server | cursor.execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 80, in execute
server | return super(CursorDebugWrapper, self).execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
server | return self.cursor.execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
server | six.reraise(dj_exc_type, dj_exc_value, traceback)
server | File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
server | raise value.with_traceback(tb)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
server | return self.cursor.execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
server | return Database.Cursor.execute(self, query, params)
server | django.db.utils.OperationalError: no such table: api_coin
server | [2018-05-11 04:36:18,056: ERROR/ForkPoolWorker-3] Task api.tasks.fill_daily_historical_prices[b69c86f5-d879-4739-a491-2da9669f6792] raised unexpected: OperationalError('no such table: api_coin',)
server | Traceback (most recent call last):
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
server | return self.cursor.execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
server | return Database.Cursor.execute(self, query, params)
server | sqlite3.OperationalError: no such table: api_coin
server |
server | The above exception was the direct cause of the following exception:
server |
server | Traceback (most recent call last):
server | File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 374, in trace_task
server | R = retval = fun(*args, **kwargs)
server | File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 629, in __protected_call__
server | return self.run(*args, **kwargs)
server | File "/srv/polyledger/api/tasks.py", line 170, in fill_daily_historical_prices
server | for coin in coins:
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 250, in __iter__
server | self._fetch_all()
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1118, in _fetch_all
server | self._result_cache = list(self._iterable_class(self))
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 53, in __iter__
server | results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
server | raise original_exception
server | File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 884, in execute_sql
server | cursor.execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 80, in execute
server | return super(CursorDebugWrapper, self).execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
server | return self.cursor.execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
server | six.reraise(dj_exc_type, dj_exc_value, traceback)
server | File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
server | raise value.with_traceback(tb)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
server | return self.cursor.execute(sql, params)
server | File "/usr/local/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
server | return Database.Cursor.execute(self, query, params)
server | django.db.utils.OperationalError: no such table: api_coin
server | Operations to perform:
server | Apply all migrations: admin, api, auth, authtoken, contenttypes, sessions, sites
server | Running migrations:
server | Applying contenttypes.0001_initial... OK
server | Applying api.0001_initial... OK
server | Applying admin.0001_initial... OK
server | Applying admin.0002_logentry_remove_auto_add... OK
server | Applying contenttypes.0002_remove_content_type_name... OK
server | Applying auth.0001_initial... OK
server | Applying auth.0002_alter_permission_name_max_length... OK
server | Applying auth.0003_alter_user_email_max_length... OK
server | Applying auth.0004_alter_user_username_opts... OK
server | Applying auth.0005_alter_user_last_login_null... OK
server | Applying auth.0006_require_contenttypes_0002... OK
server | Applying auth.0007_alter_validators_add_error_messages... OK
server | Applying auth.0008_alter_user_username_max_length... OK
server | Applying authtoken.0001_initial... OK
server | Applying authtoken.0002_auto_20160226_1747... OK
server | Applying sessions.0001_initial... OK
server | Applying sites.0001_initial... OK
server | Applying sites.0002_alter_domain_unique... OK
server | [2018-05-11 04:36:22 +0000] [42] [INFO] Starting gunicorn 19.7.0
server | [2018-05-11 04:36:22 +0000] [42] [INFO] Listening at: http://0.0.0.0:8000 (42)
server | [2018-05-11 04:36:22 +0000] [42] [INFO] Using worker: sync
server | [2018-05-11 04:36:22 +0000] [45] [INFO] Booting worker with pid: 45
web_client | Starting the development server...
web_client |
web_client | Compiled successfully!
web_client |
web_client | You can now view polyledger in the browser.
web_client |
web_client | Local: http://localhost:3000/
web_client | On Your Network: http://172.19.0.5:3000/
web_client |
web_client | Note that the development build is not optimized.
web_client | To create a production build, use yarn build.
web_client |