Migration error
Opened this issue · 12 comments
Following Docker installation guide, and hit with an error when creating django-migrations container for the first time. Here is the error -
Traceback (most recent call last):
File "/home/jovyan/py-trading-bot/manage.py", line 22, in
main()
File "/home/jovyan/py-trading-bot/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/opt/conda/lib/python3.9/site-packages/django/core/management/init.py", line 442, in execute_from_command_line
utility.execute()
File "/opt/conda/lib/python3.9/site-packages/django/core/management/init.py", line 416, in execute
django.setup()
File "/opt/conda/lib/python3.9/site-packages/django/init.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/opt/conda/lib/python3.9/site-packages/django/apps/registry.py", line 124, in populate
app_config.ready()
File "/home/jovyan/py-trading-bot/orders/apps.py", line 13, in ready
if len(StockStatus.objects.all())==0:
File "/opt/conda/lib/python3.9/site-packages/django/db/models/query.py", line 380, in len
self._fetch_all()
File "/opt/conda/lib/python3.9/site-packages/django/db/models/query.py", line 1881, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/opt/conda/lib/python3.9/site-packages/django/db/models/query.py", line 91, in iter
results = compiler.execute_sql(
File "/opt/conda/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
cursor.execute(sql, params)
File "/opt/conda/lib/python3.9/site-packages/django/db/backends/utils.py", line 102, in execute
return super().execute(sql, params)
File "/opt/conda/lib/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/opt/conda/lib/python3.9/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/opt/conda/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/opt/conda/lib/python3.9/site-packages/django/db/utils.py", line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/conda/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "orders_stockstatus" does not exist
LINE 1: ...tegy_id", "orders_stockstatus"."order_in_ib" FROM "orders_st...
I hit the same problem running with WSL2:
Traceback (most recent call last):
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/utils/autoreload.py", line 64, in wrapper
fn(*args, **kwargs)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/commands/runserver.py", line 125, in inner_run
autoreload.raise_last_exception()
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/utils/autoreload.py", line 87, in raise_last_exception
raise _exception[1]
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/__init__.py", line 394, in execute
autoreload.check_errors(django.setup)()
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/utils/autoreload.py", line 64, in wrapper
fn(*args, **kwargs)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/apps/registry.py", line 124, in populate
app_config.ready()
File "/home/hmei/py-trading-bot/py-trading-bot/orders/apps.py", line 13, in ready
if len(StockStatus.objects.all())==0:
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/models/query.py", line 382, in __len__
self._fetch_all()
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/models/query.py", line 1928, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/models/query.py", line 91, in __iter__
results = compiler.execute_sql(
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
cursor.execute(sql, params)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 122, in execute
return super().execute(sql, params)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "orders_stockstatus" does not exist
LINE 1: ...tegy_id", "orders_stockstatus"."order_in_ib" FROM "orders_st...
Ok, I will try it on my side today evening. Sorry for the inconvenience.
I confirm that there are some issues there. I will work on it.
So I added a
try:
if len(StockStatus.objects.all())==0:
create_ss_sub()
except:
pass
To handle the problem.
In addition, I added a try/except to handle the fact that keras may not be installed.
I hope it will work on your side.
Hi @psemdel,
Thanks a lot for spending time looking into the issue. I tried running again with my wsl2 environment, however there is still a problem with the missing relation.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
cursor.execute(sql, params)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 122, in execute
return super().execute(sql, params)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "orders_stockex" does not exist
LINE 1: ...in_index_id", "orders_stockex"."calc_report" FROM "orders_st...
^
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/hmei/py-trading-bot/py-trading-bot/reporting/views.py", line 14, in reportsView
return render(request, 'reporting/reports.html', context)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/shortcuts.py", line 25, in render
content = loader.render_to_string(template_name, context, request, using=using)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/loader.py", line 62, in render_to_string
return template.render(context, request)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/backends/django.py", line 61, in render
return self.template.render(context)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/base.py", line 171, in render
return self._render(context)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/base.py", line 961, in render_annotated
return self.render(context)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/base.py", line 1065, in render
return render_value_in_context(output, context)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/template/base.py", line 1042, in render_value_in_context
value = str(value)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/forms/utils.py", line 79, in __str__
return self.as_widget()
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/forms/boundfield.py", line 108, in as_widget
return widget.render(
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/forms/widgets.py", line 278, in render
context = self.get_context(name, value, attrs)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/forms/widgets.py", line 764, in get_context
context = super().get_context(name, value, attrs)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/forms/widgets.py", line 715, in get_context
context["widget"]["optgroups"] = self.optgroups(
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/forms/widgets.py", line 655, in optgroups
for index, (option_value, option_label) in enumerate(self.choices):
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django_filters/fields.py", line 256, in __iter__
yield from iterable
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/forms/models.py", line 1422, in __iter__
for obj in queryset:
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/models/query.py", line 518, in _iterator
yield from iterable
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/models/query.py", line 91, in __iter__
results = compiler.execute_sql(
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1565, in execute_sql
cursor.close()
psycopg2.errors.InvalidCursorName: cursor "_django_curs_139757766047296_sync_1" does not exist
[24/Mar/2024 11:22:35] "GET / HTTP/1.1" 500 211282
PS, I tried running the optional 'import dump.json' as well, and got the following error:
python3 manage.py loaddata dump.json
Traceback (most recent call last):
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/serializers/json.py", line 69, in Deserializer
objects = json.loads(stream_or_string)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 6 (char 5)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/hmei/py-trading-bot/py-trading-bot/manage.py", line 22, in <module>
main()
File "/home/hmei/py-trading-bot/py-trading-bot/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
utility.execute()
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/commands/loaddata.py", line 102, in handle
self.loaddata(fixture_labels)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/commands/loaddata.py", line 163, in loaddata
self.load_label(fixture_label)
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/management/commands/loaddata.py", line 251, in load_label
for obj in objects:
File "/home/hmei/miniconda3/envs/vectorbtpro/lib/python3.10/site-packages/django/core/serializers/json.py", line 74, in Deserializer
raise DeserializationError() from exc
django.core.serializers.base.DeserializationError: Problem installing fixture '/home/hmei/py-trading-bot/py-trading-bot/dump.json':
Cheers.
No problem, I like also when the code is working.
The errors you see, django_curs_139757766047296..., are because the migration did not take place. Check the migration job.
For the dump, there was that at the beginning (and yes, no idea, why I did not see that yesterday):
[2023-12-08 16:47:11] INFO 159 NumExpr defaulting to 4 threads.
I deleted it, it seems fine now.
I use Python 3.9. I am building an environment with Python 3.10... to test it with it.
It seems to work also with Python 3.10
Migration worked for me after the fix, but the bot has been throwing the following error -
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"container \"py-trading-bot\" in pod \"py-trading-bot- │
│6879c9b769-tjqwd\" is waiting to start: CreateContainerConfigError","reason":"BadRequest","code":400}
That's the log?
Yes.
I need to investigate more in details... Probably this weekend. I suppose, I still have to work on this docker image.
So sorry, that it took me so long. I am quite busy right now. So I did it all step by step (each yaml file applied one after the others, checking each step). I confirm it works. To be even more sure, I will use from now the Docker version on daily basis, normally I just run it locally.
Concerning your error, it is often caused by a missing configuration map or secret. Did you do the:
kubectl apply -f py-trading-bot/kubernetes/configmap.yml
kubectl apply -f py-trading-bot/kubernetes/secret.yml
?
Moreover, I would advise to start again from scratch. I had also problems, because I thought that I was working with a clean DB, but now, it was the old one. So did you remove the PV and PVC before retrying. I would sugger to delete your cluster (minikube delete). And try again. With DB half migrated and systems half in their old state and half in their new state, it can quickly become messy.