racedb_app restarts every 2 minutes in Docker Desktop - RaceDB v3.0.86-20230724153441

[ChainGangEvents]

Created new installation from start (all old RaceDB folders, docker volumes, docker images & containers deleted:

System information:
Microsoft Windows 10 Pro
Version 10.0.19045 Build 19045
Processor i5-5300U CPU 2.30 GHz
12GB RAM
Docker Version: v4.21.1

RaceDB Version: v3.0.86-20230724153441

With racedb_app restarting every 2 minutes not able to open RaceDB application

[esitarski]

Thanks so much for giving this a try. I feel very badly about RaceDB not working. The person who did the docker install is no longer contributing to the project. As was inevitable, the standards for some docker components changed and this broke the install. It honestly wasn't my fault, but "something" has to be done to fix it. I have spent dozens of hours trying to make it work, and failed. I know little about docker and can only follow recipes on the internet. I think that docker is simple enough, but the components require a lot of expertise to choose and configure. Getting everything to work together is also poorly documented. Postgres 11 (used by RaceDB) goes out-of-support in January 2024, and should be updated to v15 in the docker build. I was falsely under the impression that docker could handle postgres database upgrades, but this has to be managed in the application. An extract will need to be run on the existing RaceDB database, then the data will have to be imported into the upgraded release. Just work to do. RaceDB really should not be using an internal web server, rather, it should use nginx (or similar). This is all possible with a proper docker build which I don't know how to do. I asked for help from the community, but nothing yet. I am pretty sure it would not take more than an hour or so for an expert to fix up the release. I would be happy to write the pre-upgrade extract script. Hopefully, someone knowledgeable can help out (suggestions?). If not, I will try to take another crack at fixing things in the next few weeks. I now have a new Windows-11 computer available. In the meantime, the project is not dead! I will keep adding features as a result of my work with Midweek weekly races (www.racedb.ca). By adding support for usb tag readers, RaceDB can operate 100% in the cloud as it does not need a local RFID reader. It took a while, but the workflow here has been completed. There is a new Animation and Gap Chart screen on the results page. A

…

On Tue, Jul 25, 2023 at 7:16 AM ChainGangEvents ***@***.***> wrote: Created new installation from start (all old RaceDB folders, docker volumes, docker images & containers deleted: System information: Microsoft Windows 10 Pro Version 10.0.19045 Build 19045 Processor i5-5300U CPU 2.30 GHz 12GB RAM Docker Version: v4.21.1 RaceDB Version: *v3.0.86-20230724153441* With racedb_app restarting every 2 minutes not able to open RaceDB application — Reply to this email directly, view it on GitHub <#50>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABGXKIUJWSKDYOE2SXP4TDXR6TIHANCNFSM6AAAAAA2W5DCRY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Edward Sitarski

[ChainGangEvents]

Thank you for the above Edward. I believe CrossMgr and RaceDB should always be utilized together. I will keep on testing if you want (I have another laptop that I can use), but I also want to keep up to date with all the advancements you are making with CrossMgr. Currently I am stuck on the following combo (CrossMgr 3.1.18-20230418175933 and RaceDB 3.0.82-20220801110523) as they are still stable for uploads from CrossMgr to RaceDB and vice versa. Would it be prudent to convert back to the original Windows install method that I used before the Docker-Container install method? I do not mind to occasionally update python, but at least I will be able to keep up with the advancements you are making.

Juan

[mbuckaway]

Quite Franky, Ed, every time the "docker install" hasn't worked, it's because you changed something, broke it, assumed it would work, and never tested it. You can run a local build and assume it will work. Additionally, you have to test the development package on Windows using a clean install and a upgrade install. It took me 2 mins to fix the issue ever time I looked.

Go back in change logs and undo your docker changes.

Yes, Postgress 15 will work....it was working, and then you down graded it back to postgress 11.

Testing, testing, testing., You can just change something without testing it.

Mark
The docker install guy

[mbuckaway]

AS usually, setup the container, and it took a minute to find the issue:

"django.db.utils.NotSupportedError: PostgreSQL 12 or later is required (found 11.20)"

....appeared in the logs.

Looks like @esitarski updated Django, and never tried it with Postgres.

The solution is to upgrade the postgress container to Postgress 15 (or whatever is current), make it work including whatever changes are required to make your python code work with the latest progres, and the explain to users how to migrate their data to the new version.

I would suggest stop using Sqlite for local testing and switch to postgress....and these problems won't happen.

Mark

[mbuckaway]

RaceDB has been upgraded to Postgres 15. This required a change to the startup script to set the correct permissions for Django to work. I've tested the container locally on my Mac. It started and served up the web pages. I don't have a Windows machine handy to test the Windows build.

@ChainGangEvents Please download the dev build when complete. I suggest running it on a different machine if you can just to make sure it starts for you. IF that works, please let us know, and we can push an "official" build. I don't suggest running a dev build for running events.

There is no upgrade process for moving to PG11 to PG15. Users must backup their data using the backup procedure defined in the docs, wipe the data volume (on linux/Mac, use racedb.sh clean - don't remember if that works on windows - its been a while). You can use the docker desktop GUI on Windows to wipe the data volume. Once the data volume is gone, you restart the container set, and restore your data.

If @esitarski to come up with a data migration script, great. The backup and restore should work just fine.

Mark

[esitarski]

Mark, many thanks for fixing the docker container. The situation is a bit more complex. The docker container spontaneously stopped working. The alpine image dropped useradd <https://www.google.com/search?q=Docker+build+error+%22useradd+command+not+found%22&oq=Docker+build+error+%22useradd+command+not+found%22&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIGCAEQRRg90gEHNzY2ajBqN6gCALACAA&sourceid=chrome&ie=UTF-8> , dropped git <https://www.google.com/search?q=Docker+build+error+%22git+command+not+found%22&sxsrf=AB5stBi7yGSKf00pqQ2Dct5aj5EiaZIzcw%3A1690459314660&ei=slzCZN3wJ_Wc0PEPovaD4AE&ved=0ahUKEwjdkdTX666AAxV1DjQIHSL7ABwQ4dUDCA8&uact=5&oq=Docker+build+error+%22git+command+not+found%22&gs_lp=Egxnd3Mtd2l6LXNlcnAiKkRvY2tlciBidWlsZCBlcnJvciAiZ2l0IGNvbW1hbmQgbm90IGZvdW5kIjIIECEYoAEYwwQyCBAhGKABGMMEMggQIRigARjDBDIIECEYoAEYwwQyCBAhGKABGMMESOkIUJ8GWNAHcAJ4AZABAJgBZaABjAKqAQMyLjG4AQPIAQD4AQHCAgoQABhHGNYEGLADwgIKECEYoAEYwwQYCuIDBBgAIEGIBgGQBgg&sclient=gws-wiz-serp>, and a few other commands used by the scripts. My changes were attempts at fixing these "minor" issues. I would never have touched this code otherwise. After many false attempts (yes, my fault), I managed to get the new install to work on my machine. However, this didn't fix existing users. Postgres 11 support expires in Jan 2024, and the Python version <https://devguide.python.org/versions/> is getting close to end-of-life too. I decided to upgrade them while "in the scripts". The Python upgrade was relatively straightforward (BTW, the goal should be to get to Python 3.11, which has much better performance). The Postgres upgrade created problems. Around this time, my windows machine died (it could only Windows 10 anyway), and I had my Linux machine only. The first problem is that more recent Postgres versions require explicit commands to enable a user to add tables to a database. Without this, Django throws 100s of Python exceptions (every time it tries to access the database). This gives the appearance of a serious problem with the RaceDB python code, when in fact, it's a database configuration problem. Fortunately, the new commands are valid in all Postgres versions and can be used unconditionally in all Postgres versions. The script was modified to do so. The second problem is that Postgres databases are not binary-compatible across major releases. If you simply upgrade the Postgres container, Postgres can't read the old volumes. In RaceDB, the "upgrade" command doesn't work. And to be fair to the RaceDB users, the github README.md doesn't have a section about how to do a Postgres upgrade. There are a number of ways to upgrade the database <https://betterprogramming.pub/how-to-upgrade-your-postgresql-version-using-docker-d1e81dbbbdf9>. Or this way <https://stackoverflow.com/questions/62790302/how-to-upgrade-my-postgres-in-docker-container-while-maintaining-my-data-10-3-t>. Or the data dump can be done from the RaceDB code (see the new RaceDB bin/backup.sh - bin/restore.sh commands). A path forward: - Perhaps the existing "update" command can be enhanced to handle a Postgres major upgrade automatically. - Perhaps a more sophisticated script is required. Ideally, it would check whether a major Postgres upgrade was required, and if so, dump the data, clean up the volumes, install the new Postgres, then restore the backups automatically. - If neither of these can work, the README.md documentation should include detailed steps required for a Postgres major version upgrade. - I have obtained a new Windows 11 computer and will set it up for testing with RaceDB. A path further forward: Stepping back, whether using the old installation method (admittedly, really terrible) or the docker container, RaceDB is a science project compared to CrossMgr. It takes a lot of technical knowledge and motivation. Most of the most technical CrossMgr users won't touch it. Perhaps this is why adoption of RaceDB is low - maybe a dozen running worldwide. One of the limitations of RaceDB was that it required a local RFID reader. This made it impossible to run from the cloud. Not anymore. I have worked hard to enhance it so that it works with cheap usb readers that connect with computers/tablets/smartphones. CrossMgr has also been upgraded so it more easily reads and writes directly to the cloud (it should be remembered that CrossMgr runs it does not need a connection to the internet. This is done for reliability - we don't want CrossMgr to crash if the internet drops during a race). The new RaceDB has been proven at the weekly series at Midweek Cycling Club where all the critical workflows are supported (check-in, validation, etc.). The limitation is that you cannot write tags on-the-fly. Some consider this a "feature" as it is safer, you can create all the tags you need ahead-of-time, or you can use randomly coded tags. Who cares if the tag is 24 chars of hex? You scan it. Folks need to let go of the idea that the tag should match the bib number. So, all changes are made in RaceDB on the cloud, then 30 seconds before the start of the race, we open CrossMgr. It syncs with RaceDB, and we start. You can even change data in RaceDB and re-open the race in CrossMgr. This will update the rider information but preserve the timing and results. After the race is done and we are finished with any corrections, results are published back to RaceDB (see www.racedb.ca for details). With the new features, there is no reason why RaceDB could not be centrally hosted as a multi-tenant web site. This would avoid the install entirely. Thoughts and suggestions?

…

On Wed, Jul 26, 2023 at 9:24 PM Mark Buckaway ***@***.***> wrote: RaceDB has been upgraded to Postgres 15. This required a change to the startup script to set the correct permissions for Django to work. I've tested the container locally on my Mac. It started and served up the web pages. I don't have a Windows machine handy to test the Windows build. @ChainGangEvents <https://github.com/ChainGangEvents> Please download the dev build when complete. I suggest running it on a different machine if you can just to make sure it starts for you. IF that works, please let us know, and we can push an "official" build. I don't suggest running a dev build for running events. There is no upgrade process for moving to PG11 to PG15. Users must backup their data using the backup procedure defined in the docs, wipe the data volume (on linux/Mac, use racedb.sh clean - don't remember if that works on windows - its been a while). You can use the docker desktop GUI on Windows to wipe the data volume. Once the data volume is gone, you restart the container set, and restore your data. If @esitarski <https://github.com/esitarski> to come up with a data migration script, great. The backup and restore should work just fine. Mark — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABGXKM4HBEDQLWPBTTGQ5DXSG7N5ANCNFSM6AAAAAA2W5DCRY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Edward Sitarski

[ChainGangEvents]

@mbuckaway @esitarski

I successfully installed the latest Development Build RaceDB v3.0.86-beta-973cbc9 with the postgress 15 working and Docker Containers running smoothly. Container now up for more than an hour without any problems.

The restore process via the RaceDB Controller also worked and my database is complete and running in the Dev Build.

I've since downloaded the latest release of CrossMgr ( v3.1.29-20230614115457) to see if I can manage to link RaceDB with CrossMgr, but I think I've run out of skillset (I am really not good with setting IP addresses etc)

Let me know how I can be of more assistance, but please keep in mind my limited knowledge.

[ChainGangEvents]

@esitarski I see your comments about RaceDB being run in the Cloud, with all race changes done in the Cloud base DB and just before the event do the sync with CrossMgr. All good and well if you live in a 1st World country with good internet access, as well as continuous electricity supply. Here we have rolling loadshedding where the power (and subsequent internet and cellphone lines) go off for up to 4 hours - they actually forecast that it will soon go up to 7 hours. The main attraction for me about RaceDB was the fact that the database runs on a local server. Some of my events are in areas where even if there is power there are no internet or cellphone reception and with the database on the local server it works brilliantly. Our information protection act (Poppy Act) also require very stringent rules in regards to information about participants that makes cloud configurations risky, even with password protections etc.

[esitarski]

Point taken. There are no plans to discontinue stand-alone RaceDB. To tell the truth, after putting 1000's of hours into the application, it is disappointing that there are probably fewer than 12 implementations worldwide. Suggestions for improvements are welcome.

…

On Fri, Jul 28, 2023 at 10:21 AM ChainGangEvents ***@***.***> wrote: @esitarski <https://github.com/esitarski> I see your comments about RaceDB being run in the Cloud, with all race changes done in the Cloud base DB and just before the event do the sync with CrossMgr. All good and well if you live in a 1st World country with good internet access, as well as continuous electricity supply. Here we have rolling loadshedding where the power (and subsequent internet and cellphone lines) go off for up to 4 hours - they actually forecast that it will soon go up to 7 hours. The main attraction for me about RaceDB was the fact that the database runs on a local server. Some of my events are in areas where even if there is power there are no internet or cellphone reception and with the database on the local server it works brilliantly. Our information protection act (Poppy Act) also require very stringent rules in regards to information about participants that makes cloud configurations risky, even with password protections etc. — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABGXKLPGFGAO52FFHH3FNTXSPDEZANCNFSM6AAAAAA2W5DCRY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Edward Sitarski

[ChainGangEvents]

@esitarski the latest CrossMgr release (CrossMgr_3_1_30-20230728155824) fixed the sync issue with RaceDB - thank you. I will do some more testing with RaceDB v3.0.86-beta-973cbc9 and revert back to you with any issues.

[ChainGangEvents]

@esitarski I tried to "Import a Competition from File", but got the below error message.

RemovedInDjango50Warning at /RaceDB/Competitions/CompetitionImport/

Passing unsaved model instances to related filters is deprecated.

Request Method:	POST
http://localhost:8000/RaceDB/Competitions/CompetitionImport/
4.2.3
RemovedInDjango50Warning
Passing unsaved model instances to related filters is deprecated.
/usr/local/lib/python3.10/site-packages/django/db/models/fields/related_lookups.py, line 52, in get_normalized_value
core.WriteLog.new_f
/usr/local/bin/python
3.10.12
['/RaceDB', '/RaceDB', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages', '/RaceDB/src/pyllrp']
Sat, 29 Jul 2023 06:36:19 -0400

[esitarski]

OK. Can you send me the file of the Competition you are trying to import? Thanks.

…

On Sat, Jul 29, 2023 at 6:40 AM ChainGangEvents ***@***.***> wrote: @esitarski <https://github.com/esitarski> I tried to "*Import a Competition from File*", but got the below error message. RemovedInDjango50Warning at /RaceDB/Competitions/CompetitionImport/ Passing unsaved model instances to related filters is deprecated. Request Method: POST http://localhost:8000/RaceDB/Competitions/CompetitionImport/ 4.2.3 RemovedInDjango50Warning Passing unsaved model instances to related filters is deprecated. /usr/local/lib/python3.10/site-packages/django/db/models/fields/related_lookups.py, line 52, in get_normalized_value core.WriteLog.new_f /usr/local/bin/python 3.10.12 ['/RaceDB', '/RaceDB', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages', '/RaceDB/src/pyllrp'] Sat, 29 Jul 2023 06:36:19 -0400 — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABGXKJQZDNYIFXFJTG45O3XSTSB7ANCNFSM6AAAAAA2W5DCRY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Edward Sitarski

[ChainGangEvents]

@esitarski @mbuckaway I've updated to RaceDB v3.0.87-20230729105401. Every thing working, Database complete, HUB, integration with latest CrossMgr_3_1_31-20230729113344 release. I will be running some tests with CrossMgr Impinji and the other applications as well and will give feedback ASAP.

[ChainGangEvents]

@esitarski is this something that can be fixed easily, or should I revert back to the previous RaceDB version? I am preparing for a race and at this stage can not add "Mass Start Events" without getting the below error.

Issue #49 is still relevant, as I still get the below error message when adding a Mass Start Event:

ValueError at /RaceDB/Competitions/CompetitionDashboard/192/EventMassStartNew/192/

'EventMassStart' instance needs to have a primary key value before this relationship can be used.

Request Method:	GET
http://localhost:8000/RaceDB/Competitions/CompetitionDashboard/192/EventMassStartNew/192/
4.2.3
ValueError
'EventMassStart' instance needs to have a primary key value before this relationship can be used.
/usr/local/lib/python3.10/site-packages/django/db/models/fields/related_descriptors.py, line 718, in get_queryset
core.WriteLog.new_f
/usr/local/bin/python
3.10.12
['/RaceDB', '/RaceDB', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages', '/RaceDB/src/pyllrp']
Sun, 30 Jul 2023 12:11:29 -0400

[esitarski]

Apologies for the delay. This is now fixed in the latest version.

…

On Sun, Jul 30, 2023 at 12:13 PM ChainGangEvents ***@***.***> wrote: Issue #49 <#49> is still relevant, as I still get the below error message when adding a Mass Start Event: ValueError at /RaceDB/Competitions/CompetitionDashboard/192/EventMassStartNew/192/ 'EventMassStart' instance needs to have a primary key value before this relationship can be used. Request Method: GET http://localhost:8000/RaceDB/Competitions/CompetitionDashboard/192/EventMassStartNew/192/ 4.2.3 ValueError 'EventMassStart' instance needs to have a primary key value before this relationship can be used. /usr/local/lib/python3.10/site-packages/django/db/models/fields/related_descriptors.py, line 718, in get_queryset core.WriteLog.new_f /usr/local/bin/python 3.10.12 ['/RaceDB', '/RaceDB', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages', '/RaceDB/src/pyllrp'] Sun, 30 Jul 2023 12:11:29 -0400 — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABGXKONTRNGHJ3L56WWA73XS2BZPANCNFSM6AAAAAA2W5DCRY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Edward Sitarski

[ChainGangEvents]

Thank you Edward. I installed the latest and it is working perfect. I will the issue on GitHub.Thank youJuan On 21 Aug 2023, at 16:50, Edward Sitarski ***@***.***> wrote: Apologies for the delay. This is now fixed in the latest version.

…

On Sun, Jul 30, 2023 at 12:13 PM ChainGangEvents ***@***.***> wrote: Issue #49 <#49> is still relevant, as I still get the below error message when adding a Mass Start Event: ValueError at /RaceDB/Competitions/CompetitionDashboard/192/EventMassStartNew/192/ 'EventMassStart' instance needs to have a primary key value before this relationship can be used. Request Method: GET http://localhost:8000/RaceDB/Competitions/CompetitionDashboard/192/EventMassStartNew/192/ 4.2.3 ValueError 'EventMassStart' instance needs to have a primary key value before this relationship can be used. /usr/local/lib/python3.10/site-packages/django/db/models/fields/related_descriptors.py, line 718, in get_queryset core.WriteLog.new_f /usr/local/bin/python 3.10.12 ['/RaceDB', '/RaceDB', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages', '/RaceDB/src/pyllrp'] Sun, 30 Jul 2023 12:11:29 -0400 — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABGXKONTRNGHJ3L56WWA73XS2BZPANCNFSM6AAAAAA2W5DCRY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Edward Sitarski —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: ***@***.***>