Islandora-Devops/isle-site-template

Fedora resources are all 404's; container has database problems

Closed this issue · 7 comments

rosiel commented

I'm not sure if these are related.

1 - I can't write to Fedora. When I create a node then click the Fedora URI, it's a 404. When I try to upload a file, if I'm uploading at the default monthly folder which hasn't been created yet (2023-09), I get the error that everyone's been posting in Slack, upload directory fedora://2023-09 for the file field field_media_file could not be created or is not accessible.. And - what's potentially even worse, if I take out that folder so I'm writing straight to the top level e.g. fedora://dead_bug.gif, then my file appears to upload and the media saves, though trying to access the file (manually or for derivatives) yeilds a 404.

Screenshot 2023-09-25 at 5 27 44 PM

2 - The fedora container has a bunch of logs (only visible in Docker Desktop as far as I can tell) complaining about a lack of database connection:

2023-09-25 17:19:22 SQL State  : 90020
2023-09-25 17:19:22 Error Code : 90020
2023-09-25 17:19:22 Message    : Database may be already in use: "Locked by another computer: 172.22.0.14". 
Possible solutions: close all other connection(s); use the server mode [90020-210]
rosiel commented

In diving into where this database connection may be, I see some environment variables in the fcrepo container that look promising:

/opt/tomcat # env | grep DB | sort
DB_DRIVER=mysql
DB_HOST=
DB_MYSQL_HOST=mariadb
DB_MYSQL_PORT=3306
DB_NAME=default
DB_PASSWORD=password
DB_PORT=
DB_POSTGRESQL_HOST=postgresql
DB_POSTGRESQL_PORT=5432
DB_ROOT_PASSWORD=password
DB_ROOT_USER=root
DB_USER=default
FCREPO_DB_NAME=fcrepo
FCREPO_DB_PASSWORD=password
FCREPO_DB_USER=fcrepo

I assume that DB_DRIVER selects the MYSQL values for HOST and PORT (because they're otherwise missing). This would imply there's a fcrepo database in the mariadb container that's busy.

In the MariaDB container,

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| drupal_default     |
| information_schema |
| matomo             |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
6 rows in set (0.006 sec)

There does not seem to be a database set up for Fedora.

I had thought that the standard for Fedora was to use Postgres. Could your Fedora be set up to use that, even it if might not exist?

By default, unless otherwise configured, Fedora will use its file storage mechanism rather than a database.

https://github.com/Islandora-Devops/isle-buildkit/blob/c9ab7e77665a390a0a73124686339cf9a08881b1/fcrepo6/Dockerfile#L57

This seemed to have worked for years? I merged this 3 weeks ago that updated the FCREPO by a minor version Islandora-Devops/isle-buildkit@740f687

This could have dropped that feature? Or it behaves differently now https://wiki.lyrasis.org/display/FEDORA6x/Database+Cache

I would have hoped the automatic tests would have caught that though.

We don't have an automatic test for the file backend...

https://github.com/Islandora-Devops/isle-buildkit/tree/main/fcrepo6/tests

I've added a test, it appears to be working.

Islandora-Devops/isle-buildkit@20ce607

I've actually seen this bug before, some time ago....

Islandora/documentation#2094

Seems to be handled by pull #23