datapusher-plus-uwsgi:datapusher-plus-uwsgi-00 FATAL Exited too quickly (process log may have details)

[joelriverac]

There was a detail missing from the installation instructions, here's my analysis and how I solved it, although I still have a problem when trying to upload to the datastore, I suspect it's the API KEY parameter, which probably could also be included in the installation process:

I use ckan package version 2.9 Ubuntu Server 20.04

Installing the plugin Datapusher-Plus (DP+), after carrying out all the steps, I get the following error:

ckan-datapusher:ckan-datapusher-00 RUNNING pid 1656, uptime 0:00:22
ckan-uwsgi:ckan-uwsgi-00 RUNNING pid 1657, uptime 0:00:22
ckan-worker:ckan-worker-00 RUNNING pid 1658, uptime 0:00:22
**datapusher-plus-uwsgi:datapusher-plus-uwsgi-00 FATAL Exited too quickly (process log may have details)**

I think the problem is that I have to stop the datapusher that comes with ckan, but is it correct? If so, how to stop it?

Indeed, the datapusher-plus uses the same port 8800
I understand what needs to be done is to remove the original datapusher configuration file /etc/supervisor/conf.d/datapusher-uwsgi.conf and run:

sudo supervisorctl reread
sudo supervisorctl update

And yes! the problem gone

ckan-uwsgi:ckan-uwsgi-00                         RUNNING   pid 1068, uptime 0:50:55
ckan-worker:ckan-worker-00                       RUNNING   pid 1069, uptime 0:50:55
datapusher-plus-uwsgi:datapusher-plus-uwsgi-00   RUNNING   pid 1070, uptime 0:50:55

So datapusher-plus to be the only one to load

Config DP+ /etc/ckan/datapusher-plus/uwsgi.ini

[uwsgi]
http=0.0.0.0:8800
uid = www-data
guid = www-data
virtualenv = /usr/lib/ckan/dpplus_venv
module = datapusher.wsgi:application
master=true
pidfile = /tmp/%n.pid
harakiri = 50
max-requests = 5000
vacuum = true
buffer-size = 32768

see High Availability Setup
workers = 3
thread = 3
lazy-apps=true

ckan data pusher original
/etc/ckan/datapusherdatapusher-uwsgi.ini

[uwsgi]
http=127.0.0.1:8800
uid = www-data
guid = www-data
wsgi-file = /etc/ckan/datapusher/datapusher.wsgi
virtualenv = /usr/lib/ckan/datapusher
master=true
pidfile = /tmp/%n.pid
harakiri = 50
max-requests = 5000
vacuum = true
callable = application
buffer-size = 32768

datapusher_plus-uwsgi.OUT contains no records

I have datapusher_plus-uwsgi.ERR here:
datapusher_plus-uwsgi.ERR

The last boot record in datapusher_plus-uwsgi.ERR is:

Starting uWSGI 2.0.21 (64bit) on [Wed Feb 1 08:08:11 2023]
compiled with version: 9.4.0 on 31 January 2023 14:07:28
os: Linux-5.15.0-58-generic #64~20.04.1-Ubuntu SMP Fri Jan 6 16:42:31 UTC 2023
nodename: CkanSrv
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /
writing pidfile to /tmp/uwsgi.pid
detected binary path: /usr/lib/ckan/dpplus_venv/bin/uwsgi
setuid() to 33
your process number limit is 7560
your memory page size is 4096 bytes
WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
probably another instance of uWSGI is running on the same address (0.0.0.0:8800).
bind(): Address already in use [core/socket.c line 769]

By the way, where do I find the API Key to set in the parameter: ckan.datapusher.api_token = <api_token> ?

I have already managed to leave DP+ active, but I have this error when I try to upload a data

Upload error: An Error occurred while sending the job: 500 Server Error: Internal Server Error for url: http://127.0.0.1:8800/job

I suspect I am missing the above parameter (ckan.datapusher.api_token = <api_token> ) in my ckan.ini

At this point I am now, I hope you can help me to be able to solve and comment on the conclusion of the story and that it helps with the installation and configuration process of the DP+

[joelriverac]

Update
I created the API Key from the web interface, here the process:
http://localhost/user/admin/api-tokens

I set the API Key in ckan.ini sudo nano /etc/ckan/default/ckan.ini

ckan.datapusher.api_token = string:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJEVEg0eU5DQWthOU41aHpIaWJZVUt3eERBdGxoSVJ4XzVzZVZUNmxuOVpXQmNGcC1HcDktbk9KZmFSRWFlYnlhSEVsQk1HWkhIQlBIcUFJTiIsImlhdCI6MTY3NTM0NjU4N30.CiRo9gG2F0e5GlqEA7KMh31gjnJOPB9Eg0H3UbHfwO4

The above does not solve the problem
Observing the ckan error.log, an error can be seen, apparently it is a werkseug version problem, I downloaded the version with the following:
sudo env "PATH=$PATH" pip install werkzeug==2.0.3
The log error is corrected, I have no errors in the logs, but the web interface error persists

Reviewing, I notice that it was missing to establish:
SQLALCHEMY_DATABASE_URI=postgresql://datapusher_jobs:YOURPASSWORD@localhost/datapusher_jobs
in deployment/datapusher_settings.py

But, it doesn't fix the error either

[jqnatividad]

Thanks for the very detailed report @joelriverac . I'll get back with some of my findings later today...