Error with renew_certificate.sh

Question

Error with renew_certificate.sh

zimtls opened this issue 4 years ago · 45 comments

Hello,

I just installed the letsencrypt certificate on my QNAP NAS, i've follow your instructions but i have an error with the script : renew_certificate.sh

My setup is OK, i have python 3.5, my NAS is reachable from the public internet.

Error :

[/share/CACHEDEV1_DATA/qnap-letsencrypt] # ./renew_certificate.sh
Checking whether to renew certificate on Sat, 19 Jun 2021 14:16:21 +0200
Renewing certificate...
qnap-letsencrypt version: 21afc01
Using python path: python3
Stopping Qthttpd hogging port 80..
Shutting down Qthttpd services: OK.
Started python HTTP server with pid 28585
Parsing account key...
Parsing CSR...
Found domains: nas.<Mydomain>
Getting directory...
Traceback (most recent call last):
  File "/share/CACHEDEV1_DATA/qnap-letsencrypt/acme-tiny/acme_tiny.py", line 198, in <module>
    main(sys.argv[1:])
  File "/share/CACHEDEV1_DATA/qnap-letsencrypt/acme-tiny/acme_tiny.py", line 194, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact)
  File "/share/CACHEDEV1_DATA/qnap-letsencrypt/acme-tiny/acme_tiny.py", line 106, in get_crt
    directory, _, _ = _do_request(directory_url, err_msg="Error getting directory")
  File "/share/CACHEDEV1_DATA/qnap-letsencrypt/acme-tiny/acme_tiny.py", line 46, in _do_request
    raise ValueError("{0}:\nUrl: {1}\nData: {2}\nResponse Code: {3}\nResponse: {4}".format(err_msg, url, data, code, resp_data))
ValueError: Error getting directory:
Url: https://acme-v02.api.letsencrypt.org/directory
Data: None
Response Code: None
Response: <urlopen error unknown url type: https>
An error occured. Restoring system state.
Start apache proxy: OK
./renew_certificate.sh: line 11: 28585 Killed                  "$PYTHON" ../HTTPServer.py  (wd: /share/CACHEDEV1_DATA/qnap-letsencrypt/tmp-webroot)
Recover apache confiugre
Starting Qthttpd services: Qthttpd.

Any ideas for this problem ?

Thank's in advanced,

LennyM8472 commented 3 years ago

nothing

Answer 1 · 2021-06-19T12:44:31.000Z

Hi,
Can you clarify how you installed Python?

Answer 2 · 2021-06-19T12:48:36.000Z

Hi,
I install Python 3.5 app, with GUI / app center.

Answer 3 · 2021-06-19T14:22:24.000Z

Try reinstating it

Answer 4 · 2021-06-19T14:36:06.000Z

I've reinstating it, but same error 😅

Answer 5 · 2021-06-19T14:56:34.000Z

Try to diagnose following the README section How to test whether a python script fails due to missing ca certificates

You can also check this old issue: #26

Answer 6 · 2021-07-29T13:37:48.000Z

You could try finding all python executables on your system with something like this: find / -type f -name "python*", and execute this test for each.

Answer 7 · 2021-10-17T12:03:24.000Z

Hi,

I now realize I have the same problem.

Python3 is installed via QNAP Appstore - path is /share/CE_CACHEDEV2_DATA/.qpkg/Python3

Traceback (most recent call last): File "acme-tiny/acme_tiny.py", line 198, in <module> main(sys.argv[1:]) File "acme-tiny/acme_tiny.py", line 194, in main signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact) File "acme-tiny/acme_tiny.py", line 106, in get_crt directory, _, _ = _do_request(directory_url, err_msg="Error getting directory") File "acme-tiny/acme_tiny.py", line 46, in _do_request raise ValueError("{0}:\nUrl: {1}\nData: {2}\nResponse Code: {3}\nResponse: {4}".format(err_msg, url, data, code, resp_data)) ValueError: Error getting directory: Url: https://acme-v02.api.letsencrypt.org/directory Data: None Response Code: None Response: <urlopen error unknown url type: https> An error occured. Restoring system state. Start apache proxy: OK

Answer 8 · 2021-10-19T08:27:58.000Z

@LennyM8472 Please try finding all python executables on your system (find / -type f -name "python*"), and execute this test for each.

Answer 9 · 2021-10-19T08:35:02.000Z

Hi,

I get this:

`# python
Python 3.8.5 (default, Sep 10 2020, 19:39:36)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

from urllib.request import urlopen # Python 3
urlopen("https://google.com")
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'urllib.request'
Traceback (most recent call last):
File "", line 1, in
NameError: name 'urlopen' is not defined
`

Answer 10 · 2021-10-19T08:36:04.000Z

@LennyM8472 Did you try this with all python executables on the system?

What's the output of find / -type f -name "python*"?

Answer 11 · 2021-10-19T08:38:45.000Z

I get this plus a lot of thing sin Docker containers

/usr/share/Python/bin/python

Answer 12 · 2021-10-19T08:40:36.000Z

@LennyM8472 That confirms that you did not install python from a qpkg (through qnap store)

Answer 13 · 2021-10-19T08:43:23.000Z

I did 100%

Answer 14 · 2021-10-19T08:45:03.000Z

What's the output of /sbin/getcfg Python3 Install_Path -f /etc/config/qpkg.conf?

Answer 15 · 2021-10-19T08:45:39.000Z

/share/CE_CACHEDEV2_DATA/.qpkg/Python3

Answer 16 · 2021-10-19T08:46:37.000Z

Output of find /share/CE_CACHEDEV2_DATA/.qpkg/Python3 -type f -name "python*"?

Answer 17 · 2021-10-19T08:47:15.000Z

[/usr/share/Python/bin] # find /share/CE_CACHEDEV2_DATA/.qpkg/Python3 -type f -name "python*" /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/pkgconfig/python3.pc /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/pkgconfig/python-3.5m.pc /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/pkgconfig/python-3.5.pc /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/idlelib/Icons/python.gif /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.sgi /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.tiff /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.ras /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.webp /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.png /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.exr /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.gif /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.ppm /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.bmp /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.xbm /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.pgm /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.pbm /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/test/imghdrdata/python.jpg /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/config-3.5m/python-config.py /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/config-3.5m/python.o /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3-config /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3.5m /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3 /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3.5 /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3.5-config /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3.5m-config /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/include/python3.5m/pythonrun.h /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/share/man/man1/python3.1 /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/share/man/man1/python3.5.1 /share/CE_CACHEDEV2_DATA/.qpkg/Python3/python3.bash

Answer 18 · 2021-10-19T08:50:23.000Z

Try the test (https://github.com/Yannik/qnap-letsencrypt#how-to-test-whether-a-python-script-fails-due-to-missing-ca-certificates) with /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3

Answer 19 · 2021-10-19T08:53:26.000Z

[/share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin] # python3
Python 3.8.5 (default, Sep 10 2020, 19:39:36)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

from urllib.request import urlopen # Python 3
urlopen("https://google.com")
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'urllib.request'
Traceback (most recent call last):
File "", line 1, in
NameError: name 'urlopen' is not defined

Answer 20 · 2021-10-19T08:54:28.000Z

Please try again for /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3.5

Answer 21 · 2021-10-19T08:57:36.000Z

[/share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin] # /share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin/python3.5
Python 3.5.2 (default, Aug 10 2016, 18:01:45)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.

from urllib.request import urlopen # Python 3
urlopen("https://google.com")
Traceback (most recent call last):
File "", line 1, in
File "/share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/urllib/request.py", line 489, in _open
'unknown_open', req)
File "/share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/lib/python3.5/urllib/request.py", line 1324, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib.error.URLError:

Answer 22 · 2021-10-19T09:02:04.000Z

Hm, that is very interesting. Seems like the python3.5 qpkg you installed does not ship with a urllib that knows https...

Whats your output of

grep -e ^Platform -e ^DISPLAY_NAME  /etc/platform.conf
grep -e ^Version -e ^Build -e Model -e "\[" /etc/default_config/uLinux.conf | grep -v "\[System\]" | awk '1;/\[/{exit}' |grep -v "\["

Answer 23 · 2021-10-19T09:04:40.000Z

[/share/CE_CACHEDEV2_DATA/.qpkg/Python3/src/bin] # grep -e ^Version -e ^Build -e Model -e "[" /etc/default_config/uLinux.conf | grep -v "[System]" | awk '1;/[/{exit}' |grep -v "["
Model = TS-X82
Internal Model = TS-X82
Version = 5.0.0
Build Number = 20211001
Rsync Model = QNAP
Build Date = 2021-10-01

Answer 24 · 2021-10-19T09:05:42.000Z

Hm, that is very interesting. Seems like the python3.5 qpkg you installed does not ship with a urllib that knows https...

Whats your output of
grep -e ^Platform -e ^DISPLAY_NAME  /etc/platform.conf
grep -e ^Version -e ^Build -e Model -e "\[" /etc/default_config/uLinux.conf | grep -v "\[System\]" | awk '1;/\[/{exit}' |grep -v "\["

hm, I have installed also QPython - maybe that could work. But then your code has to be adjusted

Answer 25 · 2021-10-19T09:06:35.000Z

What's QPython?

Answer 26 · 2021-10-19T09:06:55.000Z

https://www.qnapclub.eu/de/qpkg/261

Answer 27 · 2021-10-19T09:07:59.000Z

Ok. That might work. Can you try the same tests and provide the path?

Answer 28 · 2021-10-19T09:08:14.000Z

https://www.forum-nas.fr/threads/qpython3-3-8-6-python3-tons-of-modules.2421/

Answer 29 · 2021-10-19T09:13:04.000Z

works

[/share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin] # /share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin/python3
Python 3.8.6 (default, Oct 12 2020, 08:31:26)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.

from urllib.request import urlopen # Python 3
urlopen("https://google.com")
<http.client.HTTPResponse object at 0x7fb1ab56b520>

Answer 30 · 2021-10-19T09:16:12.000Z

Ok.

Please provide the output for:
python3 -c "import http.server"; echo $?

"$(/sbin/getcfg Python3 Install_Path -f /etc/config/qpkg.conf)/python3/bin/python3" -c "import http.server"; echo $?

"$(/sbin/getcfg Entware Install_Path -f /etc/config/qpkg.conf)/bin/python3" -c "import http.server"; echo $?

/share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin/python3 -c "import http.server"; echo $?

Answer 31 · 2021-10-19T09:19:01.000Z

each or together? In Terminal or in python3 of Qpython3?

Answer 32 · 2021-10-19T09:19:53.000Z

each command, in the shell

Answer 33 · 2021-10-19T09:23:37.000Z

I'm not sure if this is correct

[/share/CE_CACHEDEV2_DATA/.qpkg/QPython3] # /share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin python3 -c "import http.server"; echo $?
-sh: /share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin: is a directory
126

[/share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin] # "$(/sbin/getcfg Python3 Install_Path -f /etc/config/qpkg.conf)/python3/bin/python3" -c "import http.server"; echo $?
0

[/share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin] # ~x"$(/sbin/getcfg Entware Install_Path -f /etc/config/qpkg.conf)/bin/python3" -c "import http.server"; echo $?
-sh: ~x/share/CE_CACHEDEV2_DATA/.qpkg/Entware/bin/python3: No such file or directory
127

[/share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin] # /share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin/python3 -c "import http.server"; echo $?
0

Answer 34 · 2021-10-19T09:26:46.000Z

Thanks. That helps with defining the order in which python bins are selected.

Can you post your /etc/config/qpkg.conf. I think then we have everything to implement a patch.

Answer 35 · 2021-10-19T09:29:58.000Z

there are alot of qpkg's.
Which info do you need?

Answer 36 · 2021-10-19T09:30:28.000Z

[QPython3]
Name = QPython3
Class = null
Status = complete
store = a79684706c9e3282fd9972d6737d6287
Build = 20201012
Display_Name = Python 3
Version = 3.8.6.0
Author = QoolBox
QPKG_File = QPython3.qpkg
Date = 2021-10-17
Alt_Shell = /share/CE_CACHEDEV2_DATA/.qpkg/QPython3/QPython3.sh
Volume_Select = 3
Install_Path = /share/CE_CACHEDEV2_DATA/.qpkg/QPython3
RC_Number = 109
FW_Ver_Min = 4.2.6
FW_Ver_Max = 5.0.0
Enable = TRUE

[QPython39]
Name = QPython39
Class = null
Status = complete
store = a79684706c9e3282fd9972d6737d6287
Build = 20210520
Display_Name = Python 3.9.x
Version = 3.9.5.000
Author = QoolBox
QPKG_File = QPython39.qpkg
Date = 2021-10-17
Shell = /share/CE_CACHEDEV2_DATA/.qpkg/QPython39/QPython39.sh
Volume_Select = 3
Install_Path = /share/CE_CACHEDEV2_DATA/.qpkg/QPython39
RC_Number = 109
FW_Ver_Min = 4.3.0
FW_Ver_Max = 5.0.0
Enable = TRUE

[Python3]
Name = Python3
Class = null
Status = complete
Build = 20190628
Display_Name = Python3
Version = 3.5.2.0.1
Author = QNAP Systems, Inc.
QPKG_File = Python3.qpkg
Date = 2021-10-17
Shell = /share/CE_CACHEDEV2_DATA/.qpkg/Python3/Python3.sh
Volume_Select = 3
Install_Path = /share/CE_CACHEDEV2_DATA/.qpkg/Python3
RC_Number = 101
Enable = TRUE

Answer 37 · 2021-10-19T14:19:28.000Z

I have pushed the required changes on branch use-qpython-if-available:
https://github.com/Yannik/qnap-letsencrypt/tree/use-qpython-if-available

Please test the changes

Answer 38 · 2021-10-19T15:00:12.000Z

now i get following error

[/share/Applications/qnap-letsencrypt] # sudo bash renew_certificate.sh
Checking whether to renew certificate on Tue, 19 Oct 2021 16:59:15 +0200
Renewing certificate...
qnap-letsencrypt version: 76b552c
Using python path: /share/CE_CACHEDEV2_DATA/.qpkg/QPython3/bin/python3
Stopping Qthttpd hogging port 80..
Shutting down Qthttpd services: OK.
Started python HTTP server with pid 31500
renew_certificate.sh: line 58: letsencrypt/signed.crt.tmp: No such file or directory
An error occured. Restoring system state.
renew_certificate.sh: line 11: 31500 Killed "$PYTHON" ../HTTPServer.py (wd: /share/CE_CACHEDEV2_DATA/Applications/qnap-letsencrypt/tmp-webroot)
Start apache proxy: OK
Starting Qthttpd services: OK

Answer 39 · 2021-10-19T15:22:15.000Z

Hmm, seems like an acme-tiny error but there are no acme-tiny log messages at all..

Answer 40 · 2021-10-21T07:24:41.000Z

Hmm, seems like an acme-tiny error but there are no acme-tiny log messages at all..

Any way to move ahead? Is it just me or do others have the same problems?

Answer 41 · 2021-10-21T11:20:34.000Z

With the log information you've given I can't really help you. You need to debug this on your side.

Answer 42 · 2021-10-21T12:57:53.000Z

ok, I solved it.
Somehow the init.sh didn't clone the acme-tiny into the directory.

Thanks for all the help!

Answer 43 · 2021-10-21T13:51:13.000Z

You're welcome.

Answer 44 · 2021-10-21T13:51:31.000Z

I have now merged the QPython patch into master.