ysde/grafana-backup-tool

Unable to backup Grafana 7.0.0 dashboards.

Closed this issue · 8 comments

aamkye commented

Hi, recently I've upgraded Grafana server to version 7.0, and backup tool cannot backup dashboards from it.

Here is traceback from the container:

Backing up grafana
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:986: InsecureRequestWarning: Unverified HTTPS request is being made to host 'beta-grafana.xyz.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
grafana health: https://beta-grafana.xyz.com:443//api/health
[DEBUG] status: 200
Traceback (most recent call last):
  File "./src/save_dashboards.py", line 75, in <module>
    (status, resp) = health_check()
  File "/opt/grafana-backup-tool/src/dashboardApi.py", line 29, in health_check
    return send_grafana_get(url)
  File "/opt/grafana-backup-tool/src/dashboardApi.py", line 106, in send_grafana_get
    log_response(r)
  File "/opt/grafana-backup-tool/src/commons.py", line 18, in log_response
    print("[DEBUG] body: {0}".format(resp.json()))
  File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Mikey1993 commented

Some questions:

  1. Are you sure you are using VERIFY_SSL = False in the config file?
    If not, try that first.
  2. What is your requests package version? The latest is 2.23.0, see here.
    You can check your local version by running:
Windows:
pip show requests | findstr Version

Linux:
pip show requests | grep Version
  1. Have you tried the suggested methods here ?
aamkye commented

I was running it in docker. But I'll try this 'VERIFY_SSL = False'. However I'm using valid SSL cert from let's encrypt.

Mikey1993 commented

I don't have experience using the docker version, but it should be identical to running it not in a container..

aamkye commented

I mean its container from the creator of grafana-backup-tool. And it's not working, even after setting VERIFY_SSL to False. However, its not a problem with connection but with json decode inside this tool.

Mikey1993 commented

And running the backup procedure without a container works for you?

rgarrigue commented

For the feedback, I'm fine with 7.0.1 and

~/work/grafana-backup-tool (master ✘)✹ ᐅ pip3 install -r requirements.txt
Requirement already satisfied: requests[security] in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (2.23.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (2020.4.5.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (1.25.9)
Requirement already satisfied: pyOpenSSL>=0.14; extra == "security" in /home/remy/.local/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (19.1.0)
Requirement already satisfied: cryptography>=1.3.4; extra == "security" in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (2.9.2)
Requirement already satisfied: six>=1.5.2 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from pyOpenSSL>=0.14; extra == "security"->requests[security]->-r requirements.txt (line 1)) (1.15.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from cryptography>=1.3.4; extra == "security"->requests[security]->-r requirements.txt (line 1)) (1.14.0)
Requirement already satisfied: pycparser in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=1.3.4; extra == "security"->requests[security]->-r requirements.txt (line 1)) (2.20)

I wish there wasn't the [tons of json] part, I tried to silence it w/o success.

I had a datasource + alerts channels issue KO due to permission denied, with an "admin" instead of "viewer" token its fine, could be documented.

Mikey1993 commented

For the feedback, I'm fine with 7.0.1 and

~/work/grafana-backup-tool (master ✘)✹ ᐅ pip3 install -r requirements.txt
Requirement already satisfied: requests[security] in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (2.23.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (2020.4.5.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (1.25.9)
Requirement already satisfied: pyOpenSSL>=0.14; extra == "security" in /home/remy/.local/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (19.1.0)
Requirement already satisfied: cryptography>=1.3.4; extra == "security" in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from requests[security]->-r requirements.txt (line 1)) (2.9.2)
Requirement already satisfied: six>=1.5.2 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from pyOpenSSL>=0.14; extra == "security"->requests[security]->-r requirements.txt (line 1)) (1.15.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from cryptography>=1.3.4; extra == "security"->requests[security]->-r requirements.txt (line 1)) (1.14.0)
Requirement already satisfied: pycparser in /home/remy/.pyenv/versions/3.8.3/lib/python3.8/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=1.3.4; extra == "security"->requests[security]->-r requirements.txt (line 1)) (2.20)

I wish there wasn't the [tons of json] part, I tried to silence it w/o success.

I had a datasource + alerts channels issue KO due to permission denied, with an "admin" instead of "viewer" token its fine, could be documented.

@rgarrigue ,
What do you mean by "tons of json" part?
Can you elaborate?

Also, you are right that there is a need to use the Admin role for the APIKey instead of a viewer role for the backup to succeed.

@ysde.
I have updated the README and pushed it into the documentation in my pull request.
can you merge?

ysde commented

@Mikey1993 thanks for the help
@rgarrigue Yeah, all the backups need Admin permission to proceed.

Thank you