[Question] 403 Forbidden when accessing Overview of all environments in 2.2.0

[andyroerig]

Seeing odd behavior (403 Forbidden) when trying to navigate to the overview page for all environments on our new 2.2.0 build of puppetboard.

puppetboard.example.com/%2A/ (overview of all environments) results in a 403 Forbidden page
puppetboard.example.com/%2A/nodes (nodes in all environments) works as expected

puppetboard.example.com/example_env/ (overview of example_env environment) works as expected
puppetboard.example.com/example_env/nodes (nodes in example_env environment) works as expected

Puppetboard logs the following from the request that results in 403 error:

INFO:puppetboard.utils:PuppetDB Version 7.1.0
ERROR:pypuppetdb.api:Forbidden request: /metrics/v2/read/puppetlabs.puppetdb.population%3Aname%3Dnum-nodes (method :get). Please see the server logs for details. pdb-svc:8080 over HTTP.10.2.3.4 - - [18/Mar/2021:20:21:26 +0000] "GET /%2A/ HTTP/1.1" 403 234 "https://puppetboard.example.com/%2A/nodes" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36"ERROR:puppetboard.utils:403 Client Error: Forbidden for url: http://pdb-svc:8080/metrics/v2/read/puppetlabs.puppetdb.population%3Aname%3Dnum-nodes

PuppetDB logs the following from the request that results in 403 error:

10.1.2.3 - - [18/Mar/2021:20:21:26 +0000] "GET /pdb/query/v4/environments HTTP/1.1" 200 51 "-" "python-urllib3/1.26.4"
10.1.2.3 - - [18/Mar/2021:20:21:26 +0000] "GET /pdb/meta/v1/version HTTP/1.1" 200 25 "-" "python-urllib3/1.26.4"
2021-03-18 20:21:26,735 ERROR [p.t.a.rules] Forbidden request: 10.1.2.3 access to /metrics/v2/read/puppetlabs.puppetdb.population%3Aname%3Dnum-nodes (method :get) (authenticated: false) denied by rule 'puppetlabs puppetdb metrics'.
10.1.2.3 - - [18/Mar/2021:20:21:26 +0000] "GET /metrics/v2/read/puppetlabs.puppetdb.population%3Aname%3Dnum-nodes HTTP/1.1" 403 140 "-" "python-urllib3/1.26.4"

Puppet Server version: 7.0.3
PuppetDB version: 7.1.0
puppetboard version: 2.2.0
Postgres version: 11.11

bash-5.0# pip3 list
Package      Version
------------ ---------
certifi      2020.12.5
chardet      4.0.0
click        7.1.2
commonmark   0.9.1
Flask        1.1.2
Flask-WTF    0.14.3
gunicorn     20.0.4
idna         2.10
itsdangerous 1.1.0
Jinja2       2.11.3
MarkupSafe   1.1.1
pip          21.0.1
puppetboard  2.2.0
pypuppetdb   2.3.0
requests     2.25.1
setuptools   53.0.0
urllib3      1.26.4
Werkzeug     1.0.1
wheel        0.36.2
WTForms      2.3.3

[gerardkok]

I ran into the same problem a few days ago, and it appears that the /metrics/v2 PuppetDB endpoint does not allow for unauthenticated access. Maybe this was introduced in PuppetDB 7.1.0, at least there's mention of it in the release notes. This is configured by the 'puppetlabs puppetdb metrics' rule in the /etc/puppetlabs/puppetdb/conf.d/auth.conf file.

Because my PuppetDB is not externally accessible anyway, I fixed it by allowing unauthenticated access to the /metrics/v2 endpoint, by changing the rule to this:

        {
            "allow-unauthenticated": true,
            "match-request": {
                "method": [
                    "get",
                    "post"
                ],
                "path": "/metrics",
                "query-params": {},
                "type": "path"
            },
            "name": "puppetlabs puppetdb metrics",
            "sort-order": 500
        }

and now my overview works again.

[andyroerig]

Thanks @gerardkok for your help!

Allowing unauthenticated access in the 'puppetlabs puppetdb metrics' rule resolved our issue with the overview page too!

[JvGinkel]

Is this the proper way to resolve this issue? Isn't there a way to authenticate instead of allow-unauthenticated?