[Question] How does the token file in exporter work?

[agileshaw]

Hi team. I would like to inquire about the effect of running exporter with the token file (with -t option).

From the examples and my tests, it seems that exporter can be run with our without specifying the token. where in both cases the scraping url should always have api token as bearer header.

And I'm not able to use the array_id defined in the token file as endpoint when scraping for metrics (using the actual ip address of the array works):

token.yaml

array_id:
  address: <ip-address>|<hosname1>
  api_token: <api-token1> 

scrape command:

$ wget --header 'Authorization: Bearer xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'  'https://pure-fa-exporter.your.domain:9490/metrics/array?endpoint=array_id'
wget: server returned error: HTTP/1.1 400 Bad Request

Would it be possible for you to provide some insights on this?

[chrroberts-pure]

Hi,

array_id is a string that if matched when passed by the endpoint query parameter, it will use the parameters set in the token.yaml file. If you are using a matched string you must NOT also pass a auth token.

chrroberts@chrroberts--MacBookPro16 bin % cat token.yaml                     
foobar:
  address: array03
  api_token: <GOOD_API_TOKEN>
chrroberts@chrroberts--MacBookPro16 bin % ./pure-fa-om-exporter -t token.yaml
2024/06/19 16:42:48 Start Pure FlashArray exporter v1.0.18 on 0.0.0.0:9490



chrroberts@chrroberts--MacBookPro16 bin % curl 'http://localhost:9490/metrics/array?endpoint=foobar' | grep purefa\_info
# HELP purefa_info FlashArray system information
# TYPE purefa_info gauge
purefa_info{array_name="ARRAY03",os="Purity//FA",subscription_type="",system_id="<sys_id>",version="6.3.15"} 1

If I pass a authorization token as well as provide a match to the tokens.yaml file, the query will fail. eg.

chrroberts@chrroberts--MacBookPro16 bin % curl 'http://localhost:9490/metrics/array?endpoint=foobar' --header 'Authorization: Bearer <GOOD_API_TOKEN>' 
Error connecting to FlashArray. Check your management endpoint and/or api token are correct.

HOWEVER - if I provide an endpoint that does NOT match with an array_id match, the Bearer Auth token works as documented (note it matches on the array_id string, only NOT address, so i can use array03 here)

chrroberts@chrroberts--MacBookPro16 bin % curl 'http://localhost:9490/metrics/array?endpoint=array03' --header 'Authorization: Bearer <GOOD_API_TOKEN>'
# HELP purefa_info FlashArray system information
# TYPE purefa_info gauge
purefa_info{array_name="ARRAY03",os="Purity//FA",subscription_type="",system_id="<sys_id>",version="6.3.15"} 1

Should we have an error like Bearer token provided by request when endpoint is defined in tokens file as an array_id maybe, but maybe just putting in the documents that you must not provide the token may also be enough.

[agileshaw]

Thanks a lot for the detailed response. It would definitely be helpful to explain this more clearly in the documents.

However, in my test setup, the scrape on array_id without auth token still doesn't work. I wonder whether this has something to do with the Purity/FA version. The one in my cluster is 6.1.22, but in your example it is 6.3.15.

[chrroberts-pure]

Hi. It may be helpful to setup a Zoom meeting to look over your configurations.

Please email us at pure-observability@purestorage.com to set up a time to meet. Thanks!

[chrroberts-pure]

Hi, I just noted that in your example you're using HTTPS -- by default the exporter only exposes HTTP, unless ssl certs are provided at run time.

[jddarby]

Hi, my team and I are also finding this behaviour a bit confusing. What is the reasoning for returning an error for this scenario? Could you have the exporter ignore the auth header if the token has already been provided for the endpoint instead?

For us this would be better, as the process that scrapes the exporter endpoint (telegraf) scrapes other endpoints too, some of which require a token in the auth header.

[chrroberts-pure]

@jddarby that is pretty valuable input as well. Thank you.

However, I feel like the opposite may be more logical. If I receive a newer api token via the GET call, use that vs. the tokens file. this provides a little more flexibility on the fly, as potentially the user of the exporter may not have access to edit tokens.yaml.

I still would consider sending a WARN on that case to stdout, something like “received a Bearer token for endpoint defined in tokens file, continuing with supplied Bearer token”

would this be an okay behavior for your use case?

If you’re inclined, we do accept PRs! I personally am on PTO this week, but happy to give this a review if anyone submits a PR on this issue.

[jddarby]

Thanks for the reply @chrroberts-pure. I don't think the warning alone will help us with our issue. I'll give more detail:

• We have a telegraf container (the client) which is configured to scrape many prometheus endpoints.
• Some of these endpoints require an auth header to authenticate against the kubernetes API, the token in the header comes from a service account.
• We have configured the tokens.yaml for the pure exporter as this is where we would like the exporter to get the API token from.
• Because telegraf sends the auth header with the service account token for all requests it sends out, the pure exporter is trying to use this over the configured endpoint in the tokens.yaml which is causing the 400 bad request.

Our current solution would be to add something in the middle to remove the auth header but it would be preferable for us to fix the issue here.

To reduce the confusion, I think it would be good if we had a choice of URL params. If one is provided we will expect the auth to come from the header and if the other is provided we will look for the auth in token.yaml. I think this allows the flexibility you desire whilst allowing the user to clearly understand that they are either using the endpoint configured in tokens.yaml or they are telling the exporter which address to query when they are making the request.

I have raised #147 to add this parameter. Let me know what you think and I'll tidy things up if you think this is going in the right direction. My alternative idea would be that we try to authenticate with the header token and fall back to the token in tokens.yaml if that fails but I prefer the option above.

[jddarby]

Hope you had a good holiday @chrroberts-pure. Have you had a chance to look at the PR above yet?

[chrroberts-pure]

I'll leave my comments in the PR!

[github-actions]

Stale issue message