cloudflare/python-cloudflare

Unable to connect to logpush/edge enpoint via cli4

antigenius0910 opened this issue ยท 5 comments

antigenius0910 commented

Trying to check and configure logpush instant jobs (https://developers.cloudflare.com/logs/instant-logs) and was hoping to use cli4 to do it quickly, but it doesn't know the logpush/edge endpoint.

It just returns the error /zones/:zoneid/logpush/edge - not found.
Confirmed that it does work as expected via curl.

Tested using version 2.10.2

~ แ… cli4 -v zones/:$CLOUDFLARE_ZONE_ID/logpush/edge/jobs
cli4: /zones/:$CLOUDFLARE_ZONE_ID/logpush/edge - not found
mahtin commented

The Python Cloudflare libraries are build from the contents of the https://api.cloudflare.com website. Sadly, the /zones/:zoneid/logpush/edge call has not been documented on that page. It's only on https://developers.cloudflare.com/logs/instant-logs within the developers area. (Yes, there's more than one place you need to read if you want to work with Cloudflare API's). I'll do two things.

  1. I'll notify Cloudflare.
  2. I'll code it up from the developers page and report back here if I can push it cleanly to GitHub & PyPI.

More later

  • ๐Ÿ‘1
mahtin commented

44 minutes later. Coded (it needed some slight improvement to the JSON/Yaml processing) ...

$ cli4 --post \
  ='{   
    "fields": "ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID",
    "sample": 1,
    "filter": "",
    "kind": "instant-logs"
  }' \  
  /zones/:โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ.com/logpush/edge/jobs | jq .
cli4: /zones/:โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ.com/logpush/edge/jobs - 1004 creating a new instant logs job is not allowed: exceeded max jobs allowed
$

... let me try again with a business/enterprise account. While the 1004 error doesn't say it, it means "can't use on Free or Pro accounts".

More later

antigenius0910 commented

thanks so much for the help @mahtin !!
and yes. it needs a enterprise account. reference here https://blog.cloudflare.com/instant-logs-on-the-command-line/

I also create a simple python script here https://gist.github.com/antigenius0910/2528b68510a522c4f7ffd22d32e06c49 as well but thinking on cli4 it could beneficial for more people

mahtin commented

OK - it's working. I will push some code later tonight as I have a few other items to add.

We start with an empty slate:

$ cli4 /zones/:โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ.com/logpush/edge/jobs | jq .
[]
$

Then we add monitoring:

$ cli4 --post \
	='{
		"fields": "ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID",
		"sample": 1,
		"filter": "",
		"kind": "instant-logs"
	}' \
	/zones/:โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ.com/logpush/edge/jobs | jq .
{
  "destination_conf": "wss://logs.cloudflare.com/instant-logs/ws/sessions/โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ",
  "fields": "ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID",
  "filter": "",
  "kind": "instant-logs",
  "sample": 1,
  "session_id": "โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ"
}
$

Then we can see the results:

$ cli4 /zones/:โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ.com/logpush/edge/jobs | jq .
[
  {
    "fields": "ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID",
    "filter": "",
    "kind": "instant-logs",
    "sample": 1,
    "session_id": "โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ"
  }
]
$

Yippee! Sadly, this call does not match the normal param processing for other Cloudflare calls. Hence the need to add code.

I'll push later today.

mahtin commented

Release 2.10.3 is now released (to Github and PyPl). Enjoy!

  • โค๏ธ1
  • ๐Ÿš€1