mrrfv/cloudflare-gateway-pihole-scripts

HTTP error! Status: 504

JuenTingShie opened this issue ยท 16 comments

> cloudflare-delete:list
> node cf_list_delete.js

file:///home/runner/work/cloudflare-gateway-adblock/cloudflare-gateway-adblock/lib/helpers.js:40
    throw new Error(`HTTP error! Status: ${response.status}`);
          ^

Error: HTTP error! Status: 504
    at request (file:///home/runner/work/cloudflare-gateway-adblock/cloudflare-gateway-adblock/lib/helpers.js:40:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///home/runner/work/cloudflare-gateway-adblock/cloudflare-gateway-adblock/cf_list_delete.js:9:29

Hi, my github action run failed and show up this HTTP error message, it works great before.

Is there any fixable problem? What info should I provide for maintainers?

Addition info is that I found out my Zero Trust team/lists webpage takes like forever to loading, is that the reason that cause 504 status code?

OneZK commented

Cloudflare's core data center is offline due to commercial power outage and the generators failed. Based on the latest update from cloudflarestatus.com , it says that all services has been restored but noticed in their previous update that the List (updating of list items) was temporarily suspended. It is not clear if this is the reason for error 504 since all other service such as Cloudflare API, ZT Gateway are already restored.

OneZK commented

Just to add, I have another script that will read the IP address from NO-IP and push that IP into the DNS Location in my Cloudflare Gateway account and it is now working properly. I suspect that the suspension of List (updating of list item) that was mentioned in Cloudflare's advisory is what causing the CGPS script to fail with error 504.
Their advisory is confusing. First, they suspend the List (updating of list item) and then they provided a new update saying that "All services" has been restored and that they are now monitoring the stability. I think they forgot to restore the List? ๐Ÿค”

mrrfv commented

As @OneZK said, Cloudflare is experiencing issues on their side and this is not a problem with this project. See the Cloudflare Status for details. I'll keep this open just in case other people come across this.

Thanks for contributors' reply, I am fully understand currently status.

I'll keep re-execute my github action to see if all go back to normal, if so, then close this issue.

๐Ÿ‘

According to the cloudflare status page here: https://www.cloudflarestatus.com/incidents/hm7491k53ppg

Material changes since last update: Lists have now have been moved to restored status. Lists can only be modified through the Dashboard currently.
Posted 13 hours ago. Nov 04, 2023 - 18:32 UTC

So, the lists should be accessible from the dashboard. But even after 13 hours now, it's still taking forever to load the lists page on the dashboard.
And on top of that, the lists loads once or twice after waiting for like several minutes and it shows that there are no lists.
I had a few other lists apart from this project, and they are gone now. Hope it gets recovered from some backup that cloudflare has in a few days.
I have raised a support ticket to cloudflare.

OneZK commented

According to the cloudflare status page here: https://www.cloudflarestatus.com/incidents/hm7491k53ppg

Material changes since last update: Lists have now have been moved to restored status. Lists can only be modified through the Dashboard currently.
Posted 13 hours ago. Nov 04, 2023 - 18:32 UTC

So, the lists should be accessible from the dashboard. But even after 13 hours now, it's still taking forever to load the lists page on the dashboard. And on top of that, the lists loads once or twice after waiting for like several minutes and it shows that there are no lists. I had a few other lists apart from this project, and they are gone now. Hope it gets recovered from some backup that cloudflare has in a few days. I have raised a support ticket to cloudflare.

After investigating this through github and cloudflare side, I can confirm that cloudflare gateway was able to receive the domains pushed by this script based on the logs and it seems that all chunks are successfully pushed into CF gateway even though the cgps action logs shows an error in all of the chunks. It seems like the CF Gateway is unable to create a Rule/Policy. Now another problem is that those chunks of domain that was pushed into cloudflare gateway wont appear when we manually create a policy with the following selector
-> Domain
-> In list
-> No Valid Options

It shows No Valid Options so it looks like those chunks becomes invisible in dashboard and through API. CGPS script cannot find those chunks when it runs the delete request to the CF API.

Last status from Cloudflare concerning lists

image

After a failed run, I rechecked the cloudflare status:

Starting 11/06/2023 17:00 UTC, Cloudflare is initiating a restoration of Control Plane API and Product Services to primary HA datacenters in Portland. The restoration operation should last for about ~2 hours. During this window, Cloudflare API and Dashboard availability may degrade.
Posted Nov 06, 2023 - 17:00 UTC

Just testing now. Looks like everything is working again.

mrrfv commented

I'm still getting some unexplainable authentication issues.

My github action works great now. Thanks for tracing this issue.

https://www.cloudflarestatus.com/

image

OneZK commented

I think there's still a problem with Cloudflare API_TOKEN when FAST_MODE is Enabled. I forked a CGPS repo from someone who is using API_TOKEN instead of API_KEY. I got HTTP Error 504, 500 and 409 with FAST_MODE enabled but when FAST_MODE is disabled, everything works fine. On the other hand, the CGPS repo that I forked directly from @mrrfv which us using API_KEY works great on both FAST_MODE enabled and disabled.
I'll investigate this issue when I have time.

I think there's still a problem with Cloudflare API_TOKEN when FAST_MODE is Enabled.

Is there any update for this situation? Just curious

OneZK commented

I think there's still a problem with Cloudflare API_TOKEN when FAST_MODE is Enabled.

Is there any update for this situation? Just curious

It works fine now. I ran the script with FAST_MODE enabled and it succeded without error and all chunks are created into gateway policy successfully and the old list in gateway has been deleted without error.

Okay, then I will close this issue. Thanks for everyone's help. :)