Cannot compile: GitHub API rate limitation
Closed this issue · 8 comments
This issue has started today:
Processing debug (platform: https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip; board: lolin_d32; framework: arduino)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Latest release Arduino core URL: None
Latest release Arduino core tag: None
TypeError: Traceback (most recent call last):
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\__main__.py", line 103, in main
cli() # pylint: disable=no-value-for-parameter
^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\cli.py", line 85, in invoke
return super().invoke(ctx)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\click\decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\run\cli.py", line 145, in cli
process_env(
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\run\cli.py", line 208, in process_env
).process()
^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\run\processor.py", line 81, in process
install_project_env_dependencies(
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\package\commands\install.py", line 132, in install_project_env_dependencies
_install_project_env_platform(project_env, options),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\package\commands\install.py", line 149, in _install_project_env_platform
PlatformPackageManager().install(
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\package\manager\platform.py", line 60, in install
p.configure_project_packages(project_env, project_targets)
File "C:\Users\S\.platformio\penv\Lib\site-packages\platformio\platform\base.py", line 185, in configure_project_packages
self.configure_default_packages(options, targets or [])
File "C:\Users\S\.platformio\platforms\espressif32@src-88f34194123b4b05ff6ef6b8e68c7f60\platform.py", line 57, in configure_default_packages
URL = "https://raw.githubusercontent.com/espressif/arduino-esp32/" + tag + "/package/package_esp32_index.template.json"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~
TypeError: can only concatenate str (not "NoneType") to str
============================================================
An unexpected error occurred. Further steps:
* Verify that you have the latest version of PlatformIO using
`python -m pip install -U platformio` command
* Try to find answer in FAQ Troubleshooting section
https://docs.platformio.org/page/faq/index.html
* Report this problem to the developers
https://github.com/platformio/platformio-core/issues
============================================================
The problem seems to be a rate limitation of the GitHub API. When I enter https://api.github.com/repos/espressif/Arduino-esp32/releases/latest/
in my browser, I get this:
{"message":"API rate limit exceeded for 88.186.41.204. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
The issue doesn't occur with platform = platformio/espressif32
.
Hmm that rate limiting is potentially quite annoying as the rates are quite low when not logged in.
See: https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28
@sylque I think it is hard to tell whether it does or does not occur with the platformio/espressif32
platform as you may hit it 'suddenly' when you reach that limit.
Thus the order of trying and how quick you try is making a difference.
Not sure if PlatformIO does use some authentication token while fetching the data as you can login to PlatformIO using GH, so in theory there could be some token to overcome this rate limit.
Thanks @TD-er .
it is hard to tell whether it does or does not occur with the platformio/espressif32 platform
I know it does not occur with the platformio/espressif32
platform, because platformio/espressif32
compiles fine without an Internet access. Maybe the issue here is not the rate limitation itself, but the fact that pioarduino/platform-espressif32
accesses the Internet at each compilation. See #61.
Ah check!
If you have an other solution to get latest version, a PR is very welcome.
The query is here
platform-espressif32/platform.py
Lines 49 to 59 in 9a07688
The issue doesn't occur with
platform = platformio/espressif32
.
Well pioarduino is a fork. See #61
With the change done to make offline compile work again this problem will not occur anymore. Still it would be nice to have a "clean" solution for avoiding rate limitations
Solved. Now only a request is done when Arduino framework folder does not exists locally.