Clean up the purging logic for crawlee projects
B4nan opened this issue · 0 comments
Crawlee projects (both JS and python) are purging default storages automatically (at the framework level), if we detect those, apify run
shouldn't do anything and let crawlee do the purge. apify run -p
should behave the same for them, still no manual purging at the CLI level. apify run --no-purge
(with --resurrect
alias) should disable the purging via CRAWLEE_PURGE_ON_START=0
env var. This also applies for JS SDK v3 projects, since there is crawlee behind the scenes doing the same. Similarly, the upcoming python SDK v2 needs to behave the same, since again, there will be crawlee behind the scenes.
Projects that are not using crawlee or JS SDK v3 or python SDK v2 still need to keep the purging on CLI level. apify run -p
will handle purging in the CLI. apify run
without any flags shouldn't do anything either, like apify run --no-purge
.
Also remove the --purge-xxx
flags, and mention all of this in the migration guide https://github.com/apify/apify-cli/blob/master/MIGRATIONS.md