apify/apify-cli

Clean up the purging logic for crawlee projects

B4nan opened this issue · 0 comments

Crawlee projects (both JS and python) are purging default storages automatically (at the framework level), if we detect those, apify run shouldn't do anything and let crawlee do the purge. apify run -p should behave the same for them, still no manual purging at the CLI level. apify run --no-purge (with --resurrect alias) should disable the purging via CRAWLEE_PURGE_ON_START=0 env var. This also applies for JS SDK v3 projects, since there is crawlee behind the scenes doing the same. Similarly, the upcoming python SDK v2 needs to behave the same, since again, there will be crawlee behind the scenes.

Projects that are not using crawlee or JS SDK v3 or python SDK v2 still need to keep the purging on CLI level. apify run -p will handle purging in the CLI. apify run without any flags shouldn't do anything either, like apify run --no-purge.

Also remove the --purge-xxx flags, and mention all of this in the migration guide https://github.com/apify/apify-cli/blob/master/MIGRATIONS.md