funkybob/serverless-s3-deploy

Feature Request: add parallel/sequence control

jrencz opened this issue · 6 comments

Right now the plugin is processing its configuration and it adds each file to the queue handled by serverless AWS provider.
This gives no control over the order of uploaded files

My use case:
I have a project that consists of source code of a single page app. The code is bundled into several css js and font files and one html file.
It is important for the deployment to be consistent to have a way to upload everything but the html file first and only if it succeeded then upload html file.

I see no way to do that just by using configuration of the plugin

I analysed the code and I see that the promise returned by the awsProvider@request is discarded:

this.provider.request('S3', 'putObject', details, this.options.stage, this.options.region);

File uploads could possibly be sequentialised on 2 levels:

  • assets
  • files in each asset

with current syntax. Right now everything runs in parallel (or: as parallel as awsProvider allows it to be)

I'd like to open a discussion about:

  1. whether the plugin should allow sequences

  2. should it

    • process assets and files in sequence
    • process assets in parallel but files should be uploaded sequentially
    • process assets sequentially but files should be uploaded in parallel
    • introduce new syntax to express sequences and keep current syntax for maximum parallelisation

Interesting proposal.

For your specific case I can see it easily being addressed by processing each Target in series, but files in parallel. Then have two Targets for the same source files, but one that excludes HTML, and one that includes them.

However, the idea of explicit levels of parallelism, as well as "stop on failure", is more appealing to me.

@funkybob I did a proof of concept which I'll push soon.

@funkybob take a look at #20 plz

I'm not willing for it to be merged as is. It's just something that covers my case (with mode: sequence put on the level of bucket)

Looks promising... I'll try to make time to look at it further tomorrow.

Thanks!

@funkybob have you had a chance to spend some time thinking about this idea yet?

Sorry, life got in the way :)

I think your approach looks close to what I'd go with... I'll take another look this weekend and hopefully merge and release soon.