Using Matrix Strategy results in fast-forward errors

[GerryWilko]

I'm using this action to pull in data from our content repos and commit into the main repo for building. Seems like a great little action for this.

I have a small issue when using matrix strategy with this action. Essentially the problem is that depending on the order of the execution and timing of the execution of the matrix jobs some will fail with a git fast-forward error. One of the other jobs has pushed to the repo in between. This isn't directly related to matrix strategy and any clashing workflows that perform commits to their repos will be affected here.

There are a few possible fixes non of which are perfect so I'm not sure this is necessarily a solvable problem:

• Do a git pull before git push (this will reduce the chance of this error occurring but not solve it)
• Handle the git push fast-forward error and repeat operation up to say 5 times (e.g. pull then push, pull then push, ...)
• Some other magical fix which essentially does a pull push in one operation which I'm not aware exists 😄

I'm more than happy to raise a PR to sort this but just wanted to see which one was preferable from the maintainers!

[idan]

@GerryWilko yeah, I see what you're saying. This is indeed a problem; actions which create commits cannot run concurrently, or they must handle the situation you highlight.

I'm not sure what the correct solution is here. We've employed "retry upon push failure" in similar scenarios, but that results in a "latest wins" which is nondeterministic. If you're postprocessing to append data to a given file, retries are a good solution.

Otherwise, in a matrix strategy, maybe the right approach is having each flatfile fetched include an identifier per matrix. So, if you are fetching data.json with flat, a matrix strategy would result in MATRIXA_data.json, MATRIXB_data.json, etc. This would solve the issue of matrix jobs competing for the same files, but it still doesn't solve the no-fast-forward problem.

I think the ideal solution is a combination of the matrix-guards and a retry loop of pull --rebase and push until it succeeds. We still need the loop because there's no guarantee that another matrix job hasn't pushed in between the pull and the push. I can't think of something better though!

WDYT?