Testing with source code from other repositories

Question

sosukesuzuki opened this issue 4 years ago · 9 comments

We run the latest Prettier on the other repositories' source code to check for regressions before we release.

like below:

This is hard to do manually, so I want to automate it.

Answer 1 · 2020-09-29T05:01:58.000Z

Use a separate repo,

add prettier/prettier and other repositories as submodule.
add GithubActions, when open an issue with specific title(or other events), CI send a PR with formatted code.

Answer 2 · 2020-09-29T06:30:13.000Z

What do you think about adding to this repository as a integration test and checking with a CI for each PR?

Answer 3 · 2020-09-29T07:11:09.000Z

Will this test fail, if format changed? Or are you thinking about snapshot the result?

Answer 4 · 2020-09-29T07:21:16.000Z

Sorry, I was mistaken. I'll try with your suggestion.

Answer 5 · 2020-10-01T03:56:08.000Z

I created a repository https://github.com/sosukesuzuki/prettier-regression-testing.
I don't know the way to see diffs of submodules on PR. So, I implemented an action using issue comments for checking diffs. See example issue: sosukesuzuki/prettier-regression-testing#4.
This has only typescript-eslint as a submodule because work in progress.

Answer 6 · 2020-10-01T06:23:07.000Z

Maybe no submodule, git clone -> git commit -> run format -> git commit?

Answer 7 · 2020-10-01T13:02:15.000Z

Hmm, we want to see just diffs. So I think submodule and issue comments is enough. They are also relatively easy to manage.

Answer 8 · 2020-10-01T16:14:30.000Z

I don't know how to see submodule diff either.

git clone -> git commit -> run format -> git commit

We can check the second commit to see diff.

Answer 9 · 2020-10-01T23:34:32.000Z

I don't know how to see submodule diff either.

I'll use it to check regressions in the next release. If we are unhappy with it, you can improve it afterwards.
I close this issue for now.