Submodule initialization

[leoheck]

Hi @set-soft I am facing one initial issue before being able to make the ERC.

My project uses the library as a git submodule. It is a good approach when reusing the same library within different projects.

My issue is to initialize this submodule.

I have to use the git command which seems to be missing in your default docker.
Is it possible to install the git with a command inside my git workflow test.yaml? If yes, could you please give me one example?

Or another solution if to add git in your default docker image, if you think it is promising to have it for the future.

Here are some commands that I am trying to run. They all need git
actions/checkout#116 (comment)

This is an even clean approach by reusing other actions, but they all require git.
https://github.com/marketplace?type=actions&query=submodule

Thank you.

[leoheck]

It looks like I had 3 issues :)

• Missing packages <=== This was solved by replicating and improving your docker image
• Submodule <========== I could not fix it yes
• And private repo <===== I don't know if it is fixed

These last 2 issues are so hard to overcome. Nothing seems to work here.

I am workaround them by downloading the library.zip with wget from my dropbox with one GitHub action. This is just to see if I can reach the end of this integration with my repo.

I am lucky you shared the image to create your docker, so I was able to understand and overcome them.

This is a makefile fork from your kicad_auto repo docker. It is a good starting point for automating this. It improves your initial build.sh adding more info to the new users like me.

Right now, I am having issues with the ERC. It does not find the library. I put this in another issue,

To generate a new image:

make donwload
make build_image
make upload_image user=setsoft tagname=XYZ

[set-soft]

Ok, let me see:

1. For an image containing the tools and also git try this one:
   https://hub.docker.com/repository/docker/setsoft/kicad_auto_test
   This is the image I use to run the regression tests. So this image will always have git.

2. If you need to do anything that isn't strictly related to the KiCad automation you can put it in another pipeline step or even in another job. Each job can be run in a different docker image and a lot of "GitHub Actions" (used in another step) are "self contained".
   In fact the example does a git checkout using:

• uses: actions/checkout@v2
  And we don't need git in the docker image. v1 supported submodules and people asked to add it to v2.
  Also: you can do things in a job and pass files to another job using artifacts. So you can run the commands in another docker image, I think the default image has git.

3. You should be able to checkout a private repo using "secrets" (environment variables that contains sensible data that isn't exposed)

4. If you want to add files to "kicad_auto" I'll suggest using it as base, instead of using:
   FROM setsoft/kicad_debian:latest
   use:
   FROM setsoft/kicad_auto:latest
   And only install the needed tools. Take this as example:
   https://github.com/INTI-CMNB/kicad_auto_test
   This will reduce the time needed to generate it, save disk space, etc.

5. Another option is to try GitLab. I found it more flexible.

[leoheck]

Ok. Thanks for your comments.

2.... uses: actions/checkout@v2

It looks like it has a bug, I don't know. I tried this v2 (also v1) and the master version too. Nothing of these actions worked for me yet. But I can be doing something wrong too.

4. If you want to add files to "kicad_auto" I'll suggest using it as base, instead of using: FROM setsoft/kicad_auto:latest

Oh, good, right! I'm going to use it for the next steps

5. Another option is to try GitLab. I found it more flexible.

Indeed, but it is not an option for me. Since I can' change this, unfortunately.

[leoheck]

So, if I am using FROM setsoft/kicad_auto:latest in the Dockerfile, I can remove the steps to install make and kiplot tools since they are already there in your docker imager, is it right?

Actually, my initial idea was to put my project working at the first moment and then, later, change it to use your Docker image. With that, I can report issues or help improve your Docker image, while others can use and do the same.

[set-soft]

Yes you just focuse on what's needed to add.
Remmember you can use "kicad_auto_test" which I use for testing purposes.

[leoheck]

Good. I am using this idea now. What is the reason to use 2 one for tests and one for the real CI? Is it because you want the smallest image as possible to make the CI tasks finish faster?

[set-soft]

Yes, is to make it smaller (kicad_auto is 690MB and kicad_auto_test is 862MB and growing).
But no, it isn't server time. Is just optimization for all users, I hate when people doesn't care about resources, sometimes forcing to thousands of users to waste hundreds or MB just to avoid investing a few minutes in the design. The global impact is hundreds of TB wasted.
If a tool is needed by most of the users is good candidate for "kicad_auto", is a tool is just needed to verify that tools are working as expected goes to "kicad_auto_test". The "kicad_auto" image is a whole Debian 10.4 system, so you can install tools "on-the-fly".
I need to study better how docker layers can be merged and stacked. I think we can have small layers that can be combined with "kicad_auto" to add tools.

[leoheck]

I hate when people doesn't care about

Got it. I agree, thank you.

[set-soft]

I added an example showing how to deal with a PCB inside a sub folder and using a submodule to get a library.
I'm using an "artifact" to pass the submodule to the KiCad jobs. In this way we don't need git in kicad_auto.
I think this solves this question.
I'm not using private repos, but this should be possible using SSH keys and I'm not sure is needed for the example.
I'm clolsing it, feel free to add any remaining question as another issue.