Optimize CodeQL analysis step in CI builds
Opened this issue ยท 8 comments
Optimize the CodeQL processing in the CI checks for Armada - currently, the CodeQL analysis for Go code can take up to 5 minutes.
I would like to work on this issue can you guide me on how to get started with this to solve the issue
Hi @Bharadwajshivam28 . Yes, you can take over. You need to take a look at the CodeQL workflow, in the file codeql-analysis.yml
. Pay attention to the matrix used analyze
job. You will see that codeql is run for csharp, go, and javascript. Since that the matrix is used to run 3 parallel jobs, you need to pay attention to the Autobuild
step that needs to run conditionally for csharp and javascript, and have another step running conditionally for go. This step for go needs to execute the go build commands seen in other workflows
Hi @Bharadwajshivam28 . Yes, you can take over. You need to take a look at the CodeQL workflow, in the file
codeql-analysis.yml
. Pay attention to the matrix usedanalyze
job. You will see that codeql is run for csharp, go, and javascript. Since that the matrix is used to run 3 parallel jobs, you need to pay attention to theAutobuild
step that needs to run conditionally for csharp and javascript, and have another step running conditionally for go. This step for go needs to execute the go build commands seen in other workflows
I went through the file and understood the workflows. our end goal is to optimize the analysis for Go code which currently takes upto 5 minutes. Can you share some more details how can i tackle it? @pavlovic-ivan
The idea is to replace the autobuild step, with another that will execute the build natively as it is done in other workflows, and leave other steps in the analyse job as is. Please take a look at how conditional expressions are added to steps, so that autobuild
step is executed for csharp
and javascript
, and how to add another step called (eg) Go build
, that is run only when go
comes from the matrix. That would be the first step. We can discuss further improvements after this. How does that sound?
The idea is to replace the autobuild step, with another that will execute the build natively as it is done in other workflows, and leave other steps in the analyse job as is. Please take a look at how conditional expressions are added to steps, so that
autobuild
step is executed forcsharp
andjavascript
, and how to add another step called (eg)Go build
, that is run only whengo
comes from the matrix. That would be the first step. We can discuss further improvements after this. How does that sound?
I have implemented condition expressions before but I am confused about what type of conditions we need to apply in the workflows....
And I understood that we need a step which will run only when go comes out of the Matrix.
Ya we can improve it... Let me work and give the updates..
Hey @pavlovic-ivan we need to add some condition that when matrix language is csharp or javascript and for go we will have a seperate step which ensures that it will run only when the matrix language is go.
also we can run the Mage Proto step only when the matrix language is golang..
Is it what we need right? This will overall improve the performance i guess... and we can make more changes if needed.
correct me please if i missed something
Hi @Bharadwajshivam28 let me take over, as i was already assigned to the issue, and afterwards we can sync on the work i would do, and that will give you some insights for the future. How does that sound?
Hi @Bharadwajshivam28 let me take over, as i was already assigned to the issue, and afterwards we can sync on the work i would do, and that will give you some insights for the future. How does that sound?
okay works for me ....