- What is taking most of the time in your builds? Cloning, building, testing?
- Improve cloning times by
- shallow clones (only if a fresh clone is needed every time)
- reduced repository sizes by moving large binary files to Git LFS using bfg-repo cleaner
- push based build triggering, + use ngrok or open a port if needed
- get version control system server and artifact repository closer to the build server (maybe by hosting it yourself or moving the build server closer to the rest)
- Improve build times by
- only building components once and transport them to all downstream build jobs and machines
- using an artifact repository for shared components
- SSDs, more cores and more memory
- Optimized compiler switches - mvn client VM, parallel build options, memory options
- use a build system that is caching build results globally and does build parallelization automatically
- Improve test times by
- identifying which tests are still testing used functionality and disable the rest
- rewriting integration and system tests to use more mocking (one process without IPC and file system interaction preferred)
- analyze bottlenecks and silent exceptions using New Relic, Dynatrace, OverOps or just VisualVM
- Break monolythic applications done into multiple components that can be tested in isolation (and only retested if they changed)
- dedicate a person per team to improve test execution time
- test partitioning across multiple machines using Jenkins Parallel Test Executor Plugin (this repository)
- SSDs, more cores and more memory
- learn more about Jenkins Pipelines, its job resuming functionalities and its GitHub Integration
- if you like to use a pre-setup environment using Docker that integrates with GitHub.com, you can follow this tutorial.
- pre-heating of integration test systems (e.g. by unfreezing memory snapshots of already fully up and running application server and database VMs)
- only do smoke tests in build, all tests in nightly build (CI result is now also shown in code browser out of the pull request)
- optimized compiler switches, parallel test runs (Maven 3 now supports this better), compile options like -XX:+TieredCompilation -XX:TieredStopAtLevel=1 for short running mvn invocations
- use different commit status contexts for different tests, only make smoke test status checks mandatory
- find the best tradeoff of number of tests/test duration and code coverage
- use a build system that is automatically caching test results parallelizing tests (if code tested and test have not changed)
nuuyoo/speedup-builds
Some ideas on how to speed up builds / tests with Jenkins partitioning example
Shell