DotnetPerfMonitor is an automated system for monitoring the performance of various components in the .NET ecosystem, including NuGet, F# compiler, C# compiler, and MSBuild. It builds upon the foundation of the NuGet Performance Monitor and expands its functionality to encompass a broader range of tools and technologies.
In our company, we work extensively with .NET solutions consisting of hundreds of projects. Developer productivity is a key focus for us, and we are deeply invested in enhancing the performance of the .NET ecosystem.
One crucial aspect of developer productivity is the time spent waiting for NuGet to restore packages. We also want to take advantage of the runtime improvements that accompany each new release of .NET. However, transitioning to a new runtime version relies on being able to use the corresponding .NET SDK and NuGet version.
In the past, we have encountered situations where certain changes in NuGet resulted in poor performance for large solutions. These performance issues prevented us from adopting a new .NET release until the bug was fixed. To avoid such unpleasant surprises in the future, we developed the NuGet Performance Monitor.
Now, we aim to expand its capabilities to cover other aspects of the .NET ecosystem, such as the F# compiler, C# compiler, and MSBuild.
DotNetPerfMonitor is this extended monitoring system. By encompassing additional components, we strive to gain insights into the performance characteristics of the entire .NET ecosystem. This will enable us to identify and address any performance regressions promptly, ensuring a smooth transition to new releases and an overall improved development experience.
- Scripts from the NuGet.Client repository with custom test cases.are are used for benchmarks,
- GitHub Actions and GitHub-hosted runners are used to run benchmarks on a daily schedule
- Python script is used to process results and append it to the data.csv file that makes it easy to plot charts and can be used further data analysis.
- There is also another python script that is used to generate alerts. In case of performance regression is found a new Issue is opened.
- Plotly.js is used to generate charts with results.
It is worth noting that DotNet Performance Monitor uses GitHub-hosted runners to run the benchmark, therefore it cannot just assume what is the performance of the particular runner's VM or how it is going to change over time. Therefore, each test job runs actually two tests, one test for the baseline version and one test for the current version. By measuring the relative performance of the current version, the DotNet Performance Monitor is independent of the infrastructure that is running it.
DotNetPerfMonitor is licensed under the Apache License, Version 2.0. See LICENSE for the full license details.
Contributions are welcome! Whenever you have any suggestions or ideas to be implemented, we encourage you to: