With the release of .NET 6 AWS Lambda now supports .NET Core 3.1 and .NET 6 as managed runtimes. With the availability of ARM64 using Graviton2 there have been vast improvements to using .NET with Lambda.
But how does that translate to actual application performance? And how does .NET compare to other available runtimes. This repository contains a simple serverless application across a range of .NET implementations and the corresponding benchmarking results.
The application consists of an Amazon API Gateway backed by four Lambda functions and an Amazon DynamoDB table for storage.
It includes the below implementations as well as benchmarking results for both x86 and ARM64:
- .NET Core 3.1
- .NET Core 3.1 with Open Telemetry tracing
- .NET 6 Lambda
- .NET 6 Top Level statements
- .NET 6 Minimal API
- .NET 6 NativeAOT compilation
- .NET 7 Custom Runtime
- .NET 7 NativeAOT compilation
- .NET 7 Minimal API with NativeAOT compilation
- .NET 8
- .NET 8 Minimal API
There are four implementations included in the repository, covering a variety of Lambda runtimes and features. All the implementations use 1024MB of memory with Graviton2 (ARM64) as default. Tests are also executed against x86_64 architectures for comparison.
There is a separate project for each of the four Lambda functions, as well as a shared library that contains the data access implementations. It uses the hexagonal architecture pattern to decouple the entry points, from the main domain and storage logic.
This implementation is the simplest route to upgrade a .NET Core 3.1 function to use .NET 6 as it only requires upgrading the function runtime, project target framework and any dependencies as per the final section of this link.
This implementation uses the new features detailed in this link including
- Top Level Statements
- Source generation
- Executable assemblies
There is a single project named ApiBootstrap that contains all the start-up code and API endpoint mapping. The SAM template still deploys a separate function per API endpoint to negate concurrency issues.
It uses the new minimal API hosting model as detailed here.
The code is compiled natively for either Linux-x86_64 or Linux-ARM64 and then deployed manually to Lambda as a zip file. The SAM deploy can still be used to stand up the API Gateway endpoints and DynamoDb table, but won't be able to deploy native AOT .NET Lambda functions yet. Packages need to be published from Linux, since cross-OS native compilation is not supported yet.
Details for compiling .NET 6 native AOT can be found here
The code is compiled on a custom runtime and deployed to the provided.al2 Lambda runtime because Lambda doesn't have a .NET 7 runtime. The code is compiled as ReadyToRun and Self-Contained because there is not .NET runtime on provided.al2 to depend on. This type of deployment is expected to be slower than a fully supported Lambda runtime like .NET 6. This sample should be able to be tested with sam build and then sam deploy --guided.
The code is compiled natively for Linux-x86_64 then deployed manually to Lambda as a zip file.
Details for compiling .NET 7 native AOT can be found here
There is a single project named ApiBootstrap that contains all the start-up code and API endpoint mapping. The code is compiled natively for Linux-x86_64 then deployed manually to Lambda as a zip file. Microsoft do not fully support ASP.NET for .NET 7 native AOT. This sample demonstrates that minimal API's can run on Lambda with native AOT, but the full ASP.NET feature set may not be supported.
Details for compiling .NET 7 native AOT can be found here
To deploy the architecture into your AWS account, navigate into the respective folder under the src folder and run 'sam deploy --guided'. This will launch a deployment wizard, complete the required values to initiate the deployment. For example, for .NET 6:
cd src/NET6
sam build
sam deploy --guidedBenchmarks are executed using Artillery. Artillery is a modern load testing & smoke testing library for SRE and DevOps.
To run the tests, use the below scripts. Replace the $API_URL with the API URL output from the deployment:
cd loadtest
artillery run load-test.yml --target "$API_URL"Below is the cold start and warm start latencies observed. Please refer to the load test folder to see the specifics of the test that were executed.
All latencies listed below are in milliseconds.
is used to make 100 requests / second for 10 minutes to our API endpoints.
AWS Lambda Power Tuning is used to optimize the cost/performance. 1024MB of function memory provided the optimal balance between cost and performance.
The below CloudWatch Log Insights query was used to generate the results:
filter @type="REPORT"
| fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart
| stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart
| Cold Start (ms) | Warm Start (ms) | |||||||
|---|---|---|---|---|---|---|---|---|
| p50 | p90 | p99 | max | p50 | p90 | p99 | max | |
| ARM64 | 1122.70 | 1170.83 | 1225.92 | 1326.32 | 5.55 | 8.74 | 19.85 | 256.55 |
| X86 | 1004.80 | 1135.81 | 1422.78 | 1786.78 | 6.11 | 10.82 | 29.40 | 247.32 |
| X86 with Open Telemetry | 1615.31 | 1704.93 | 1931.82 | 2067.97 | 7.04 | 12.08 | 35.57 | 1059.78 |
| Cold Start (ms) | Warm Start (ms) | |||||||
|---|---|---|---|---|---|---|---|---|
| p50 | p90 | p99 | max | p50 | p90 | p99 | max | |
| ARM64 | 873.59 | 909.23 | 944.42 | 945.25 | 5.50 | 9.24 | 19.53 | 421.72 |
| X86 | 778.74 | 966.39 | 1470.50 | 1659.51 | 6.41 | 11.90 | 31.33 | 255.98 |
| x86 with Powertools | 855.45 | 915.61 | 1031.25 | 1381.09 | 5.82 | 9.83 | 27.59 | 748.08 |
| Container Image on X86 | 980.98 | 1256.94 | 1532.01 | 1755.68 | 5.82 | 9.84 | 24.42 | 260.25 |
| ARM64 with top level statements | 916.53 | 955.82 | 985.90 | 1021.40 | 5.73 | 9.38 | 20.65 | 417.23 |
| Minimal API on ARM64 | 1149.95 | 1194.47 | 1239.47 | 1315.07 | 6.10 | 10.00 | 22.91 | 1315.07 |
| Native AOT on ARM64 | 448.97 | 467.75 | 493.20 | 516.6 | 6.30 | 10.49 | 21.50 | 461.35 |
| Native AOT on X86 | 466.81 | 542.86 | 700.45 | 730.51 | 6.21 | 11.34 | 24.69 | 371.16 |
| Cold Start (ms) | Warm Start (ms) | |||||||
|---|---|---|---|---|---|---|---|---|
| p50 | p90 | p99 | max | p50 | p90 | p99 | max | |
| X86 | 1467.56 | 1651.26 | 2423.83 | 2757.49 | 7.16 | 13.51 | 45.13 | 1378.88 |
| Native AOT on X86 | 372.43 | 435.70 | 581.62 | 740.17 | 6.77 | 12.52 | 45.44 | 118.93 |
| Native AOT container image on X86 | 237.7 | 266.78 | 266.78 | 266.78 | 6.20 | 10.66 | 25.41 | 243.72 |
| Native AOT container image on ARM64 | 237.29 | 260.40 | 356.40 | 400.73 | 5.86 | 9.69 | 22.37 | 264.14 |
| Native AOT Minimal API on X86* | 621.53 | 707.56 | 1112.84 | 1112.84 | 5.92 | 9.99 | 24.69 | 283.20 |
*Microsoft do not officially support ASP.NET Core for native AOT, some features of ASP.NET may not be supported.
Native AOT container samples use an Alpine base image. A cold start latency of ~1s was seen the first time an image was pushed and invoked.
On future invokes, even after forcing new Lambda execution environments, cold start latency is as seen above. Potential reasons why covered in an AWS blog post on optimizing Lambda functions packaged as containers.
.NET 8 is still in preview, and these numbers are subject to change as .NET 8 moves towards GA. Check back regularly for further updates.
The .NET 8 benchmarks include the number of cold and warm starts, alongside the performance numbers. Typically, the cold starts account for 1% or less of the total number of invocations.
| Cold Start (ms) | Warm Start (ms) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Invoke Count | p50 | p90 | p99 | max | Invoke Count | p50 | p90 | p99 | max | |
| X86 | 892 | 1476.39 | 1556.70 | 1878.51 | 2071.26 | 155,368 | 6.01 | 10.65 | 28.94 | 272.89 |
| Minimal API | 204 | 1672.86 | 1737.62 | 1912.61 | 1931.42 | 154,627 | 5.92 | 9.83 | 26.31 | 247.47 |
| X86 Native AOT* | 245 | 361.79 | 419.05 | 515.88 | 568.71 | 150,431 | 6.01 | 9.83 | 26.31 | 243.57 |
| Native AOT with Minimal API* | 84 | 535.52 | 611.54 | 835.28 | 835.28 | 155,679 | 6.11 | 10.65 | 28.03 | 230.66 |
*The .NET 8 native AOT examples need to be compiled on Amazon Linux 2, this is a temporary solution as a pre-cursor to a SAM build image being available for .NET 8.
You can find implementations of this project in other languages here:
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.