/serverless-dotnet-demo

Primary LanguageC#MIT No AttributionMIT-0

Lambda Demo with .NET

With the release of .NET 6 AWS Lambda now supports .NET Core 3.1 and .NET 6 as managed runtimes. With the availability of ARM64 using Graviton2 there have been vast improvements to using .NET with Lambda.

But how does that translate to actual application performance? And how does .NET compare to other available runtimes. This repository contains a simple serverless application across a range of .NET implementations and the corresponding benchmarking results.

Application

The application consists of an Amazon API Gateway backed by four Lambda functions and an Amazon DynamoDB table for storage.

It includes the below implementations as well as benchmarking results for both x86 and ARM64:

  • .NET Core 3.1
  • .NET Core 3.1 with Open Telemetry tracing
  • .NET 6 Lambda
  • .NET 6 Top Level statements
  • .NET 6 Minimal API
  • .NET 6 NativeAOT compilation
  • .NET 7 Custom Runtime
  • .NET 7 NativeAOT compilation
  • .NET 7 Minimal API with NativeAOT compilation
  • .NET 8
  • .NET 8 Minimal API

Requirements

Software

There are four implementations included in the repository, covering a variety of Lambda runtimes and features. All the implementations use 1024MB of memory with Graviton2 (ARM64) as default. Tests are also executed against x86_64 architectures for comparison.

There is a separate project for each of the four Lambda functions, as well as a shared library that contains the data access implementations. It uses the hexagonal architecture pattern to decouple the entry points, from the main domain and storage logic.

.NET 6

This implementation is the simplest route to upgrade a .NET Core 3.1 function to use .NET 6 as it only requires upgrading the function runtime, project target framework and any dependencies as per the final section of this link.

.NET 6 Top Level Statements

This implementation uses the new features detailed in this link including

  • Top Level Statements
  • Source generation
  • Executable assemblies

Minimal API

There is a single project named ApiBootstrap that contains all the start-up code and API endpoint mapping. The SAM template still deploys a separate function per API endpoint to negate concurrency issues.

It uses the new minimal API hosting model as detailed here.

.NET 6 native AOT

The code is compiled natively for either Linux-x86_64 or Linux-ARM64 and then deployed manually to Lambda as a zip file. The SAM deploy can still be used to stand up the API Gateway endpoints and DynamoDb table, but won't be able to deploy native AOT .NET Lambda functions yet. Packages need to be published from Linux, since cross-OS native compilation is not supported yet.

Details for compiling .NET 6 native AOT can be found here

.NET 7 Custom Runtime

The code is compiled on a custom runtime and deployed to the provided.al2 Lambda runtime because Lambda doesn't have a .NET 7 runtime. The code is compiled as ReadyToRun and Self-Contained because there is not .NET runtime on provided.al2 to depend on. This type of deployment is expected to be slower than a fully supported Lambda runtime like .NET 6. This sample should be able to be tested with sam build and then sam deploy --guided.

.NET 7 native AOT

The code is compiled natively for Linux-x86_64 then deployed manually to Lambda as a zip file.

Details for compiling .NET 7 native AOT can be found here

.NET 7 minimal API with native AOT

There is a single project named ApiBootstrap that contains all the start-up code and API endpoint mapping. The code is compiled natively for Linux-x86_64 then deployed manually to Lambda as a zip file. Microsoft do not fully support ASP.NET for .NET 7 native AOT. This sample demonstrates that minimal API's can run on Lambda with native AOT, but the full ASP.NET feature set may not be supported.

Details for compiling .NET 7 native AOT can be found here

Deployment

To deploy the architecture into your AWS account, navigate into the respective folder under the src folder and run 'sam deploy --guided'. This will launch a deployment wizard, complete the required values to initiate the deployment. For example, for .NET 6:

cd src/NET6
sam build
sam deploy --guided

Testing

Benchmarks are executed using Artillery. Artillery is a modern load testing & smoke testing library for SRE and DevOps.

To run the tests, use the below scripts. Replace the $API_URL with the API URL output from the deployment:

cd loadtest
artillery run load-test.yml --target "$API_URL"

Summary

Below is the cold start and warm start latencies observed. Please refer to the load test folder to see the specifics of the test that were executed.

All latencies listed below are in milliseconds.

is used to make 100 requests / second for 10 minutes to our API endpoints.

AWS Lambda Power Tuning is used to optimize the cost/performance. 1024MB of function memory provided the optimal balance between cost and performance.

Results

The below CloudWatch Log Insights query was used to generate the results:

filter @type="REPORT"
| fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart
| stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart

.NET Core 3.1

Cold Start (ms) Warm Start (ms)
p50 p90 p99 max p50 p90 p99 max
ARM64 1122.70 1170.83 1225.92 1326.32 5.55 8.74 19.85 256.55
X86 1004.80 1135.81 1422.78 1786.78 6.11 10.82 29.40 247.32
X86 with Open Telemetry 1615.31 1704.93 1931.82 2067.97 7.04 12.08 35.57 1059.78

.NET 6

Cold Start (ms) Warm Start (ms)
p50 p90 p99 max p50 p90 p99 max
ARM64 873.59 909.23 944.42 945.25 5.50 9.24 19.53 421.72
X86 778.74 966.39 1470.50 1659.51 6.41 11.90 31.33 255.98
x86 with Powertools 855.45 915.61 1031.25 1381.09 5.82 9.83 27.59 748.08
Container Image on X86 980.98 1256.94 1532.01 1755.68 5.82 9.84 24.42 260.25
ARM64 with top level statements 916.53 955.82 985.90 1021.40 5.73 9.38 20.65 417.23
Minimal API on ARM64 1149.95 1194.47 1239.47 1315.07 6.10 10.00 22.91 1315.07
Native AOT on ARM64 448.97 467.75 493.20 516.6 6.30 10.49 21.50 461.35
Native AOT on X86 466.81 542.86 700.45 730.51 6.21 11.34 24.69 371.16

.NET 7

Cold Start (ms) Warm Start (ms)
p50 p90 p99 max p50 p90 p99 max
X86 1467.56 1651.26 2423.83 2757.49 7.16 13.51 45.13 1378.88
Native AOT on X86 372.43 435.70 581.62 740.17 6.77 12.52 45.44 118.93
Native AOT container image on X86 237.7 266.78 266.78 266.78 6.20 10.66 25.41 243.72
Native AOT container image on ARM64 237.29 260.40 356.40 400.73 5.86 9.69 22.37 264.14
Native AOT Minimal API on X86* 621.53 707.56 1112.84 1112.84 5.92 9.99 24.69 283.20

*Microsoft do not officially support ASP.NET Core for native AOT, some features of ASP.NET may not be supported.

Native AOT container samples use an Alpine base image. A cold start latency of ~1s was seen the first time an image was pushed and invoked.

On future invokes, even after forcing new Lambda execution environments, cold start latency is as seen above. Potential reasons why covered in an AWS blog post on optimizing Lambda functions packaged as containers.

.NET 8 Preview 4

.NET 8 is still in preview, and these numbers are subject to change as .NET 8 moves towards GA. Check back regularly for further updates.

The .NET 8 benchmarks include the number of cold and warm starts, alongside the performance numbers. Typically, the cold starts account for 1% or less of the total number of invocations.

Cold Start (ms) Warm Start (ms)
Invoke Count p50 p90 p99 max Invoke Count p50 p90 p99 max
X86 892 1476.39 1556.70 1878.51 2071.26 155,368 6.01 10.65 28.94 272.89
Minimal API 204 1672.86 1737.62 1912.61 1931.42 154,627 5.92 9.83 26.31 247.47
X86 Native AOT* 245 361.79 419.05 515.88 568.71 150,431 6.01 9.83 26.31 243.57
Native AOT with Minimal API* 84 535.52 611.54 835.28 835.28 155,679 6.11 10.65 28.03 230.66

*The .NET 8 native AOT examples need to be compiled on Amazon Linux 2, this is a temporary solution as a pre-cursor to a SAM build image being available for .NET 8.

👀 With other languages

You can find implementations of this project in other languages here:

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.