package-vulnerability-scanner-assessment

Purpose

A simple project/solution containing a number of known 3rd party vulnerabilities to test the accuracy and coverage of various tools (both integrated and CLI) in identifying vulnerable third party packages.

** DISCLAIMER ** This repository is in no way an endorsement nor condemnation of any of the tools listed. The findings and results contained herein are based off an incredibly small sample size and should not be used as a measure of the protection gained from any of the tools mentioned.

Why

Each vulnerability scanner will likely be curating their own vulnerability database from many different sources (or integrating with an existing vulnerability DB) and verifying that against a projects installed packages. This process involves identifying, researching/assessing and cataloging these vulnerabilities which can involve a considerable resources, as such, it is unlikely any one vulnerability database will encompass all known vulnerabilities.

Some prominent vulnerability databases for scanners include:

Snyk vulnerability DB : ~780 Nuget vulnerabilities
Github advisory DB : ~142 Nuget vulnerabilities
OSS index : unknown Nuget vulnerabilities

Method

A template dotnet core 5 app has been created with a small number of known vulnerable Nuget packages added to the project file. Integrated scanning tools will be setup manually outside of this repository and configured to scan regularly. CLI tools will be manually executed periodically.

Vulnerability Scanners

Snyk/Snyk vulnerability DB
- Enabled as Github Integration.
Dependabot alerts/Github advisory DB
- Configured in repository settings.
.Net Core CLI/Github advisory DB - Requires .Net Core SDK 5.0.200+
- Executed locally via dotnet list package --include-transitive
Sonatype Lift/OSS index - Does not yet support Nuget
- Enabled as Github App
Sonatype DepSheild/OSS index - Does not yet support Nuget
- Enabled as Github App

Vulnerable Packages

System.Text.Encodings.Web: 5.0.0
IdentityServer4: 2.1.2
Microsoft.AspNetCore.Authentication.JwtBearer: 5.0.8
System.Text.RegularExpressions: 4.3.0
Angularjs.Core: 1.6.0

Results

Nuget

Vulnerability	Snyk	Dependabot	.Net Core CLI	Sonatype Lift	Sonatype DepSheild
System.Text.Encodings.Web	✔️	✔️	✔️	❌	❌
IdentityServer4	✔️	❌	✔️	❌	❌
Microsoft.AspNetCore.Authentication.JwtBearer	✔️	✔️	✔️	❌	❌
System.Text.RegularExpressions	✔️	✔️	✔️	❌	❌
Angularjs.Core	✔️	❌	❌	❌	❌

Future

Add more projects that utilise the other common package managers e.g. npm, pip, RubyGems etc.

codeLeeek/package-vulnerability-scanner-assessment