Verify
Verify is a snapshot tool that simplifies the assertion of complex data models and documents.
Verify is called on the test result during the assertion phase. It serializes that result and stores it in a file that matches the test name. On the next test execution, the result is again serialized and compared to the existing file. The test will fail if the two snapshots do not match: either the change is unexpected, or the reference snapshot needs to be updated to the new result.
Part of the .NET Foundation
NuGet packages
- https://nuget.org/packages/Verify.Xunit/
- https://nuget.org/packages/Verify.NUnit/
- https://nuget.org/packages/Verify.Expecto/
- https://nuget.org/packages/Verify.MSTest/
Snapshot management
Accepting or declining a snapshot file is part of the core workflow of Verify. There are several ways to do this and the approach(s) selected is a personal preference.
- In the Windows Tray via DiffEngineTray
- ReSharper test runner support
- Rider test runner support
- Via the clipboard.
- Manually making the change in the launched diff tool. Either with a copy paste, or some tools have commands to automate this via a shortcut or a button.
- Manually on the file system. By renaming the
.received.
file to.verified.
. This can be automated via a scripted to bulk accept all (by matching a pattern).received.
files.
Usage
All examples use ImplicitUsings. Ensure the following is set to have examples compile correctly <ImplicitUsings>enable</ImplicitUsings>
Class being tested
Given a class to be tested:
public static class ClassBeingTested
{
public static Person FindPerson()
{
return new()
{
Id = new("ebced679-45d3-4653-8791-3d969c4a986c"),
Title = Title.Mr,
GivenNames = "John",
FamilyName = "Smith",
Spouse = "Jill",
Children = new()
{
"Sam",
"Mary"
},
Address = new()
{
Street = "4 Puddle Lane",
Country = "USA"
}
};
}
}
xUnit
Support for xUnit
[UsesVerify]
public class Sample
{
[Fact]
public Task Test()
{
var person = ClassBeingTested.FindPerson();
return Verify(person);
}
}
NUnit
Support for NUnit
[TestFixture]
public class Sample
{
[Test]
public Task Test()
{
var person = ClassBeingTested.FindPerson();
return Verify(person);
}
}
Expecto
Support for Expecto
open Expecto
open VerifyTests
open VerifyExpecto
[<Tests>]
let tests =
testTask "findPerson" {
let person = ClassBeingTested.FindPerson();
do! Verifier.Verify("findPerson", person)
}
Caveats
Due to the nature of the Expecto implementation, the following APIs in Verify are not supported.
settings.UseTypeName()
settings.UseMethodName()
settings.UseParameters()
settings.UseTextForParameters()
Instead use a custom name
parameter.
MSTest
Support for MSTest
[TestClass]
public class Sample :
VerifyBase
{
[TestMethod]
public Task Test()
{
var person = ClassBeingTested.FindPerson();
return Verify(person);
}
}
Initial Verification
When the test is initially run will fail with:
First verification. Sample.Test.verified.txt not found.
Verification command has been copied to the clipboard.
The clipboard will contain the following:
cmd /c move /Y "C:\Code\Sample\Sample.Test.received.txt" "C:\Code\Sample\Sample.Test.verified.txt"
Notes:
- More Clipboard info.
- An alternative to using the clipboard is the DiffEngineTray tool.
If a Diff Tool is detected it will display the diff:
To verify the result:
- Execute the command from the clipboard, or
- Use the diff tool to accept the changes, or
- Manually copy the text to the new file
Verified result
This will result in the Sample.Test.verified.txt
being created:
{
GivenNames: John,
FamilyName: Smith,
Spouse: Jill,
Address: {
Street: 4 Puddle Lane,
Country: USA
},
Children: [
Sam,
Mary
],
Id: Guid_1
}
Subsequent Verification
If the implementation of ClassBeingTested
changes:
public static class ClassBeingTested
{
public static Person FindPerson()
{
return new()
{
Id = new("ebced679-45d3-4653-8791-3d969c4a986c"),
Title = Title.Mr,
// Middle name added
GivenNames = "John James",
FamilyName = "Smith",
Spouse = "Jill",
Children = new()
{
"Sam",
"Mary"
},
Address = new()
{
// Address changed
Street = "64 Barnett Street",
Country = "USA"
}
};
}
}
And the test is re run it will fail with
Verification command has been copied to the clipboard.
Assert.Equal() Failure
↓ (pos 21)
Expected: ···\n GivenNames: 'John',\n FamilyName: 'Smith',\n Spouse: 'Jill···
Actual: ···\n GivenNames: 'John James',\n FamilyName: 'Smith',\n Spouse:···
↑ (pos 21)
The clipboard will again contain the following:
cmd /c move /Y "C:\Code\Sample\Sample.Test.received.txt" "C:\Code\Sample\Sample.Test.verified.txt"
See also: Clipboard
Diff Tool will display the diff:
TheThe same approach can be used to verify the results and the change to Sample.Test.verified.txt
is committed to source control along with the change to ClassBeingTested
.
VerifyJson
VerifyJson
performs the following actions
- Convert to
JToken
(if necessary). - Apply ignore member by name for keys.
- PrettyPrint the resulting text.
[Fact]
public Task VerifyJsonString()
{
var json = "{'key': {'msg': 'No action taken'}}";
return VerifyJson(json);
}
[Fact]
public Task VerifyJsonStream()
{
var json = "{'key': {'msg': 'No action taken'}}";
var stream = new MemoryStream(Encoding.UTF8.GetBytes(json));
return VerifyJson(stream);
}
[Fact]
public Task VerifyJsonJToken()
{
var json = "{'key': {'msg': 'No action taken'}}";
var target = JToken.Parse(json);
return VerifyJson(target);
}
Results in:
{
key: {
msg: No action taken
}
}
Received and Verified
- All
*.verified.*
files should be committed to source control. - All
*.received.*
files should be excluded from source control.
Static settings
Most settings are available at the both global level and at the instance level.
When modifying settings at the both global level it should be done using a Module Initializer:
[UsesVerify]
public class StaticSettings
{
[Fact]
public Task Test()
{
return Verify("String to verify");
}
}
public static class StaticSettingsUsage
{
[ModuleInitializer]
public static void Initialize()
{
VerifierSettings.AddScrubber(_ => _.Replace("String to verify", "new value"));
}
}
Versioning
Verify follows Semantic Versioning. The same applies for extensions to Verify. Small changes in the resulting snapshot files may be deployed in a minor version. As such nuget updates to Verify.*
should be done as follows:
- Updates all
Verify.*
packages in isolation - Re-run all tests.
- If there are changes, ensure they look correct given the release notes. If the changes do not look correct, raise an issue.
- Accept those changes.
Snapshot changes do not trigger a major version change to avoid causing Diamond dependency issues for downstream extensions.
Media
- Verify Xunit Intro (26 Apr 2020)
- OSS Power-Ups: Verify (14 Jul 2021)
- Unhandled Exception podcast: Snapshot Testing (26 Nov 2021)
- Testing an incremental generator with snapshot testing (14 Dec 2021)
- 5 helpful Nuget package for Unit Testing in .NET (16 Oct 2021)
- 5 open source .NET projects that deserve more attention (9 Sep 2021)
Extensions
- Verify.AngleSharp: Html verification utilities via AngleSharp.
- Verify.AspNetCore: Verification of AspNetCore bits.
- Verify.Aspose: Verification of documents (pdf, docx, xslx, and pptx) via Aspose.
- Verify.Blazor: Verification of Blazor Component via bunit or via raw Blazor rendering.
- Verify.Cosmos: Verification of Azure CosmosDB.
- Verify.DiffPlex: Comparison of text via DiffPlex.
- Verify.DocNet: Verification of pdfs via DocNet.
- Verify.EntityFramework: Verification of EntityFramework bits.
- Verify.GrapeCity: Verification of documents (pdf, docx, and xslx) via GrapeCity Documents API.
- Verify.HeadlessBrowsers: Verification of Web UIs using Playwright, Puppeteer Sharp, or Selenium.
- Verify.Http: Verification of Http bits.
- Verify.ICSharpCode.Decompiler: Comparison of assemblies and types via ICSharpCode.Decompiler.
- Verify.ImageHash: Comparison of images via ImageHash.
- Verify.ImageMagick: Verification and comparison of images via Magick.NET.
- Verify.ImageSharp: Verification of images via ImageSharp.
- Verify.NServiceBus: Verify NServiceBus Test Contexts.
- Verify.MicrosoftLogging: Verify MicrosoftLogging.
- Verify.MongoDB: Verification of MongoDB bits.
- Verify.NodaTime: Support for NodaTime.
- Verify.NSubstitute: Support for NSubstitute types
- Verify.Phash: Comparison of images via Phash.
- Verify.Quibble: Comparison of objects via Quibble.
- Verify.RavenDb: Verification of RavenDb bits.
- Verify.SqlServer: Verification of SqlServer bits.
- Verify.SourceGenerators: Verification of C# Source Generators.
- Verify.WinForms: Verification of WinForms UIs.
- Verify.Xamarin: Verification of Xamarin UIs.
- Verify.Xaml: Verification of Xaml UIs.
- Spectre.Verify.Extensions: Add an attribute driven file naming convention to Verify.
- Verify.Syncfusion: Verification of documents (pdf, docx, xslx, and pptx) via Syncfusion File Formats.
More Documentation
- Clipboard
- Compared to assertions
- Verify options
- Serializer Settings
- File naming
- Parameterised tests
- Named Tuples
- Scrubbers
- Diff Engine
- Diff Tools
- Diff Tool Order
- Custom Diff Tool
- Using anonymous types
- Verifying binary data
- Build server
- Comparers
- Converters
- FSharp Usage
- Compared to ApprovalTests
Alternatives
Projects/tools that may be a better alternative to Verify
Icon
Helmet designed by Leonidas Ikonomou from The Noun Project.