Windows Azure Storage Extensions

Info

Version	Dependencies	Information
	Windows.Azure.Storage	This dependency is declared deprecated by Microsoft.
	Microsoft.Azure.Cosmos.Table
	Microsoft.Azure.Cosmos.Table - Preview	This is still a preview version.

Windows Azure Storage Extensions is a .NET library aimed at managing and querying entities from Azure Storage Tables.

It's built on top of the Azure .NET SDK, provides async interfaces (Task-based Asynchronous Pattern) and LINQ to Azure Table queries via TableSet context by using POCO entities.

Features
Code Samples
Sponsors

Features

Attribute Mapping

POCO properties and fields should be marked by one or both of PartitionKey and RowKey attributes for defining composite table key. Also can be used Timestamp, ETag, Property and Ignore attributes.

Fluent Mapping

Fluent mapping is the namesake mapping style that we use as an alternative to the AttributeMapping. It's a fluent interface that allows you to map your entities completely in code, with all the compile-time safety and refactorability that brings.

EntityTypeMap class

EntityTypeMap class is the basis of all your mappings, you derive from this to map anything.

public class AddressMap : EntityTypeMap<Address>
{
  public AddressMap()
  {
      this.PartitionKey(p => p.CountryCode)
          .RowKey(p => p.Id)
          .Ignore(p => c.Country);
  }
}

You map your entities properties inside the constructor.

Syntax note: Every mapping inside a EntityTypeMap is built using lambda expressions, which allow us to reference the properties on your entities without sacrificing compile-time safety. The lambdas typically take the form of x => x.Property. The x on the left is the parameter declaration, which will implicitly be of the same type as the entity being mapped, while the x.Property is accessing a property on your entity (coincidentally called "Property" in this case).

Once you've declared your EntityTypeMap you're going to need to map the properties on your entity. There are several methods available that map your properties in different ways, and each one of those is a chainable method that you can use to customise the individual mapping.

PartitionKey and RowKey

Every mapping requires an PartitionKey and RowKey of some kind.

The PartitionKey is mapped using the PartitionKey method, which takes a lambda expression that accesses the property on your entity that will be used for the PartitionKey.

PartitionKey(x => x.MyPartitionKeyProperty);

The PartitionKey is mapped using the RowKey method, which takes a lambda expression that accesses the property on your entity that will be used for the PartitionKey.

RowKey(x => x.MyRowKeyProperty);

Ignore

If you need to ignore a property by using the Ignore method, which takes a lambda expression that accesses the property on your entity that will ignored by the serializer.

Ignore(x => x.MyPropertyToIgnore);

ETag and Timestamp

The ETag property is mapped using the methods ETag, which takes a lambda expression that accesses the property on your entity that will be mapped.

ETag(x => x.MyETagProperty);

The Timestamp property is mapped using the methods Timestamp, which takes a lambda expression that accesses the property on your entity that will be mapped.

Timestamp(x => x.MyTimestampProperty);

Register custom mapping assembly

The TableEntityConverter class will try to find the mapping classes for your entities on the same assembly of the entitie, if you are using a different assembly for mappings classes you'll need to register this assembly by using the RegisterAssembly method from class EntityTypeMap. ATTENTION: this method just need to be called once and before instantiating the TableSet class. Below there is a sample of how we call the

EntityTypeMap.RegisterAssembly(typeof(MyEntity).Assembly)

Entities Management

Generic TableSet context provides a synchronous & asynchronous (TAP) methods for managing entities:

Synchronous: Add, AddOrUpdate, Update and Remove.
Asynchronous: AddAsync, AddOrUpdateAsync, UpdateAsync and RemoveAsync.

To avoid restrictions of group operations in Azure Storage all entities sorted by partition keys and merged into groups by 100 entities. Execution of requests with such batch operations can be configured via TableSet's ExecutionMode property. Allowed values:

Sequential
Parallel

Default ExecutionMode is Sequential.

LINQ Queries

TableSet context implements IQueryable interface for using LINQ Expressions. Provider supports next synchronous LINQ methods:

First
FirstOrDefault
Single
SingleOrDefault
Take
Where

To utilize filtering capabilities of string properties it supports:

Also you can use Contains method. In this case query statement for each collection's item will be joined by using OData or operator.

NOTE: For creating a custom queries you should take a look at next article: Mixing LINQ Providers and LINQ to Objects.

Asynchronous LINQ Queries

In addition TableSet can be used for asynchronous queries powered by LINQ extensions (TAP) in EF 6 Async style.

Available methods:

FirstAsync
FirstOrDefaultAsync
SingleAsync
SingleOrDefaultAsync
TakeAsync
ToListAsync

LINQ Projections

LINQ Projections supported with a limitation - projection class should be a reference type.

TAP-based Extensions

Library contains TAP-based extensions for following Azure Storage Library classes:

CloudBlobClient
CloudBlobContainer
CloudTableClient
CloudTable

To use it just add Async postfix to synchronous method name for instance:

blobs = cloudBlobContainer.ListBlobs();
blobs = await cloudBlobContainer.ListBlobsAsync();

Task Cancellation

All of TAP-based methods accepts optional CancellationToken parameter for Task Cancellation.

Code Samples

Declaring a new POCO class and using attribute mapping:

public sealed class Country
{
    [PartitionKey]
    public string Continent { get; set; }

    [RowKey]
    public string Name { get; set; }

    public long Population { get; set; }
    public double Area { get; set; }
    public DateTime Formed { get; set; }
}

Declaring a new POCO class and using fluent mapping:

public sealed class Country
{
    public string Continent { get; set; }
    public string Name { get; set; }
    public long Population { get; set; }
    public double Area { get; set; }
    public DateTime Formed { get; set; }
}

public class CountryMapping : WindowsAzure.Table.EntityConverters.TypeData.EntityTypeMap<Country>
{
    public CountryMapping() {
        this.PartitionKey(p => p.Continent)
            .RowKey(p => p.Name);
    }
}

Creating a new table context:

var storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
var tableClient = storageAccount.CreateCloudTableClient();

var countryTable = new TableSet<Country>(tableClient);

Adding a new entity:

var resultSync = countryTable.Add(country);
var resultAsync = await countryTable.AddAsync(country);

Updating an entity:

resultSync.Area += 333333;
resultSync = countryTable.Update(resultSync);

resultAsync.Population *= 2;
resultAsync = await countryTable.UpdateAsync(resultAsync);

Removing entities:

countryTable.Remove(resultSync);
await countryTable.RemoveAsync(resultAsync);

Querying entities:

var query = countryTable.Where(
        p => p.Formed > new DateTime(1950, 1, 1) &&
             (p.PresidentsCount < 10 ||
              p.Population < 10000000 && p.PresidentsCount > 10 && p.IsExists));

resultsSync = query.ToList();
resultsAsync = await query.ToListAsync();

Using LINQ projections:

var projection = from country in countryTable
                 where country.Area > 400000
                 select new { country.Continent, country.Name };

var result = projection.ToList();
result = await projection.ToListAsync();

Using Contains in the LINQ query:

var countryNames = new List<string> { "Germany", "Finland" };
var countries = countryTable.Where(p => countryNames.Contains(p.Name)).ToList();

Build script

We create a build script using Cake.

Files of interest

We need somes files for a build script work.

build.ps1, build.sh and build.cmd (an alias to build.ps1)

These are bootstrapper scripts that ensure you have Cake and other required dependencies installed. The bootstrapper scripts are also responsible for invoking Cake. These files are optional, and not a hard requirement. If you would prefer not to use these scripts you can invoke Cake directly from the command line, once you have downloaded and extracted it.

tools/ folder and tools/packages.config

This is the package configuration that tells the bootstrapper script what NuGet packages to install in the tools folder. An example of this is Cake itself or additional tools such as unit test runners, ILMerge etc.

build.cake

This is the actual build script, it doesn't have to be named this but this will be found by default and we keeped. Our build script executes on base of two list of projects:

List of projects to build and when needed package and publish
List of projects to build, test and when needed collect code coverage

Because of that the build needs two main file glob arguments, that are used to find the projects that matches this glob pattern, more about the argument bellow.

Build Script Arguments

More arguments can be found on the build.cake script.

--Project file glob pattern for the projects to build, package and publish.
--Tests file glob pattern for the projects to build, test and collect code coverage.
--Target defines the actions/task to be executed.
--Configuration defines the build configuration to be used on projects.
--PackageVersion defines the version to bump into the project before build, package and publish
--NugetSource nuget source api URL
--NugetApiKey nuget source api key

Build samples

This sample we'll only build the foundation project.

For Windows

.\build.cmd --Target=Build --Projects="./WindowsAzure/*.csproj"

For MacOS / Linux

.\build.sh --Target=Build --Projects="./WindowsAzure/*.csproj"

Build and Test sample

This sample we'll only build the foundation project.

For Windows

.\build.cmd --Target=Test --Projects="./WindowsAzure/*.csproj" --Tests="./WindowsAzure.Tests/*.csproj"

For MacOS / Linux

.\build.sh --Target=Test --Projects="./WindowsAzure/*.csproj" --Tests="./WindowsAzure.Tests/*.csproj"

Build, Test and Collect Code Coverage sample

This sample we'll only build the foundation project.

For Windows

.\build.cmd --Target=TestCoverage --Projects="./WindowsAzure/*.csproj" --Tests="./WindowsAzure.Tests/*.csproj"

For MacOS / Linux

.\build.sh --Target=TestCoverage --Projects="./WindowsAzure/*.csproj" --Tests="./WindowsAzure.Tests/*.csproj"

Build, Test and Publish to Visual Studio NuGet

This sample we'll only build the foundation project.

For Windows

.\build.cmd --Target=Publish --PackageVersion=1.6.0 --Projects="./WindowsAzure/*.csproj" --Tests="./WindowsAzure.Tests/*.csproj" --NugetApiKey={the nuget apikey come here}

For MacOS / Linux

.\build.sh --Target=Publish --PackageVersion=1.6.0 --Projects="./WindowsAzure/*.csproj" --Tests="./WindowsAzure.Tests/*.csproj" --NugetApiKey={the nuget apikey come here}

Contributors

Great thanks to all of projects contributors.

Dmitry Tretyakov for project idea.
Timothy Makarov for performance optimizations.
Gabriel Marquez for Fluent API mapping.
Sebastian Betzin for additional TableSet methods.
Tom Dietrich for his contributions.
Stef Heyenrath for his contributions related to Microsoft.Azure.Cosmos.Table.
And Community for a valuable feedback.

See complete list of project contributors.

We appreciate all kinds of feedback, so please feel free to send a PR or write an issue.

dtretyakov/WindowsAzure

Windows Azure Storage Extensions

Info

Table Of Contents

Features

Attribute Mapping

Fluent Mapping

EntityTypeMap class

PartitionKey and RowKey

Ignore

ETag and Timestamp

Register custom mapping assembly

Entities Management

LINQ Queries

Asynchronous LINQ Queries

LINQ Projections

TAP-based Extensions

Task Cancellation

Code Samples

Build script

Files of interest

Build Script Arguments

Build samples

Build and Test sample

Build, Test and Collect Code Coverage sample

Build, Test and Publish to Visual Studio NuGet

Contributors