Microsoft Graph SDK Code Generator

Source code writers for VIPR utilizing T4 templates. The GraphODataTemplateWriter receives an OdcmModel from VIPR and uses it to fill in a T4 template located within this repository.

Currently the following target languages are supported by this writer:

CSharp
Java
Objective-C
Python
TypeScript
PHP

Prerequisites
Getting started
Using Vipr with this writer
Contributing
License

Prerequisites

Getting started

This project uses git submodules to integrate upstream dependencies, specifically Vipr. If you need an alternate branch to include special fixes you'll need to check that out manually within the submodule.

For the solution to open properly, ensure submodules are updated before opening it in Visual Studio. When initially cloning this repo, use git clone --recursive to update submodules at the same time. Later, run git submodule update to manually update submodules. If you don't use the --recursive switch when cloning, run git submodule init first to initialize the submodule.

Once setup is complete, you can work with the GraphODataTemplateWriter solution as usual. If you encounter problems, make sure NuGet packages and project references are all up-to-date.

For more information on submodules read this chapter from the Git book and search the Web.

Using Typewriter

Typewriter is a new solution for generating code files using the GraphODataTemplateWriter and VIPR. It is an executable that is intended to simplify the generation of code files. Build the solution to find the typewriter executable in \MSGraph-SDK-Code-Generator\src\Typewriter\bin\Release. The typewriter run options are:

-l, -language: The target language for the generated code files. The values can be: Java, ObjC, CSharp, PHP, Python, TypeScript, or GraphEndpointList. The default value is CSharp. This is not applicable when only generating clean and annotated metadata as specified by the -generationmode Metadata option.
-m, -metadata: The local file path or URL to the target input metadata. The default value is https://graph.microsoft.com/v1.0/$metadata. This value is required.
-v, -verbosity: The log verbosity level. The values can be: Minimal, Info, Debug, or Trace. The default value is Minimal.
-o, -output: Specifies the path to the output folder. The default value is the directory that contains typewriter.exe. The structure and contents of the output directory will be different based on the -generationmode and -language options.
-d, -docs: Specifies the path to the base endpoint directory in the microsoft-graph-docs repo. For example, use c:\microsoft-graph-docs\api-reference\v1.0\ for the v1.0 endpoint documentation. The documentation is parsed to provide documentation annotations to the metadata which is then used to add doc comments in the generated code files. This option is required when using -generationmode values of Metadata, Full, or TransformWithDocs.
-g, -generationmode: Specifies the generation mode. The values can be: Full, Metadata, Files, Transform or TransformWithDocs. Full (default) generation mode produces the output code files by cleaning the input metadata, parsing the documentation, and adding annotations before generating the output files. Metadata generation mode produces an output metadata file by cleaning metadata, documentation parsing, and adding documentation annotations. Files generation mode produces code files from an input metadata and bypasses the cleaning, documentation parsing, and adding documentation annotations. Transform generation mode processes the metadata according to the XSLT provided with the -t option. TransformWithDocs generation mode processes the metadata according to the XSLT and adds documentation annotations. Transform and TransformWithDocs require the -t argument to specify the XSLT file.
-f, -outputMetadataFileName: The base output metadata filename. Only applicable for -generationmode Metadata. The default value is cleanMetadataWithDescriptions which is used with the value of the -endpointVersion to generate a metadata file named cleanMetadataWithDescriptionsv1.0.xml.
-e, -endpointVersion: The endpoint version used when naming a metadata file. Expected values are v1.0 and beta. Only applicable for -generationmode Metadata.
-p, -properties: Specify properties to support generation logic in the T4 templates. Properties must take the form of key-string:value-string. Multiple properties can be specified by setting a space in between property. The only property currently supported is the php.namespace property to specify the generated model file namespace. This property is optional.
-t, -transform: Specify the URI to the XSLT that will preprocess the metadata. Only applicable for -generationmode Transform or -generationmode TransformWitDocs.
-r, -removeannotations: Specifies whether to remove capability annotations from the metadata. The default value is true. Only applicable for -generationmode Transform or -generationmode TransformWithDocs.

Example typewriter usage

Transform metadata with XSLT.

The output cleanMetadata.xml will be located in the same directory as typewriter.exe.

.\typewriter.exe -v Info -m https://raw.githubusercontent.com/microsoftgraph/msgraph-metadata/master/v1.0_metadata.xml -g Transform -t https://raw.githubusercontent.com/microsoftgraph/msgraph-metadata/master/transforms/csdl/preprocess_csdl.xsl

Transform metadata with XSLT and add documentation annotations.

The output cleanMetadataWithDescriptionsv1.0.xml will be located in the same directory as typewriter.exe.

.\typewriter.exe -v Info -m https://raw.githubusercontent.com/microsoftgraph/msgraph-metadata/master/v1.0_metadata.xml -g TransformWithDocs -t https://raw.githubusercontent.com/microsoftgraph/msgraph-metadata/master/transforms/csdl/preprocess_csdl.xsl -d D:\repos\microsoft-graph-docs

Generate TypeScript typings from a CSDL (metadata) file without cleaning or annotating the CSDL.

The output will go in to the outputTypeScript directory.

.\typewriter.exe -v Info -m D:\cleanMetadataWithDescriptions_v10.xml -o outputTypeScript -l TypeScript -g Files

Clean and annotate a metadata file with documentation annotations sourced from the documentation repo

The output metadata file will go in to the output2 directory. The output metadata file will be named cleanMetadataWithDescriptionsv1.0.xml based on the default values.

.\typewriter.exe -v Info -m D:\v1.0_2018_10_23_source.xml -o output2 -d D:\repos\microsoft-graph-docs -g Metadata

Generate C# code files from the metadata that will be cleaned and annotated with documentation annotations sourced from the documentation repo

The output C# code files will go in to the output directory.

.\typewriter.exe -v Info -m D:\v1.0_2018_10_23_source.xml -o output -l CSharp -d D:\repos\microsoft-graph-docs -g Full

Use/debug Typewriter in VSCode

Prerequisite is that msgraph-metadata is checked out at the same root folder as MSGraph-SDK-Code-Generator.
launch.json file contains main generation scenarios for each language/endpoint pair. Running Typewriter using one of those generation scenarios (e.g. Generate CSharp beta) will output the corresponding models into a folder, which will be a sibling to MSGraph-SDK-Code-Generator. The folder will be named as generator-output-<language>-<endpoint> e.g. generator-output-CSharp-beta.

Use Typewriter to test your beta metadata

We assume that the metadata you are using is based off the beta metadata file.

Download the latest Typewriter release.
Run Typewriter to generate a clean metadata file from your test metadata file. The output file will be cleanMetadata.xml.

.\typewriter.exe -v Info -m <TODO-SET-PATH-TO-YOUR-METADATA> -g Transform -t https://raw.githubusercontent.com/microsoftgraph/msgraph-metadata/master/transforms/csdl/preprocess_csdl.xsl

Generate .NET code files with Typewriter using the generated cleanMetadata.xml.

.\typewriter.exe -v Info -m D:\cleanMetadata.xml -o outputDirectory -g Files

Clone the Beta .NET SDK https://github.com/microsoftgraph/msgraph-beta-sdk-dotnet.
Replace the files under src/Microsoft.Graph/Requests/Generated and src/Microsoft.Graph/Models/Generated in the Beta .Net SDK with the generated files.
Build the Beta .Net SDK to validate that the generated files compile.

At this point you should have a valid SDK.

Using Vipr with this Writer

Build the solution in Visual Studio.
Go to the src\GraphODataTemplateWriter\bin\debug folder to find all compiled components.
In that folder, modify .config\TemplateWriterSettings.json to specify your template mapping see Template Writer Settings for more details.
Open a command prompt as administrator in the same folder and run Vipr.exe <path-or-url-to-metadata> --writer="GraphODataTemplateWriter". An example metadata file can be found in the root of this project.

By default, output source code will be put in a folder named "output" next to the Vipr executable.

Template Writer Settings

Available Languages

There are five languages to choose from at the moment. Java, ObjC, CSharp, TypeScript and Python. Specify which language you want to generate in the TargetLanguage setting.

Templates

You must specify a template directory under the TemplatesDirectory Settings. The directory can be a full path or relative to the running directory. The directory must contain a sub directory for each platform you want to generate code for. See the Templates directory for an example.

Template Mapping

You must specify the mapping of T4 Templates to specific SubProcessors for each platform you wish to generate. The TemplateMapping setting is a dictionary of languages and list of templates. Each template must specify :

Template, the name of the template without the extensions.
SubProcessor The SubProcessor for the template see SubProcessors
Type The type of template.
Name The format string for the name.

and optionally :

Include, a semicolon delimited list of objects to include in the subprocessor.
Exclude, a semicolon delimited list of objects to exclude from the subprocessor.
Ignore, a semicolon delimited list of objects to ignore from the subprocessor.
Matches, a semicolon delimited list of objects to include in the subprocessor.
FileCasing, UpperCamel, LowerCamel or Snake for the file casing for the specific file being created.

Note: Many of these optional parameters were used before Vipr had full support for annotations; now that annotations have been added to Vipr usage of these parameters should be limited to legacy scenarios

Example :

{ "Template": "EntityCollectionPage", "SubProcessor": "NavigationCollectionProperty", "Type": "Request", "Name": "<Class><Property>CollectionPage", "Matches" : "includeThisType", "Exclude" : "ExcludedTypeName;OtherExcludedTypeName" }

It is important to understand that subprocessors are mapped to methods that query the OdcmModel and return a set of OData objects. This mapping is maintained in TemplateProcess.InitializeSubprocessor(). The language specific mappings exist in the config directory. Each OData object returned by the subprocessor is applied to the mapped template which results in a code file output per each OData object.

In the above example, the objects in result set of the NavigationCollectionProperty subprocessor will each be applied to the EntityCollectionPage template. Each result will be a code file for each object returned by the NavigationCollectionProperty subprocessor.

SubProcessors

The SubProcessors determine what type of OData object will be passed into the template generating the code file.

CollectionMethod All Methods that are of type Collections
CollectionProperty Properties that are of type collection
CollectionReferenceProperty All Navigation Properties that are of type Collection which are used in Non-Containment Collections
ComplexType All Complex types
EntityContainer The EntityContainer
EntityReferenceType All Entity types which are used in Non-Containment Collections
EntityType All Entity types
EnumType All Enumerable types
MediaEntityType All Media Entity types
Method All Actions and Functions
MethodWithBody All Methods and Functions that send a body within the http request
NavigationCollectionProperty All Navigation Properties that are of type Collection
NonCollectionMethod All Methods and Functions that do not return a collection
Other The entire model.
Property All Properties types
StreamProperty All Properties types that return a Streams

Types

The type of template.

Request A template that will make a request
Model A model
Shared A template that will not output any code but is included by other templates
Client The template used to create the Client object
Other Any other type

Template Name

To set the name of the template using the Name format string. You can insert <Class>, <Property>, <Method>, and <Container> the values will be replaced by the names of the corresponding object. If you insert an item that doesn't exist it will be replaced with an empty string. Note: You can also set the template name from inside the template by : host.SetTemplateName("foo");

Template Editing

The solution contains a non-building project to host the actual T4 templates and make browsing/editing them easier. New template files will be automatically discovered by this project.

Includes/Excludes

There may be specific times when you want to exclude or only process certain objects from the SubProcessor. To Do this you can either set a semicolon delimited list of objects you wanted to include : Include : foo;bar. This will only process objects whose names are foo or bar. The opposite of this is the exclude setting where the SubProcessor will include all objects except for those whose names are in the exclude list, exclude and include can not be used together.

Ignore/Matches

When you can't use the name of an object to include or exclude you can use the long description element on any object. Insert a long description with a semicolon delimited list of strings like : foo;bar;baz. If you add a "Matches" : "foo;baz" only objects who contain foo and baz in their long description will be processed. The opposite is true for Ignore.

Note: You can also check in a template by odcjObject.LongDescriptionContains("foo");

Note: Includes/Excludes and Ignore/Matches were used before Vipr had full support for annotations; now that annotations have been added to Vipr usage of these parameters should be limited to legacy scenarios

Building against Graph Metadata

There are currently several steps we take to form the metadata into one that will successfully generate SDKs in the shape we expect:

Remove capability annotations (see #132)

Add navigation annotation to thumbnail

<Annotation String="navigable" Term="Org.OData.Core.V1.LongDescription"/>

Remove HasStream properties from onenotePage and onenoteEntityBaseModel
Add ContainsTarget="true" to navigation properties that do not have a corresponding EntitySet. This currently applies to navigation properties that contain plannerBucket, plannerTask, plannerPlan, and plannerDelta.
Add long descriptions to types and properties from docs

In order to build against metadata other than that stored in the metadata directory, you will need to perform the first four on this list.

Contributing

Before we can accept your pull request, you'll need to electronically complete Microsoft's Contributor License Agreement. If you've done this for other Microsoft projects, then you're already covered.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Why a CLA? (from the FSF)

MaggieKimani1/MSGraph-SDK-Code-Generator