LiterateCS needs to be able to self-weave

[john-burns-arundo]

[johnarleyburns@Johns-MacBook-Pro LiterateCS]$ dotnet run --project=LiterateCS -t -o tmp -s LiterateCS.sln **.cs **.md Using launch settings from LiterateCS/Properties/launchSettings.json... Fatal error in document generation: Macro 'Base class' already exists. at LiterateProgramming.Macro.Add(String name, BlockList start, BlockList end) in /Users/johnarleyburns/src/LiterateCS/LiterateCS/Macro.cs:line 76 at LiterateProgramming.BlockBuilder.VisitRegionDirectiveTrivia(RegionDirectiveTriviaSyntax node) in /Users/johnarleyburns/src/LiterateCS/LiterateCS/BlockBuilder.cs:line 387 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.Visit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 35 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.VisitTrivia(SyntaxTrivia trivia) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 104 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.VisitLeadingTrivia(SyntaxToken token) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 82 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.VisitToken(SyntaxToken token) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 72 at LiterateProgramming.BlockBuilder.VisitToken(SyntaxToken token) in /Users/johnarleyburns/src/LiterateCS/LiterateCS/BlockBuilder.cs:line 298 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.DefaultVisit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 64 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.Visit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 35 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.DefaultVisit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 54 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.Visit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 35 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.DefaultVisit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 54 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.Visit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 35 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.DefaultVisit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 54 at Microsoft.CodeAnalysis.CSharp.CSharpSyntaxWalker.Visit(SyntaxNode node) in /_/src/Compilers/CSharp/Portable/Syntax/CSharpSyntaxWalker.cs:line 35 at LiterateProgramming.BlockBuilder.CreateForTree(SyntaxTree tree) in /Users/johnarleyburns/src/LiterateCS/LiterateCS/BlockBuilder.cs:line 200 at LiterateProgramming.BlockBuilder.FromDocument(Document document) in /Users/johnarleyburns/src/LiterateCS/LiterateCS/BlockBuilder.cs:line 117 at LiterateProgramming.Weaver.BlockListFromDocument(Document document) in /Users/johnarleyburns/src/LiterateCS/LiterateCS/Weaver.cs:line 257 at LiterateProgramming.MdWeaver.WeaveFromCSharpDocument(SplitPath codeFile, Document document) in /Users/johnarleyburns/src/LiterateCS/LiterateCS/MdWeaver.cs:line 70 at LiterateProgramming.MdWeaver.GenerateFromSolution() in /Users/johnarleyburns/src/LiterateCS/LiterateCS/MdWeaver.cs:line 43 at LiterateProgramming.Weaver.GenerateDocumentation() in /Users/johnarleyburns/src/LiterateCS/LiterateCS/Weaver.cs:line 83 at LiterateProgramming.Program.GenerateDocumentation(Options options) in /Users/johnarleyburns/src/LiterateCS/LiterateCS/Program.cs:line 52

[john-burns-arundo]

Also got this error using literatecs as a dotnet tool:

[johnarleyburns@Johns-MacBook-Pro LiterateCS]$ literatecs -t -o docs -s LiterateCS.sln **.cs **.md
Fatal error in document generation:
Macro 'Base class' already exists.
at LiterateProgramming.Macro.Add(String name, BlockList start, BlockList end) in C:\projects\literatecs\LiterateCS\Macro.cs:line 74

[johtela]

Hi,

Creating docs for LiterateCS is a bit finicky with regards to the glob filters. In short, you should not try to create documentation for the DefaultTheme project. It doesn't have any documentation inside it, and it contains some C# files generated from the T4 templates, which have code regions inside them.

The error is really ugly, but it tells what the problem is. If you give the -v flag as well, LiterateCS will output which file it is processing. This helps pinpointing the offending file and region. In this case there is a region called "Base class" inside two auto-generated files: DefaultPage.cs and LandingPage.cs.

LiterateCS treats code regions as macros that can be embedded in markdown files. But as mentioned in the documentation, this means that the region names must be unique.

If you use different input file patterns, the docs are created correctly (at least in my machine):

literatecs -tv -o docs -s LiterateCS.sln LiterateCS/*.cs LiterateCS.Theme/*.cs **.md

Note that I added the -v option to see the verbose output. Note also that this command line creates markdown output. If you want to generate HTML, use the -f html option.

[john-burns-arundo]

Using that command works and generates successfully. Thanks for the pointer to regions, that's helped me to eliminate the same problem in my own code. It might be good to change the error message to something like "Macro already exists - you must have unique region names solution-wide, duplicate region found: "

[john-burns-arundo]

Idea: make the default globs for a c# project the default options looking for the sln file, so you just run "literatecs" and it finds the sln file in the current directory and outputs to "docs" all the c# and markdown files in html format. Generally this is the dotnet tool methodology: do what makes most sense by default without having to specify on the command line.

[john-burns-arundo]

Question I'm not sure about from the docs, looking at ExtensionCord as an example for doing my own literate project: https://raw.githubusercontent.com/johtela/ExtensionCord/master/index.md I'm not sure if this file is autogenerated from the TOC.yml, or if I need to create my own index.md? Thanks, John

…

On Tue, Jul 17, 2018 at 12:06 PM, Tommi Johtela ***@***.***> wrote: Hi, Creating docs for LiterateCS is a bit finicky with regards to the glob filters. In short, you should not try to create documentation for the DefaultTheme project. It doesn't have any documentation inside it, and it contains some C# files generated from the T4 templates, which have code regions inside them. The error is really ugly, but it tells what the problem is. If you give the -v flag as well, LiterateCS will output which file it is processing. This helps pinpointing the offending file and region. In this case there is a region called "Base class" inside two auto-generated files: DefaultPage.cs and LandingPage.cs. LiterateCS treats code regions as macros that can be embedded in markdown files. But as mentioned in the documentation <https://johtela.github.io/LiterateCS/Introduction.html#inserting-code-regions-in-markdown>, this means that the region names must be unique. If you use different input file patterns, the docs are created correctly (at least in my machine): literatecs -tv -o docs -s LiterateCS.sln LiterateCS/*.cs LiterateCS.Theme/*.cs **.md Note that I added the -v option to see the verbose output. Note also that this command line creates markdown output. If you want to generate HTML, use the -f html option. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Am6qxJD-hf-DwgEpGnHDmNNXHcZnK8qlks5uHhmYgaJpZM4VRuH2> .

-- [image: arundo-primary-100.jpg] <https://www.arundo.com/> Houston | Oslo | Palo Alto [image: linkedin-3-xxl (1).png] <https://www.linkedin.com/company/arundo-analytics-inc-> [image: twitter-3-xxl (1).png] <http://www.twitter.com/arundoanalytics> [image: facebook-3-xxl (1).png] <http://www.facebook.com/arundoanalytics> John Burns Senior Software Engineer Arundo Analytics john.burns@arundo.com <dan.reid@arundo.com> +1 (713) 582-7183 This email is confidential and may be privileged. If you have received it in error, please notify us immediately and then delete it. Please do not copy it, disclose its contents or use it for any unintended purpose.

[johtela]

Answering to your previous comment first about not having to specify input file filters; that was actually the idea I used in the original LiterateProgramming project. It uses Microsoft.Build assemblies to parse the .csproj file and processes just files referred in the project. I was also looking into the option of using some metadata stored in the project file that you could set to indicate which files to process. I did not find a suitable attribute I could use, though.

However, with .NET core Microsoft introduced a new .csproj format which does not store references to source files anymore. I guess it uses the equivalent of **.cs pattern to find the source files in the project directory.

Since, I don't want to include LiterateCS as a MSBuild step (it takes way too long to run, so build times would multiply), the best option I can think of is to store the filters in some other config file. Maybe it could be inside the front matter in globals.yml, for example.

I'll have to think that a bit, but I agree with your suggestion. Simplifying the command line would be a good idea.

---

Regarding the index.md file, that has to be created manually. It is not auto-generated.

[john-burns-arundo]

Thanks, LiterateCS is a great tool and I'm using it now!

…

On Tue, Jul 17, 2018 at 12:56 PM, Tommi Johtela ***@***.***> wrote: Answering to your previous comment first about not having to specify input file filters; that was actually the idea I used in the original LiterateProgramming project. It uses Microsoft.Build assemblies to parse the .csproj file and processes just files referred in the project. I was also looking into the option of using some metadata stored in the project file that you could set to indicate which files to process. I did not find a suitable attribute I could use, though. However, with .NET core Microsoft introduced a new .csproj format <https://natemcmaster.com/blog/2017/03/09/vs2015-to-vs2017-upgrade/> which does not store references to source files anymore. I guess it uses the equivalent of **.cs pattern to find the source files in the project directory. Since, I don't want to include LiterateCS as a MSBuild step (it takes way too long to run, so build times would multiply), the best option I can think of is to store the filters in some other config file. Maybe it could be inside the front matter in globals.yml, for example. I'll have to think that a bit, but I agree with your suggestion. Simplifying the command line would be a good idea. ------------------------------ Regarding the index.md file, that has to be created manually. It is not auto-generated. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Am6qxOC0fD14tpPAzvDQlOtyvDoee01qks5uHiVLgaJpZM4VRuH2> .

-- [image: arundo-primary-100.jpg] <https://www.arundo.com/> Houston | Oslo | Palo Alto [image: linkedin-3-xxl (1).png] <https://www.linkedin.com/company/arundo-analytics-inc-> [image: twitter-3-xxl (1).png] <http://www.twitter.com/arundoanalytics> [image: facebook-3-xxl (1).png] <http://www.facebook.com/arundoanalytics> John Burns Senior Software Engineer Arundo Analytics john.burns@arundo.com <dan.reid@arundo.com> +1 (713) 582-7183 This email is confidential and may be privileged. If you have received it in error, please notify us immediately and then delete it. Please do not copy it, disclose its contents or use it for any unintended purpose.

[john-burns-arundo]

Another quick question, I notice when I define "index.md" at the top level it's converted into an index.html, but when I do this for subdirectories (being projects in the solution), no index.html file is generated. What's the correct way I should be handling documenting projects in the solution? Thanks, John

…

On Tue, Jul 17, 2018 at 1:00 PM, John Burns ***@***.***> wrote: Thanks, LiterateCS is a great tool and I'm using it now! On Tue, Jul 17, 2018 at 12:56 PM, Tommi Johtela ***@***.***> wrote: > Answering to your previous comment first about not having to specify > input file filters; that was actually the idea I used in the original > LiterateProgramming project. It uses Microsoft.Build assemblies to parse > the .csproj file and processes just files referred in the project. I was > also looking into the option of using some metadata stored in the project > file that you could set to indicate which files to process. I did not find > a suitable attribute I could use, though. > > However, with .NET core Microsoft introduced a new .csproj format > <https://natemcmaster.com/blog/2017/03/09/vs2015-to-vs2017-upgrade/> > which does not store references to source files anymore. I guess it uses > the equivalent of **.cs pattern to find the source files in the project > directory. > > Since, I don't want to include LiterateCS as a MSBuild step (it takes way > too long to run, so build times would multiply), the best option I can > think of is to store the filters in some other config file. Maybe it could > be inside the front matter in globals.yml, for example. > > I'll have to think that a bit, but I agree with your suggestion. > Simplifying the command line would be a good idea. > ------------------------------ > > Regarding the index.md file, that has to be created manually. It is not > auto-generated. > > — > You are receiving this because you modified the open/close state. > Reply to this email directly, view it on GitHub > <#2 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/Am6qxOC0fD14tpPAzvDQlOtyvDoee01qks5uHiVLgaJpZM4VRuH2> > . > -- [image: arundo-primary-100.jpg] <https://www.arundo.com/> Houston | Oslo | Palo Alto [image: linkedin-3-xxl (1).png] <https://www.linkedin.com/company/arundo-analytics-inc-> [image: twitter-3-xxl (1).png] <http://www.twitter.com/arundoanalytics> [image: facebook-3-xxl (1).png] <http://www.facebook.com/arundoanalytics> John Burns Senior Software Engineer Arundo Analytics ***@***.*** ***@***.***> +1 (713) 582-7183 This email is confidential and may be privileged. If you have received it in error, please notify us immediately and then delete it. Please do not copy it, disclose its contents or use it for any unintended purpose.

-- [image: arundo-primary-100.jpg] <https://www.arundo.com/> Houston | Oslo | Palo Alto [image: linkedin-3-xxl (1).png] <https://www.linkedin.com/company/arundo-analytics-inc-> [image: twitter-3-xxl (1).png] <http://www.twitter.com/arundoanalytics> [image: facebook-3-xxl (1).png] <http://www.facebook.com/arundoanalytics> John Burns Senior Software Engineer Arundo Analytics john.burns@arundo.com <dan.reid@arundo.com> +1 (713) 582-7183 This email is confidential and may be privileged. If you have received it in error, please notify us immediately and then delete it. Please do not copy it, disclose its contents or use it for any unintended purpose.

[johtela]

I just tested your scenario and realized that the way how markdown files are treated currently is a bit counter-intuitive.

To get the the index.md and other markdown files in the sub-folders processed , you need to add the --recursive flag (or its short version: -r ) to the command line.

When a solution file is used as an input, LiterateCS automatically processes all the C# files, no matter how deep they are in the folder structure. For markdown files, however, the scanning works in the same way as when a folder is used as an input. In that case, you have to specify the --recursive flag for LiterateCS to search for sub-folders.

Maybe it would be good idea to remove the -r flag altogether, and have the recursive mode on as default. In the end, the filters actually determine which files will be included and which not.

[john-burns-arundo]

Yeah it might make sense to remove the "**.cs **.md" command argument entirely, kinda like how "dotnet build" does everything it's supposed to by default, ideally "literatecs" would do everything by itself (find the *.sln, generate "docs", etc.)

…

On Thu, Jul 19, 2018 at 2:03 PM, Tommi Johtela ***@***.***> wrote: I just tested your scenario and realized that the way how markdown files are treated currently is a bit counter-intuitive. To get the the index.md and other markdown files in the sub-folders processed , you need to add the --recursive flag (or its short version: -r ) to the command line. When a solution file is used as an input, LiterateCS automatically processes all the C# files, no matter how deep they are in the folder structure. For markdown files, however, the scanning works in the same way as when a folder is used as an input. In that case, you have to specify the --recursive flag for LiterateCS to search for sub-folders. Maybe it would be good idea to remove the -r flag altogether, and have the recursive mode on as default. In the end, the filters actually determine which files will be included and which not. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Am6qxOT7lg4ppZ1r6eOJymXTWYJ-pUT5ks5uINf3gaJpZM4VRuH2> .

-- [image: arundo-primary-100.jpg] <https://www.arundo.com/> Houston | Oslo | Palo Alto [image: linkedin-3-xxl (1).png] <https://www.linkedin.com/company/arundo-analytics-inc-> [image: twitter-3-xxl (1).png] <http://www.twitter.com/arundoanalytics> [image: facebook-3-xxl (1).png] <http://www.facebook.com/arundoanalytics> John Burns Senior Software Engineer Arundo Analytics john.burns@arundo.com <dan.reid@arundo.com> +1 (713) 582-7183 This email is confidential and may be privileged. If you have received it in error, please notify us immediately and then delete it. Please do not copy it, disclose its contents or use it for any unintended purpose.