Support for several source dirs

[antoniogamiz]

Problem

At this moment, we can only specify a single source directory with :source. My idea convert :source to an array.

Why do I want this? Perl6::Documentable supports pods inside files containing code (.p6, .pm6, etc), so in order to use them, we would need to specify `:source("lib"), to gather all documentation in our module.

In that case we face two problems:

• Pod::To::Cached does not support those extensions (I suppose that's easy to fix)
• We have already used :source. What if we want to create an additional directory containing tutorials in pod6? We would need to use :source("<module-root-directory>").

The last fix is a hack I don't like because you can have pods in your module root directory, which you do not want to be used by Perl6::Documentable.

Suggestion

• Add support for .p6 and .pm6 files.
• Add support for :@source.

This is only my idea, please let me know your opinion.

[finanalyst]

This seems to be two issues, not one.

1. I am looking in more detail at sources. More later

2. Support for other extensions is already possible in Pod::To::Cached. Set line 57 @!extensions to include p6 and pm.

There may be problems though not in Pod::To::Cached. I think when I was developing Pod::To::Cached I included p6 and pm, but ran into problems elsewhere.

Or perhaps, I was focussing on the documentation suite, which does not contain p6 or pm files.

[finanalyst]

Ok, I just included p6 and pm as possible extensions. The tests all passed.

However, my test suite does not contain p6 or pm files.

It would be useful to add tests that include p6 and pm files.

So part 2 of this issue seems to be easily resolved, but I anticipate problems somewhere done the line. Updating Pod::To::Cached

[finanalyst]

Regarding multiple sources.

• The original use case was for perl6 documentation, which is all contained with a single directory tree, hence it was sufficient to have a scalar $.source
• Having more than one sources, viz., @.sources is understandable but I anticipate some problems that I want to discuss.
• First lets consider where we want the documentation collection to go.
• Antonio suggests that he wants to have one source tree for perl6 documentation and another for tutorials. However, tutorials for perl6 are already included within the pod source tree.
• Antonio, though, is thinking about including tutorials related to his module, rather than perl6. This makes sense because tutorials (and documentation) for a module should be different from the tutorials (and other documentation) for the language.
• Lets extend this. All modules will have documentation and could all provide a source to be included in the documentation collection
• However, what happens when there are changes to the directory structures of other (non perl6 language) modules?
• I am concerned that this will introduce many weaknesses down the line.

I am reviewing the code to see how to change it, but I would appreciate some help on the test suite.

[antoniogamiz]

Mmm, you're right, this feature would introduce a lot of weird cases that, probably, we cannot cover. Maybe we should simplify this and only let the use of a "docs" folder and "lib". Or maybe only the "docs" directory.

I do not understand the last point you have made, can you clarify, please?

Of course, I will help with the test suite.

[finanalyst]

I've been looking at the code, and the change to multiple sources has numerous problems. It may even require a change in design.

Source is not just a directory, it specifies a directory tree, under which documentation is contained. This assumption is used in the naming convention for the cache, in that the full path to each document is truncated using the length of the path for the source.

The aim of this naming sequence is to make it easier to freeze the cache by making the names relative to the cache, not to the full path.

Changing from one source tree to many source trees means that every document will need to be named with the full file path of the document.

• This is not in reality a big problem because as far as the cache is concerned, the path+file name is only a name, it is not used to access the pod. The pod is accessed using the cache key.
• However, the full file name will be stored in the yaml index. Suppose that a developer created a documentation cache and froze it. If the full file path is used, that would contain information about the developer's system. So allowing for multiple arbitrary sources might create a security flaw.

[finanalyst]

@antoniogamiz

1. Yes, I think that creating a separate tree, eg. docs, which in turn contains a sub-directory, eg., doc that is the tree to the Perl 6 documentation, and also a sub-directory eg my-tutorials, which contains the other documentation you want to include, is the best solution.

• Given that the sub-directories (under *nix) can be references or aliases to directories in other places, I think that this is by far the best solution. Then Pod::To::Cached collects all the files within that tree which could contain POD6 data (viz., .pod .pod6 .p6 .pm), and stores the names in the cache index.

2. You asked for clarification about 'last point'. This reference was ambiguous. Which point? About extending to multiple modules?

[antoniogamiz]

@finanalyst

1. I did not realize there were so many problems. Thanks a lot for the deep analysis. Hence, I will specify that all documentation should be present under only one directory, which can be specified by the user. Perl6::Documentable is independent of the tree directory (since the use of in-file metadata was supported).

2. As for the clarification request, I was referring to the second-to-last point (non perl6 languages). Sorry for the mistake.

[finanalyst]

@antoniogamiz
Regarding clarification (non perl6 language), there is high possibility of misunderstanding - my bad.

Note that I said 'language' not 'languages'! I meant documentation for modules as opposed to documentation of the perl6 language itself. So when you refer to documentation of Perl6::Documentable, this is a module but not a part of the Perl 6 language.

In principle, your extension idea is important. In the future, it will be important for a developer to know about functions that are supplied by all the modules installed on a system, not just the functions that are available from the Perl 6 language. We are focussing on the Perl 6 language documentation at the present, but we should keep in mind that extensions will be needed for all modules, and perhaps on line, for all modules in the ecosystem (???)

[antoniogamiz]

Ah I see. You're right, @JJ wants that Perl6::Documentable can be used for other modules too (in addition to the official documentation). It could be used by Cro, or Perl6::Documentable itself.

Nonetheless, I think that support to all modules installed in a system is a bit out of the scope of Perl6::Documentable. In my opinion, that feature fits with 'p6doc'.

[JJ]

Could this be fixed with several copies of Pod::To::Cached? Or of Perl6::Documentable?

[JJ]

@antoniogamiz Perl6::Documentable is a high-level API for all kind of documentation. You might have different instances for different root dirs or the same instance, it's not really a big difference. p6doc should always rely on Perl6::Documentable, not go directly to the cache.

[antoniogamiz]

Ah, that's a good idea. I will do it, thanks for the point.

[finanalyst]

we are agreed, I think. This suggestion has a better work around. But testing required for multiple instances.