Generate edoc for multiple apps

[drozzy]

If I have an OTP layout like this:

foo/

• apps/
  • bar1/
    • src/
  • bar2
    • src/

I want to generate edoc for all apps with rebar3 edoc.

If I run:

rebar3 edoc

what I get is separate .html files in bar1/doc and bar2/doc, and none of the links between them work (i.e. export types are not linked correctly).

Any thoughts?

Ref: http://stackoverflow.com/questions/39043889/rebar3-generate-edoc-for-multiple-apps/39134151

[drozzy]

P.S.: Investigating the possibility of writing a plugin for erldocs to do the same thing: erldocs/erldocs#54

[ferd]

I actually never had to play with that specific use case where documentation cross-links. If you have a repository for it, it would be useful. I'm guessing there's some specific configuration that can be given to edoc (the Erlang app) to help with this, but would like to experiment with a reasonable set up.

We can't necessarily change how the provider runs without breaking compatibility, but if what you need cannot be supported otherwise, there's the possibility to add command line argument for it, but we'll need to see.

[drozzy]

I can't share the one from work, but I can create a dummy one.

On a related question, how do you organize large OTP projects?
I assumed that apps folder was actually for multiple apps (each created with rebar3 new app)

[drozzy]

Here is a sample repo:

https://github.com/drozzy/foo

If you navigate to any app's doc directory, e.g.:
https://github.com/drozzy/foo/tree/master/apps/foo/doc

and open index.html in your browser you'll see cross-links to other apps:

But they won't work if you click on them, since the docs for the app are in another app's directory. I.e. in this case in D:\Projects\foo\apps\bar1\doc

[ferd]

So if I put {edoc_opts, [{dir, "doc"}]} all the documentation generated will be at the top-level doc directory, which will work fine and make links work with one caveat: the index page will be busted and only show the latest library built.

The actual proper way to do it though, would be to sort all libraries in order of dependencies (the same way we do for compiling them) and then set the generated doc paths for all of them dynamically using {doc_path, ["/tmp/foo/apps/bar1/doc", "/tmp/foo/apps/bar2/doc", "/tmp/foo/apps/foo/doc"]}, and so on.

This command works, but the problem is that it will generate warnings when the generated doc paths do not exist and ignore the links so you currently need to run it twice and specify directories by hand.

---

In short I'd say that for the time being, {edoc_opts, [{dir, "doc"}]} is a workaround.

What would be good as a permanent fix is to add a mechanism by which we detect when there is more than 1 top-level app, and then do a topological sort of them. All apps without dependencies must be documented first, and their generated absolute path gets to be added to the config of subsequent runs. This will allow everything to work while silencing the edoc warnings and without requiring 2 runs.

This also sounds like a bit of a pain in the butt to do, but would be quite neat to have working out of the box.

[ferd]

Further details: the apps are already sorted fine if the dependencies between them are inside the .app.src file, meaning the topological sort is implicitly done already. We just need to add the bits that dynamically figures out paths. Should be an easy fix.

[ferd]

Here's a fix: #1308 let me know if it works and I'll start adding tests.

For this to get triggered, you must not specify the dir option (since it will move directories around). You can however specify doc_path, the auto-generated one will just be added to the list.

[drozzy]

Thanks for looking into this!

1. The fix with edoc_opts works satisfactory, with the problem you mentioned (the table-of-contents only shows one app)
2. I'm not really sure where to put the doc_path directives to try out the second solution.
3. Re fix #1308 - I would like to try it, but I'm not sure how, as I only know how to use the binary rebar3 you provide on http://www.rebar3.org/ (I'm on windows too, but have a centos7 box)

[ferd]

I've got tests added, pretty sure that the one at #1308 is gonna take care of it, you'll just need to add the apps you depend on in .app.src's application tuple so they are built in the right order (if you rely on their type, you probably do have an active dependency between them anyway)

[drozzy]

I just compiled #1308 and tested it. Here is what I found:

1. Cross linking works now without {edoc_opts, [{dir, "doc"}]}..
2. Cross liking still works with {edoc_opts, [{dir, "doc"}]}.. It puts all docs in the same directory.
3. The Table for Contents for both cases above is still only showing one app's modules.

Now, I am not sure if No.3 above is a limitation of edoc.

[ferd]

No.3 is a limitation I think. In this case, the edoc generation is done on a per-app basis, and so the total index is going to be per-app. I'm not quite sure how to make it generate a global index for everything it found since that appears to go past options there are. Even the regular compiling options when running without an app's context ask you to specify an application.

I don't know how the OTP libraries do it themselves, but as far as I know they don't use straight-up edoc since they generate PDFs and XML files and whatnot in between.

[drozzy]

In particular, it is the modules-frame.html file that is not adding all the *.html files.

[drozzy]

Thanks, I can probably hack something together from here on.

Seems like the solution might be:

1. Build the modules-frames.html dynamically based on all the filenames

I wonder if I can write a rebar3 plugin for that.

[drozzy]

What about subpackages option? Will that no accomplish what we need?

{subpackages, boolean()}
If the value is true, all subpackages of specified packages will also be included in the documentation. The default value is false. no_subpackages is an alias for {subpackages, false}.

Subpackage source files are found by recursively searching for source code files in subdirectories of the known source code root directories. (Also see the source_path option.) Directory names must begin with a lowercase letter and contain only alphanumeric characters and underscore, or they will be ignored. (For example, a subdirectory named test-files will not be searched.)

[ferd]

Oh, you could try them, but I thought this referred to the old deprecated "packages" of old days, given the description of searching subdirectories recursively.

[drozzy]

So I hacked together a rebar3 plugin, that will rewrite the toc file (which is the "modules-frame.html") after running edoc.

Called it medoc (for "multiple" apps edoc):
https://github.com/drozzy/medoc