Error running build_docs in Linux using local repos
justincr-elastic opened this issue · 4 comments
I am trying to run this example from https://github.com/elastic/docs/blob/master/README.asciidoc. I am getting an error, so I am looking for help to workaround the error.
For a local repo
When you are making changes to documentation in a locally checked out repository, and you want to check that they are building correctly, use build_docs with the --doc parameter to generate the HTML version of the docs:
cd path/to/your/repo
~/path/to/docs/repo/build_docs --doc /full/path/to/index.asciidoc
I tried running that, but it fails. The command and error are below.
Note, I am running in Debian (Buster). My local repos are on Windows, so I used Vagrant and Virtual Box to mount them into a Debian VM.
config.vm.synced_folder 'C:/GitHub/elasticsearch', '/docs/elasticsearch'
config.vm.synced_folder 'C:/GitHub/docs_build', '/docs_build'
The VM sees the folders OK.
root@builddocs:~# ls -l /doc/elasticsearch/|head
total 246
-rwxrwxrwx 1 vagrant vagrant 10454 Feb 28 19:46 BUILDING.md
-rwxrwxrwx 1 vagrant vagrant 172 Feb 28 19:46 CHANGELOG.md
-rwxrwxrwx 1 vagrant vagrant 47509 Mar 9 14:16 CONTRIBUTING.md
-rwxrwxrwx 1 vagrant vagrant 553 Feb 28 19:46 LICENSE.txt
-rwxrwxrwx 1 vagrant vagrant 155 Feb 28 19:46 NOTICE.txt
-rwxrwxrwx 1 vagrant vagrant 2800 Feb 28 19:46 README.asciidoc
-rwxrwxrwx 1 vagrant vagrant 35931 Mar 9 14:16 TESTING.asciidoc
-rwxrwxrwx 1 vagrant vagrant 15782 Feb 28 19:46 Vagrantfile
drwxrwxrwx 1 vagrant vagrant 4096 Mar 24 13:45 benchmarks
root@builddocs:~# ls -l /docs_build/|head
total 647
-rwxrwxrwx 1 vagrant vagrant 4730 Mar 22 18:46 Dockerfile
-rwxrwxrwx 1 vagrant vagrant 51 Mar 22 18:46 Dockerfile.dockerignore
-rwxrwxrwx 1 vagrant vagrant 765 Mar 22 18:46 Gemfile
-rwxrwxrwx 1 vagrant vagrant 1373 Mar 22 18:46 Gemfile.lock
-rwxrwxrwx 1 vagrant vagrant 873 Mar 22 18:46 Makefile
-rwxrwxrwx 1 vagrant vagrant 53770 Mar 22 18:46 README.asciidoc
drwxrwxrwx 1 vagrant vagrant 4096 Mar 22 18:46 air_gapped
-rwxrwxrwx 1 vagrant vagrant 25757 Mar 23 19:17 build_docs
-rwxrwxrwx 1 vagrant vagrant 38239 Mar 23 19:16 build_docs.pl
I also mounted a folder to hold the build_docs output directories. That is to let me view the output in a browser on Windows.
config.vm.synced_folder 'C:/Vagrant/build_docs', '/build_docs'
Here is the example I tried, with the error.
root@builddocs:~# y# cd /doc/elasticsearch/x-pack/docs/en/security
root@builddocs:/doc/elasticsearch/x-pack/docs/en/security# python3 /docs_build/build_docs --doc /doc/elasticsearch/x-pack/docs/en/security/index.asciidoc --out /build_docs/mybuild1
INFO:build_docs:Building HTML from /doc/elasticsearch/x-pack/docs/en/security/index.asciidoc
INFO:build_docs:Couldn't find an Elastic remote for /doc/elasticsearch. Generating edit links targeting the first remote instead.
INFO:build_docs:Guessed toplevel=[/doc/elasticsearch] remote=[git@github.com:justincr-elastic/elasticsearch.git] branch=[master] repo=[elasticsearch]
INFO:build_docs:
INFO:build_docs:asciidoctor: ERROR: configuring-stack-security.asciidoc: line 91: include file not found: /doc/elasticsearch/x-pack/docs/en/security/{es-ref-dir}/setup/install/connect-clients.asciidoc
INFO:build_docs:asciidoctor: ERROR: configuring-stack-security.asciidoc: line 102: include file not found: /doc/elasticsearch/x-pack/docs/en/security/{es-ref-dir}/setup/install/security-files-reference.asciidoc
INFO:build_docs:asciidoctor: ERROR: partintro block can only be used when doctype is book and must be a child of a book part. Excluding block content.
INFO:build_docs:asciidoctor: WARNING: invalid reference: search
INFO:build_docs:asciidoctor: WARNING: authorization/overview.asciidoc: line 19: can't read image [security/authorization/images/authorization.png] at any of ["/docs_build/resources/asciidoc-8.6.8/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/auditing/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/authentication/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/authorization/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/ccs-clients-integrations/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/images/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/operator-privileges/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/reference/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/securing-communications/security/authorization/images/authorization.png", "/doc/elasticsearch/x-pack/docs/en/security/authorization/images/security/authorization/images/authorization.png"]
The file does exist, but it is in a different directory where build_docs does not see.
root@builddocs:/doc/elasticsearch/x-pack/docs/en/security# find /doc/elasticsearch/x-pack/docs/en/ -name authorization.png | xargs ls -l
-rwxrwxrwx 1 vagrant vagrant 184137 Feb 28 19:46 /doc/elasticsearch/x-pack/docs/en/security/authorization/images/authorization.png
How can I fix my build_docs command to get it to work?
Note, I ran into a number of undocumented assumptions for build_docs. I might be helpful to include these in https://github.com/elastic/docs/blob/master/README.asciidoc, because I had to work through these issues first.
- Elasticsearch repo must be located at /doc/elasticsearch?
- Docs repo must be located at /build_docs?
- Must setup ssh-agent for other build_docs options?
Found a workaround.
python3 /docs_build/build_docs --doc /doc/elasticsearch/docs/reference/index.asciidoc --resource=/doc/elasticsearch/x-pack/docs/ --out /build_docs/mybuild1
Actually this seems better to get closer to the page layout of the hosted docs.
python3 /docs_build/build_docs --doc /doc/elasticsearch/docs/reference/index.asciidoc --resource=/doc/elasticsearch/x-pack/docs/ --out /build_docs/mybuild1 --chunk 1
Glad you figured this out, @justincr-elastic .
* Elasticsearch repo must be located at /doc/elasticsearch? * Docs repo must be located at /build_docs?
Generally speaking, a lot of the code assumes that the docs repo and elasticsearch repo (and other repos used to build the docs) are siblings in the same parent directory, so mounting one at /docs_build and the other under /docs is likely to cause some issues. This is certainly what the aliases assume.
We use a lot of relative paths to handle cross-repo resources and includes. There have been some discussions about using {docs-root}/subpath rather than paths like ../../docs/subpath, which would solve a lot of these issues, but we haven't
* Must setup ssh-agent for other build_docs options?
ssh-agent is needed when you're using --all to build all the docs within the Docker container. We clone the repos into a .repos directory rather than relying them all being checked out and mounted into the Docker container. I don't know anything about how ssh-agent works on Windows, though, so if you're able to help document this for others, that would be great!