jekyll/jekyll

Avoid mk_dir permission error in jekyll 4 on azure build pipeline/docker

fabsenet opened this issue · 15 comments

fabsenet commented

I solved my issue but guessed someone may search for it or you may change something anyway for jekyll4, I dont know?!

I upgraded my blog (src) to jekyll 4 pre alpha 1 and run into issues in the build pipeline of azure devops:

/usr/local/lib/ruby/2.6.0/fileutils.rb:239:in `mkdir': Permission denied @ dir_s_mkdir - /srv/jekyll/.jekyll-cache (Errno::EACCES)

It uses the jekyll/builder docker image to build the actual site. Long story short, adding an empty directory ".jekyll-cache" to my blog source solved the issue.

For reference, here is my currently working azure-pipelines.yml

not sure, what could be done or what the real issue is here?!

My Environment

Software Version(s)
Operating System docker: jekyll/builder
jekyll 4.0.pre.alpha1
github-pages no

my gemfile.lock

mattr- commented

If the cache doesn't exist, we'll try to create it, but rather than continuing without cache if we can't create the cache, we just kill the program. That's not ideal and should be fixed. Thanks for reporting this @fabsenet!

ashmaroli commented

@fabsenet Will you be able to run the build with a --trace switch? The output from that will point us to the exact method that's causing the issue (will help in confirming our hunch)
Thanks.

fabsenet commented

Easy as I did this to get this errorstring in the first place :-)

4_Run Jekyll (STAGING, next).txt

Trace log 
Configuration file: /srv/jekyll/_config.yml
Configuration file: /srv/jekyll/_config_staging.yml
/usr/local/lib/ruby/2.6.0/fileutils.rb:239:in `mkdir': Permission denied @ dir_s_mkdir - /srv/jekyll/.jekyll-cache (Errno::EACCES)
  from /usr/local/lib/ruby/2.6.0/fileutils.rb:239:in `fu_mkdir'
  from /usr/local/lib/ruby/2.6.0/fileutils.rb:217:in `block (2 levels) in mkdir_p'
  from /usr/local/lib/ruby/2.6.0/fileutils.rb:215:in `reverse_each'
  from /usr/local/lib/ruby/2.6.0/fileutils.rb:215:in `block in mkdir_p'
  from /usr/local/lib/ruby/2.6.0/fileutils.rb:200:in `each'
  from /usr/local/lib/ruby/2.6.0/fileutils.rb:200:in `mkdir_p'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/cache.rb:177:in `dump'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/cache.rb:68:in `[]='
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/cache.rb:120:in `clear_if_config_changed'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/site.rb:113:in `reset'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/site.rb:33:in `initialize'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/commands/build.rb:30:in `new'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/commands/build.rb:30:in `process'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/command.rb:89:in `block in process_with_graceful_fail'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/command.rb:89:in `each'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/command.rb:89:in `process_with_graceful_fail'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/lib/jekyll/commands/build.rb:18:in `block (2 levels) in init_with_program'
  from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `block in execute'
  from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `each'
  from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `execute'
  from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary/program.rb:42:in `go'
  from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary.rb:19:in `program'
  from /usr/local/bundle/gems/jekyll-4.0.0.pre.alpha1/exe/jekyll:15:in `'
  from /usr/local/bundle/bin/jekyll:29:in `load'
  from /usr/local/bundle/bin/jekyll:29:in `'

After investing like a day or two, azure devops is quite good. it saved this log for example!

ashmaroli commented

@fabsenet Thank you for the log. Now all this needs is determining why executing
mkdir_p /srv/jekyll/.jekyll-cache has an issue but mkdir_p /srv/jekyll/_site (destination directory is also auto-generated likewise) does not raise an error once you manually added a .jekyll-cache folder..

fabsenet commented

I think it is related to the environment jekyll is running in.

the blog-source directory and the _site directory are mapped to some volume, the cache folder is not:
https://github.com/fabsenet/blog_de/blob/4a78156250d0a7f92481a4d7a444a59942754613/azure-pipelines.yml#L30-L32

maybe jekyll has no permissions in the source folder to create directories? I dont know. It depends on who you would ask:

  • jekyll is errorfree
  • jekyll/builder is errorfree
  • microsoft would say, their azure-pipelines are errorfree as well (or probably they would not answer at all)

but the error is there.

What would really help me is to change the jekyll clean command to not delete the /.jekyll-cache/.gitignore file!

ashmaroli commented

and the _site directory are mapped to some volume

That would only be possible if _site exists beforehand.

to change the jekyll clean command to not delete the /.jekyll-cache/.gitignore file!

Hmm.. perhaps, we can introduce a --skip-cache-dir switch..
/cc @pathawks Any thoughts?

jekyllbot commented

This issue has been automatically marked as stale because it has not been commented on for at least two months.

The resources of the Jekyll team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the latest 3.x-stable or master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, please consider building it first as a plugin. Jekyll 3 introduced hooks which provide convenient access points throughout the Jekyll build pipeline whereby most needs can be fulfilled. If this is something that cannot be built as a plugin, then please provide more information about why in order to keep this issue open.

This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.

araphiel commented

Just received this error using jekyll/builder:latest via Github Actions CI.

Running mkdir .jekyll-cache _site before the docker jekyll command resolved this issue for me.

  • 👍4
  • ❤️3
a-weinert commented

Dear fabsenet ,
using jekyll/jekyll in a SVN (I know we're on GIT here) post commit hook on the "mkdir /srv/jekyll/.jekyll-cache (Errno::EACCES)"-bug did cost a lot of time with no remedy in sight.
You probably saved the project.
Thanks a lot
Albrecht

  • ❤️1
briandrennan commented

I ran into this issue as well. I resolved it using the following YAML for the Azure Pipelines task:

- task: Docker@0
  displayName: 'Run Jekyll'
  inputs:
    containerregistrytype: 'Container Registry'
    action: 'Run an image'
    imageName: 'jekyll/builder:4.1.0'
    volumes: |
      $(build.sourcesDirectory):/srv/jekyll
      $(build.sourcesDirectory)/.jekyll-cache:/srv/jekyll/.jekyll-cache
      $(build.binariesDirectory):/srv/jekyll/_site
    containerCommand: 'jekyll build --future --trace'
    detached: false

I'm still new to using Docker, and am beyond novice with Jekyll, but the .jekyll-cache entry (the second in the volumes list) fixed the issue for my build.

KeepAliveIT commented

Hi,

We had this issue in our Travis CI build today. We have looked into the solutions above and can confirm that creating the directories ahead of time before running the docker command will result in no error being outputted.

Many thanks,
Keep-Alive I.T

t-kozak commented

I have encountered the same problem with Google Cloud Build. This is because of the use of the /srv/jekyll volume inside the Dockerfile here: https://github.com/envygeeks/jekyll-docker/blob/master/repos/jekyll/Dockerfile.

I solved this problem by not using the default jekyll docker images and just falling back to generic ruby:

cloudbuild.yaml:

steps:
  - name: 'ruby:latest'
    args: ['bash', '-c', './build.sh'] 
      
  - name: "gcr.io/google.com/cloudsdktool/cloud-sdk"
    args: ["gsutil", "rsync", "-r", "_site", "gs://BUCKET"]

build.sh:

#!/bin/bash
bundle install
bundle exec jekyll build
t-pawelb commented

The issue seems to be still open.

Just in case anyone looking for a dirty workaround, here is my docker-compose.yml with binding current directory to the faulty /srv/jekyll :

version: '3.4'

services:
  jeckyll:
    image: jekyll/jekyll
    command: jekyll build
    volumes:
      - type: bind
        source: .
        target: /srv/jekyll
  • 👍1
matthew-a-thomas commented

This isn't just an Azure issue. Also happens with GitHub Actions. For example, the below action fails with the above error if I leave out the indicated line:

name: deploy
on:
  push:
    branches:
      - master
  workflow_dispatch:

jobs:
  build:
    name: deploy
    runs-on: ubuntu-latest
    container:
      image: jekyll/jekyll:4
    steps:
      - name: Checkout ungenerated site
        uses: actions/checkout@v2
      - name: Checkout generated site to '_site' directory
        uses: actions/checkout@v2
        with:
          ref: 'gh-pages'
          path: '_site'
      - name: Install dependencies
        run: |
          bundle install
### "Build website" fails if you remove the `mkdir .jekyll-cache` ###
      - name: Build website
        run: |
          mkdir .jekyll-cache
          jekyll build
#####################################################################
      - name: Publish
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITHUB_REPOSITORY: ${{ secrets.GITHUB_REPOSITORY }}
          GITHUB_ACTOR: ${{ secrets.GITHUB_ACTOR }}
        run: |
          cd _site
          touch .nojekyll
          git config user.name "${GITHUB_ACTOR}"
          git config user.email "${GITHUB_ACTOR}@users.noreply.github.com"
          git add -A
          git status
          git commit -m "Github Actions"
          git push
leviem1 commented

I was accidentally mounting a volume within a already mounted volume. Removing that fixed it for me

EDIT: This ended up appearing again the next time I touched my workflow and my previous fix was already applied. I ended up solving it the second time by setting JEKYLL_UID=1001 and JEKYLL_GID=1001