make clean permission errors
Closed this issue · 19 comments
Hi just started using this template, it's pretty awesome. Had to make some modifications for it work when using cgo but I'm not sure if that is causing this issue. I think it has to do with how go downloads modules.
Running make clean results permission errors when make is trying rm -rf .go/pkg/mod.
example:
rm: cannot remove '.go/pkg/mod/golang.org/x/crypto@v0.0.0-20200622213623-75b288015ac9/sha3/testdata/keccakKats.json.deflate': Permission denied
Not sure if others are running into this problem as well
Running Docker version 20.10.8, build 3967b7d
go v1.17
The solution I came up with is to have a go-clean step
go-clean: # @HELP runs go clean -modcache to remove the modules under pkg/mod
go-clean: $(BUILD_DIRS)
@echo "# launching a shell in the containerized build environment to run go clean -modcache"
@docker run \
-ti \
--rm \
-u $$(id -u):$$(id -g) \
-v $$(pwd):/src \
-w /src \
-v $$(pwd)/.go/pkg:/go/pkg:z \
$(BUILD_IMAGE) \
/bin/sh -c 'go clean -modcache'Not sure if you have a better solution but I wasn't able to find anything about it. There is a thread about the permissions here: golang/go#27161
How about this?
diff --git a/Makefile b/Makefile
index b99045e..bc8fb50 100644
--- a/Makefile
+++ b/Makefile
@@ -268,6 +268,7 @@ container-clean:
rm -rf .container-* .dockerfile-* .push-* $(LICENSES)
bin-clean:
+ chmod -R u+w .go
rm -rf .go bin
help: # @HELP prints this message
It does not resolve the issue unfortunately. I'm on Ubuntu 20 LTS as well.
➜ hft-system git:(main) ✗ make bin-clean
chmod -R -u+w .go
chmod: cannot read directory '.go': Permission denied
make: *** [Makefile:301: bin-clean] Error 1
➜ hft-system git:(main) ✗ Actually scratch that, it did delete the directory but the permission issue remained. Going to look into it further later.
I have a more complete fix
check HEAD now
Close but no cigar
➜ go-build-template git:(master) go get github.com/google/go-licenses
go: downloading golang.org/x/crypto v0.0.0-20191117063200-497ca9f6d64f
go: downloading golang.org/x/sys v0.0.0-20191119060738-e882bf8e40c2
go get: added github.com/google/go-licenses v0.0.0-20210816172045-3099c18c36e1
➜ go-build-template git:(master) ✗ make build && make container
# building for linux/amd64
-ne binary: bin/linux_amd64/myapp-1
-ne binary: bin/linux_amd64/myapp-2
# building for linux/amd64
-ne binary: bin/linux_amd64/myapp-1
(cached)
/bin/sh: 1: pushd: not found
/bin/sh: 3: popd: not found
make: *** [Makefile:178: .licenses] Error 127
➜ go-build-template git:(master) ✗
Not testing this is my current project because I have the license step disable at the moment.
This works perfectly fine for me for now
clean: # @HELP removes built binaries and temporary files
clean: container-clean go-clean bin-clean
go-clean: # @HELP runs go clean -modcache to remove the modules under pkg/mod
go-clean: $(BUILD_DIRS)
@echo "# launching a shell in the containerized build environment"
@docker run \
-ti \
--rm \
-u $$(id -u):$$(id -g) \
-v $$(pwd):/src \
-w /src \
-v $$(pwd)/.go/pkg:/go/pkg \
$(BUILD_IMAGE) \
/bin/sh -c 'go clean -modcache'Don't know why they had to make cleaning modules so difficult.
What OS is this? the -ne suggests it is using a weird toolchain. Mac ?
I'm developing on Ubuntu 20.04.3 LTS. I'm working with apache kafka and am using Confluent Kafka Go which requires me to set CG_ENABLED=1. I was able to get my app using tetafro/golang-gcc but decided to just switch to the official image for just building. This combo works if you're looking to expand the build configuration to support Go projects needing C/C++ libraries.
BASEIMAGE ?= gcr.io/distroless/base-debian10
DEBUG_IMAGE ?= gcr.io/distroless/base-debian10:debug
BUILD_IMAGE ?= golang:1.17-busterIt's saying you have a weird shell - no echo -ne, no pushd, and no popd ? Is it bash or something else?
Oh it might be complaining because I run zsh / tmux haha
try under bash - I can make make force bash if I need. Probably a good idea
Strange, still getting it.
dank@ifrit:~/git/github/go-build-template$ rm -rf .go
dank@ifrit:~/git/github/go-build-template$ make clean
rm -rf .container-* .dockerfile-* .push-* .licenses
rm -rf .go bin
dank@ifrit:~/git/github/go-build-template$ make build && make container
# building for linux/amd64
-ne binary: bin/linux_amd64/myapp-1
-ne binary: bin/linux_amd64/myapp-2
# building for linux/amd64
-ne binary: bin/linux_amd64/myapp-1
(cached)
/bin/sh: 1: pushd: not found
no required module provides package github.com/google/go-licenses; to add it:
go get github.com/google/go-licenses
/bin/sh: 3: popd: not found
make: *** [Makefile:178: .licenses] Error 127
dank@ifrit:~/git/github/go-build-template$ Add SHELL := /usr/bin/env bash near the top - just to see
That worked!
➜ go-build-template git:(master) ✗ make build && make container
# building for linux/amd64
binary: bin/linux_amd64/myapp-1
binary: bin/linux_amd64/myapp-2
# building for linux/amd64
binary: bin/linux_amd64/myapp-1 (cached)
Sending build context to Docker daemon 49.98MB
Step 1/6 : FROM gcr.io/distroless/static
---> e0851a4aa136
Step 2/6 : COPY bin/linux_amd64/myapp-1 /myapp-1
---> 00453cab3d31
Step 3/6 : COPY .licenses/ /LICENSES/
---> ccc7dd6719fe
Step 4/6 : USER 65535:65535
---> Running in 2d02a2253f81
Removing intermediate container 2d02a2253f81
---> 5d29b89e8cc3
Step 5/6 : ENV HOME /
---> Running in 17d108d7445a
Removing intermediate container 17d108d7445a
---> 5e83225eaf32
Step 6/6 : ENTRYPOINT ["/myapp-1"]
---> Running in 7184c72ab18c
Removing intermediate container 7184c72ab18c
---> 4e3363da26bb
Successfully built 4e3363da26bb
Successfully tagged example.com/myapp-1:6d246ce-dirty__linux_amd64
binary: bin/linux_amd64/myapp-2 (cached)
Sending build context to Docker daemon 49.98MB
Step 1/6 : FROM gcr.io/distroless/static
---> e0851a4aa136
Step 2/6 : COPY bin/linux_amd64/myapp-2 /myapp-2
---> 46733bb556f0
Step 3/6 : COPY .licenses/ /LICENSES/
---> 9cc1ff934406
Step 4/6 : USER 65535:65535
---> Running in 848e4181afe2
Removing intermediate container 848e4181afe2
---> 9968966c6326
Step 5/6 : ENV HOME /
---> Running in 90af33154d56
Removing intermediate container 90af33154d56
---> 18d5f8cb3e93
Step 6/6 : ENTRYPOINT ["/myapp-2"]
---> Running in ce28a9e98842
Removing intermediate container ce28a9e98842
---> c1edc9d60948
Successfully built c1edc9d60948
Successfully tagged example.com/myapp-2:6d246ce-dirty__linux_amd64
container: example.com/myapp-1:6d246ce-dirty__linux_amd64
container: example.com/myapp-2:6d246ce-dirty__linux_amd64
➜ go-build-template git:(master) ✗ Going to need to investigate my zsh configuration. I just recently got into Go so I'm new to it all. Thanks for your help!
Great. I will commit that in a bit, then close this. :) Thanks for finding a corner case!
No problem! Glad to be of help, this is a really good starting point and compliments what is talked about in this repo (slides, presentations, etc) - https://github.com/golang-standards/project-layout
I tested out making a small app using gonic and can confirm that make clean works as intended now.
Some folks hate the pkg convention. This should be adaptable enough to let you put code wherever you want now.
BTW, make build && make container is redundant - make container implies make build :)
Pushed.
Haha yea I guess it is. I was just stuck on different build issues that I have to be sure everything was running. Sanity checks. That's great! Thanks Tim!