Issue: #567 breaks `act` on Windows

Question

Issue: #567 breaks `act` on Windows

catthehacker opened this issue 4 years ago · 26 comments

catthehacker commented 4 years ago

Act version

6cde8f6
https://github.com/nektos/act/releases/tag/v0.2.21

Expected behaviour

act doesn't fail

Actual behaviour

act fails because paths on Windows (C:\users\something\path\) differ from paths on Linux (/home/something/path)

Workflow and/or repository

workflow

on:
  push:
jobs:
  tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

Steps to reproduce

Run act on Windows

`act` output

Log

cat  ~\..\..\..\..\act git: (6c258cf...) ≢ +1 ~0 -0 ! 
❯❯❯ go run main.go -W .\.github\workflows\test.yml
[test.yml/tests-1] 🧪  Matrix: map[deno:1.1.0 os:ubuntu-latest]
[test.yml/tests-2] 🚧  Skipping unsupported platform 'macos-latest'
[test.yml/tests-1] 🚀  Start image=catthehacker/ubuntu:act-latest
[test.yml/tests-1]   🐳  docker run image=catthehacker/ubuntu:act-latest platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[test.yml/tests-1]   🐳  docker cp src=C:\Users\cat\go\src\github.com\nektos\act\. dst=\github\workspace
[test.yml/tests-1] ⭐  Run actions/checkout@v2
[test.yml/tests-1]   ✅  Success - actions/checkout@v2
Error: context canceled
exit status 1
cat  ~\..\..\..\..\act git: (6c258cf...) ≢ +1 ~0 -0 !
❯❯❯
cat  ~\..\..\..\..\act git: (6cde8f6...) ≢ +1 ~0 -0 !
❯❯❯ go run main.go -W .\.github\workflows\test.yml
[test.yml/tests-1] 🧪  Matrix: map[deno:1.1.0 os:ubuntu-latest]
[test.yml/tests-2] 🚧  Skipping unsupported platform 'macos-latest'
[test.yml/tests-1] 🚀  Start image=catthehacker/ubuntu:act-latest
Error: Error response from daemon: the working directory 'C:\Users\cat\go\src\github.com\nektos\act' is invalid, it needs to be an absolute path
exit status 1
cat  ~\..\..\..\..\act git: (6cde8f6...) ≢ +1 ~0 -0 !
❯❯❯

Answer 1 · 2021-03-29T19:57:22.000Z

Work in progress: https://github.com/catthehacker/act-fork/tree/windows-fix

cat  ~\githubactions git: master ↓3
❯❯❯ act -j build -W .\.github\workflows\test.yml -v
time="2021-03-29T21:56:15+02:00" level=debug msg="Loading environment from C:\\Users\\cat\\githubactions\\.env"
time="2021-03-29T21:56:15+02:00" level=debug msg="Loading secrets from C:\\Users\\cat\\githubactions\\.secrets"
time="2021-03-29T21:56:15+02:00" level=debug msg="Loading workflow 'C:\\Users\\cat\\githubactions\\.github\\workflows\\test.yml'"
time="2021-03-29T21:56:15+02:00" level=debug msg="Reading workflow 'C:\\Users\\cat\\githubactions\\.github\\workflows\\test.yml'"
time="2021-03-29T21:56:15+02:00" level=debug msg="Planning job: build"
time="2021-03-29T21:56:15+02:00" level=debug msg="Loading slug from git directory 'C:\\Users\\cat\\githubactions\\.git'"
time="2021-03-29T21:56:15+02:00" level=debug msg="Found revision: 2cccdeba5c75811a18f55ce9609eb7015c0e36ec\n"
time="2021-03-29T21:56:15+02:00" level=debug msg="Loading revision from git directory 'C:\\Users\\cat\\githubactions\\.git'"
time="2021-03-29T21:56:15+02:00" level=debug msg="Found revision: 2cccdeba5c75811a18f55ce9609eb7015c0e36ec\n"
time="2021-03-29T21:56:15+02:00" level=debug msg="HEAD points to '2cccdeba5c75811a18f55ce9609eb7015c0e36ec'"
time="2021-03-29T21:56:15+02:00" level=debug msg="HEAD matches refs/heads/master"
time="2021-03-29T21:56:15+02:00" level=debug msg="using github ref: refs/heads/master"
time="2021-03-29T21:56:15+02:00" level=debug msg="context env => map[ACT:true ImageOS:ubuntu18 MY_2ND_ENV_VAR:my 2nd env var value MY_ENV_VAR:MY_ENV_VAR_VALUE]"
[test.yml/build] 🚀  Start image=catthehacker/ubuntu:act-latest
[test.yml/build] 🚀  Workdir=C:\Users\cat\githubactions
[test.yml/build] 🚀  Split=[C \Users\cat\githubactions]
[test.yml/build] 🚀  Workdir=/mnt/c/Users/cat/githubactions
time="2021-03-29T21:56:15+02:00" level=warning msg="unable to get git repo: CreateFile C:\\mnt\\c\\Users\\cat\\githubactions: The system cannot find the path specified."
time="2021-03-29T21:56:15+02:00" level=warning msg="unable to get git revision: CreateFile C:\\mnt\\c\\Users\\cat\\githubactions: The system cannot find the path specified."
time="2021-03-29T21:56:15+02:00" level=warning msg="unable to get git ref: CreateFile C:\\mnt\\c\\Users\\cat\\githubactions: The system cannot find the path specified."
[test.yml/build]   🐳  docker pull catthehacker/ubuntu:act-latest
time="2021-03-29T21:56:15+02:00" level=debug msg="Image exists? true"
[test.yml/build] Removed container: f35e4f82297cc50f1753c6a3a6893946fa685372f95eadb413a34f0187eacc5a
[test.yml/build]   🐳  docker volume rm act-test-yml-build
[test.yml/build]   🐳  docker create image=catthehacker/ubuntu:act-latest platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[test.yml/build] Created container name=act-test-yml-build id=fa58ad1905b72b05addae0b9f4e9f3a27ad6c2bc12bc97e218667a7e595eb68a from image catthehacker/ubuntu:act-latest (platform: linux/amd64)
[test.yml/build] ENV ==> [RUNNER_TOOL_CACHE=/opt/hostedtoolcache RUNNER_OS=Linux RUNNER_TEMP=/tmp]
[test.yml/build]   🐳  docker run image=catthehacker/ubuntu:act-latest platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[test.yml/build] Starting container: fa58ad1905b72b05addae0b9f4e9f3a27ad6c2bc12bc97e218667a7e595eb68a
[test.yml/build] Started container: fa58ad1905b72b05addae0b9f4e9f3a27ad6c2bc12bc97e218667a7e595eb68a
[test.yml/build]   🐳  docker cp src=/mnt/c/Users/cat/githubactions\. dst=\mnt\c\Users\cat\githubactions
[test.yml/build] Exec command '[mkdir -p /mnt/c/Users/cat/githubactions]'
time="2021-03-29T21:56:16+02:00" level=debug msg="Writing tarball C:\\Users\\cat\\AppData\\Local\\Temp\\act243004607 from /mnt/c/Users/cat/githubactions\\."
time="2021-03-29T21:56:16+02:00" level=debug msg="Stripping prefix:\\mnt\\c\\Users\\cat\\githubactions\\ src:/mnt/c/Users/cat/githubactions\\."
time="2021-03-29T21:56:16+02:00" level=debug msg="Error loading .gitignore: open \\mnt\\c\\Users\\cat\\githubactions: The system cannot find the path specified."
time="2021-03-29T21:56:16+02:00" level=debug msg="CreateFile /mnt/c/Users/cat/githubactions\\.: The system cannot find the path specified."
time="2021-03-29T21:56:16+02:00" level=debug msg="CreateFile /mnt/c/Users/cat/githubactions\\.: The system cannot find the path specified."
time="2021-03-29T21:56:16+02:00" level=debug msg="CreateFile /mnt/c/Users/cat/githubactions\\.: The system cannot find the path specified."
time="2021-03-29T21:56:16+02:00" level=debug msg="CreateFile /mnt/c/Users/cat/githubactions\\.: The system cannot find the path specified."
Error: CreateFile /mnt/c/Users/cat/githubactions\.: The system cannot find the path specified.
cat  ~\githubactions git: master ↓3
❯❯❯

Answer 2 · 2021-03-29T20:14:26.000Z

Let me know if you need any testing, but these working directories seem like they would be with bind mount. Since its in the context of the docker container, couldn't you just use the same paths, and focus more on a volume binding on the windows side?

Answer 3 · 2021-03-29T20:23:01.000Z

Not sure what you mean exactly.

Answer 4 · 2021-03-29T21:47:56.000Z

Recommendation to fix the issue for windows:

The CWD (working directory) of the docker commands that get executed should always be from the perspective of "inside" the container, so it should be the variable which resolves to /github/workspace or /home/runner/work/PROJECT or whatever regardless of if it is linux or windows. Docs: https://docs.docker.com/engine/reference/commandline/exec/
The Copy action to copy the working dir into the workspace via a docker command. Since act runs on windows there's no path translation that has to happen here. Example: https://stackoverflow.com/a/40313917/5511129
The Bind action (act -b) should mount the workspace dir to the docker volume. You can totally use the windows path in the docker command for making the volume and/or the binding. Example: https://stackoverflow.com/a/57553028/5511129

Your WIP seems to be trying to translate the windows path into the WSL /mnt/c/etc. path and that should not be necessary. I'll attempt a PR of the above if I have some time today.

Answer 5 · 2021-03-29T21:59:33.000Z

@catthehacker also on the first point, it's my opinion that it should be the dockerfile that sets the working dir, and act just rolls with it and omits the workdir parameter entirely.

Answer 6 · 2021-04-05T15:26:07.000Z

Just confirming, I've just found this project (hi!) and it does indeed break on windows with this error.

Answer 7 · 2021-04-05T15:31:40.000Z

@metamoof v0.2.20 should still work FYI

https://github.com/nektos/act/releases/tag/v0.2.20

Answer 8 · 2021-04-05T15:34:47.000Z

Yeah! seem to work, thanks @JustinGrote!

Answer 9 · 2021-04-05T16:46:47.000Z

Note to everyone trying to decide if they have the right issue, it is possible the issue you're looking for is in fact #550.

A simple check: if you have a problem w/ act on Windows, try using VirtualBox (maybe choco install VirtualBox --params /ExtensionPack ?) w/ Debian or Ubuntu and see if you have the same (general) problem. If you do, then, this is probably not the (only?) issue that needs fixing before you'll be happy.

Answer 10 · 2021-04-05T17:01:02.000Z

@jsoref no the latest release introduced a regression due to the pathing of the exec commands in docker not being interpreted properly.

Answer 11 · 2021-04-05T18:10:23.000Z

Note to everyone trying to decide if they have the right issue, it is possible the issue you're looking for is in fact #550.

A simple check: if you have a problem w/ act on Windows, try using VirtualBox (maybe choco install VirtualBox --params /ExtensionPack ?) w/ Debian or Ubuntu and see if you have the same (general) problem. If you do, then, this is probably not the (only?) issue that needs fixing before you'll be happy.

No, it's not the right issue. I'm 100% confident in what I wrote and why is it's breaking.

Answer 12 · 2021-04-12T09:11:41.000Z

The Bind action (act -b) should mount the workspace dir to the docker volume. You can totally use the windows path in the docker command for making the volume and/or the binding. Example: stackoverflow.com/a/57553028/5511129

~~3. The problem is not about if Windows path is supported or not but it breaks somewhere in between during path translation and it arrives at path mounting as not absolute~~
I replied without remembering the context of issue. Problem is that path arrives as Windows path in Linux container. Since it does not start with / it is not absolute.

Answer 13 · 2021-04-12T09:21:26.000Z

assertion failed: error is not nil: Binds: [/var/run/docker.sock:/var/run/docker.sock C:\Users\cat\go\src\github.com\nektos\act\pkg\runner\testdata:C:\Users\cat\go\src\github.com\nektos\act\pkg\runner\testdata] | Mounts: [{volume act-toolcache /toolcache false  <nil> <nil> <nil>} {volume act-actions /actions false  <nil> <nil> <nil>} {volume act-basic-check C:\Users\cat\go\src\github.com\nektos\act\pkg\runner false  <nil> <nil> <nil>}] | Error: Error response from daemon: the working directory 'C:\Users\cat\go\src\github.com\nektos\act\pkg\runner\testdata' is invalid, it needs to be an absolute path: C:\Users\cat\go\src\github.com\nektos\act\pkg\runner\testdata\basic

Answer 14 · 2021-04-12T09:44:47.000Z

I think it makes no sense to keep breaking PRs behaviour on Windows. It does not allow to overcome the issue unless you tap into WSL2 instance of Docker Desktop and have sources available there.
Unless someone uses LCOW but it has been officially discontinued from Docker side so I think it's better for us to not support it as well.
Not sure how all of that will behave while using act on Windows host with docker over ssh:// since runtime.GOOS == "windows", perhaps checking if engine is docker-desktop?

 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 1.871GiB
 Name: docker-desktop

Answer 15 · 2021-04-12T10:36:45.000Z

time="2021-04-12T10:33:48Z" level=debug msg="Binds: [  /var/run/docker.sock:/var/run/docker.sock C:\\Users\\cat\\go\\src\\github.com\\nektos\\act\\pkg\\runner\\testdata:/win/c/Users/cat/go/src/github.com/nektos/act/pkg/runner/testdata] | Mounts: [{volume act-toolcache /toolcache false  <nil> <nil> <nil>} {volume act-actions /actions false  <nil> <nil> <nil>} {volume act-basic-check /win/c/Users/cat/go/src/github.com/nektos/act/pkg/runner false  <nil> <nil> <nil>}] | Error: Error response from daemon: Mount denied:\nThe source path \"\"\ndoesn't contains colon
    runner_test.go:66: assertion failed: error is not nil: Binds: [  /var/run/docker.sock:/var/run/docker.sock C:\Users\cat\go\src\github.com\nektos\act\pkg\runner\testdata:/win/c/Users/cat/go/src/github.com/nektos/act/pkg/runner/testdata] | Mounts: [{volume act-toolcache /toolcache false  <nil> <nil> <nil>} {volume act-actions /actions false  <nil> <nil> <nil>} {volume act-basic-check /win/c/Users/cat/go/src/github.com/nektos/act/pkg/runner false  <nil> <nil> <nil>}] | Error: Error response from daemon: Mount denied:
        The source path ""
        doesn't contains colon: C:\Users\cat\go\src\github.com\nektos\act\pkg\runner\testdata\basic
    --- FAIL: TestRunEvent/basic (5.37s)

I hate it

Answer 16 · 2021-04-23T02:22:24.000Z

So it's taken me a while to untangle this codebase (I'm both a golang n00b and I'm bad with lots of generic abstractions and interfaces) but I've noticed a couple key items:

Workdir is optional, it can be omitted and it will just use the WORKDIR defined in the dockerfile which is probably more appropriate anyways. I just set it to a blank string and things started moving forward without issues
The only remaining issue is the mount to filepath. I don't quite see the point of this mount (Github Actions runner itself doesn't do it), maybe it was done for performance? Anyhow, I just removed the mount and my actions started working fine again.

@catthehacker Thoughts? I'll tweak with it a bit more and get a PR with my working config that hopefully doesn't break anything.

Answer 17 · 2021-04-23T07:13:35.000Z

The mount is for whatever happens in your codebase during workflow it's not happening on your host but in the volume which can be easily discarded later

Answer 18 · 2021-04-23T14:39:09.000Z

@catthehacker why have a mount at all? Why not just do all the work inside the container? Is this a performance thing?

Answer 19 · 2021-04-23T14:44:33.000Z

Because you can also attach that volume to other container later. I'd like to preserve the behaviour across platforms

Answer 20 · 2021-04-23T14:50:10.000Z

If on Linux WorkingDir is set to path /same/path/as/host I'd like for any other platform to behave the same way if it's possible even if in code it behaves differently.

Answer 21 · 2021-04-23T16:16:35.000Z

@catthehacker: seems like you want to check is runtime.GOOS == "windows" && bridge = file-socket.

fwiw I have:

- cluster:
  name: k3d-k3s-default
     server: https://0.0.0.0:50724

which is a different local thing...

Answer 22 · 2021-04-23T18:44:18.000Z

Because you can also attach that volume to other container later. I'd like to preserve the behaviour across platforms

OK but currently the workdir mount is relative to the internal container but shouldn't it be relative to the container workdir?

So for instance (on linux for example):
MyProjectLocation: /home/jgrote/myproject
Dockerfile WORKDIR: /home/runner (using your runner dockerfile as reference)

Inside the container a new volume gets created with the name of the job and is mounted inside the container at path /home/jgrote/myproject, but when the github actions run, they should be running in /home/runner (e.g. if I do a checkout it will checkout here) so basically nothing happens to this empty volume I've created, so what's the point of it?

How I think it should work:
The volume should be mounted on the WORKDIR regardless of OS, since that's where all the actions will execute, then if I do a checkout, the code gets checked out there and whatever I do to it happens as part of my workflow, and then when it ends that volume is still available to inspect/debug. That I do see the value in

Because the volume is still named the same as the job the volumes are still unique even if they all get mounted inside the container to the same path.

That said, as a "quick hack", a regex could just detect a windows file path and translate it to the WSL /mnt/{driveletter}/path and bind there, but it's not optimal due to the WSL drivers.

Answer 23 · 2021-04-23T18:58:40.000Z

So for instance (on linux for example):
MyProjectLocation: /home/jgrote/myproject
Dockerfile WORKDIR: /home/runner (using your runner dockerfile as reference)

Dockerfile WORKDIR doesn't mean here anything since it's overwritten by act and that's ok.
On Linux everything is running (by default) in the same path as it is stored on host.

Answer 24 · 2021-04-23T19:02:36.000Z

Alright, for now I'll pursue the WSL path translation as a method onto Config and replace a direct reference to workdir where appropriate, that will at least get things going again.

Answer 25 · 2021-04-24T06:20:53.000Z

@catthehacker I basically have this method now to translate the paths. I was going to put it on Input however I think it would make more sense either on Config or RunContext but then I'd either have to import input for the initial absolute path resolution or just duplicate it to avoid the dependency, do you have a preference or recommendation?

// Resolves the equivalent path in the container
// This is required for windows and WSL 2 to translate things like C:\Users\Myproject to /mnt/c/Users/Myproject
// For use in docker volumes and binds
func (i *Input) ResolveContainerPath(path string) string {
	var rpath = i.resolve(path) //extends existing resolve
	if rpath == "" {
		return rpath
	}

	//Test if the path is a windows path
	windowsPathRegex := regexp.MustCompile(`^([a-zA-Z]):\\(.+)$`)
	windowsPathComponents := windowsPathRegex.FindStringSubmatch(rpath)

	//Return as-is if no match
	if windowsPathComponents == nil {
		return rpath
	}

	//Convert to WSL2-compatible path
	//NOTE: Cannot use filepath because this runs on Windows and we need linux paths
	driveLetter := strings.ToLower(windowsPathComponents[1])
	translatedPath := strings.ReplaceAll(windowsPathComponents[2], `\`, `/`)
	// Should make something like /mnt/c/Users/person/My Folder/MyActProject
	result := filepath.Join("/mnt", driveLetter, translatedPath, `/`)
	return result
}

Answer 26 · 2021-05-02T06:13:14.000Z

While waiting on the PRs to land, attached is a build that fixes all known issues I can tell on the windows side, as well as updated workers that run jobs in /home/runner as a non-root user to more closely match the actual github actions runner. To use these just update your .actrc with the following:

-P ubuntu-latest=ghcr.io/justingrote/act-pwsh-dotnet
-P ubuntu-20.04=ghcr.io/justingrote/act-pwsh-dotnet
-P ubuntu-18.04=ghcr.io/justingrote/act-pwsh-dotnet
-P ubuntu-16.04=ghcr.io/justingrote/act-pwsh-dotnet

(Remove the -dotnet if you want a smaller image that is just pwsh)

https://github.com/JustinGrote/act/tree/release/ActForWindows
act.zip

Related Dockerfile repo:
https://github.com/JustinGrote/act-dockerimage/