extraPullNames does not seem to work
terlar opened this issue · 17 comments
I have a job that looks like this:
---
name: Build
on:
pull_request:
push:
jobs:
builds:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v12
with:
install_url: https://github.com/numtide/nix-flakes-installer/releases/download/nix-2.4pre20201221_9fab14a/install
extra_nix_config: |
experimental-features = nix-command flakes
- uses: cachix/cachix-action@v8
with:
name: terlar
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
extraPullNames: nix-community
- run: cat $HOME/.config/nix/nix.conf
- run: nix build --print-build-logs .#defaultPackage.x86_64-linux
The output of cachix/cachix-action@v8:
Run cachix/cachix-action@v8
Cachix: installing
/home/runner/.nix-profile/bin/cachix authtoken ***
Written to /home/runner/.config/cachix/cachix.dhall
Cachix: using cache terlar
/nix/var/nix/profiles/per-user/runner/profile/bin/cachix use terlar
Configured https://terlar.cachix.org binary cache in /home/runner/.config/nix/nix.conf
Cachix: using extra caches nix-community
/nix/var/nix/profiles/per-user/runner/profile/bin/cachix use nix-community
Configured https://terlar.cachix.org binary cache in /home/runner/.config/nix/nix.conf
/bin/sh -c nix path-info --all | grep -v '.drv$' > /tmp/store-path-pre-build
The output of the cat:
Run cat $HOME/.config/nix/nix.conf
substituters = https://cache.nixos.org https://terlar.cachix.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= terlar.cachix.org-1:M8CXTOaJib7CP/jEfpNJAyrgW4qECnOUI02q7cnmh8U=
Seems it always add the cache name also for the extra caches as indicated by the output of cachix/cachix-action@v8. I started looking into this as when I transitioned from adding the cache myself to using the extraPullNames my build times went up a lot. Any idea what is going on?
This seems to be an issue with cachix CLI itself. I managed to reproduce this myself locally.
$ cachix use -O . iohk
Configured https://terlar.cachix.org binary cache in ./nix.conf
$ cat ~/.config/cachix/cachix.dhall
{ authToken =
"SNIP"
, binaryCaches = [] : List { name : Text, secretKey : Text }
} $ mv ~/.config/cachix/cachix.dhall{,.bak}
$ cachix use -O . iohk
Configured https://iohk.cachix.org binary cache in ./nix.conf
If I have auth token configured in the $XDG_CONFIG_HOME/cachix/cachix.dhall it always uses the cache that token belongs to and I can basically not use any other caches.
@domenkozar ping, in case this information helps you, as you are looking into it.
@terlar what is the contents of ./nix.conf?
@terlar According to your output this feature works exactly as expected, maybe you can explain what you're trying to achieve?
No, it always adds https://terlar.cachix.org, no matter what CACHE-NAME I try to use. The source of that was the cachix.dhall file. I expect it to add the iohk cache when that is the one I add.
What I am trying to achieve is to have multiple substituters, managed by cachix. My original case was exactly as I mentioned for the GitHub actions. However when I tried using cachix locally I noticed the same behavior and found the culprit.
In my test the ./nix.conf was non-existent. Then I ran it several times adding different cache names.
When I have the cachix.dhall file, it always ends up being the terlar CACHE-NAME.
$ file nix.conf
nix.conf: cannot open `nix.conf' (No such file or directory)
$ cachix use -O . iohk
Configured https://terlar.cachix.org binary cache in ./nix.conf
$ cat nix.conf
substituters = https://cache.nixos.org https://terlar.cachix.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= terlar.cachix.org-1:M8CXTOaJib7CP/jEfpNJAyrgW4qECnOUI02q7cnmh8U=
$ cachix use -O . nix-community
Configured https://terlar.cachix.org binary cache in ./nix.conf
$ cat nix.conf
substituters = https://cache.nixos.org https://terlar.cachix.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= terlar.cachix.org-1:M8CXTOaJib7CP/jEfpNJAyrgW4qECnOUI02q7cnmh8U=
Now without cachix.dhall:
$ mv ~/.config/cachix/cachix.dhall{,.bak}
$ rm nix.conf
$ file nix.conf
nix.conf: cannot open `nix.conf' (No such file or directory)
$ cachix use -O . iohk
Configured https://iohk.cachix.org binary cache in ./nix.conf
$ cat nix.conf
substituters = https://cache.nixos.org https://iohk.cachix.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= iohk.cachix.org-1:DpRUyj7h7V830dp/i6Nti+NEO2/nhblbov/8MW7Rqoo=
$ cachix use -O . nix-community
Configured https://nix-community.cachix.org binary cache in ./nix.conf
$ cat nix.conf
substituters = https://cache.nixos.org https://iohk.cachix.org https://nix-community.cachix.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= iohk.cachix.org-1:DpRUyj7h7V830dp/i6Nti+NEO2/nhblbov/8MW7Rqoo= nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
I hope this is verbose enough to demonstrate the issue. I am just doing this to demonstrate the issue.
@domenkozar Hope this clarified the issue a bit. Locally I am using:
$ cachix --version
cachix 0.5.1
On GitHub Actions I guess it is using whatever is defined within the cachix/cachix-action@v8
So the culprit seems to be:
Cachix: using extra caches nix-community
/nix/var/nix/profiles/per-user/runner/profile/bin/cachix use nix-community
Configured https://terlar.cachix.org binary cache in /home/runner/.config/nix/nix.conf
Where it's saying to use nix-community but it's really configuring terlar cache?
@terlar could you link me the github action that reproduces this bug, I can't reproduce it locally.
#71 (comment)
@domenkozar Yes, that describes what happens.
Seeing as it happens both locally and on GitHub actions, I don't think it is environment specific. But both environments are using the same authToken.
Here is an example of GitHub action that reproduces this bug:
https://github.com/terlar/emacs-config/runs/1617492237?check_suite_focus=true
Ah, I think I know what the problem is.
You're using cache token, which will always configure just that one cache.
Need to think about how this should be handled.
@terlar I found a way to fix this, hopefully tomorrow.
Thank you for the quick identification and response @domenkozar.
I followed the instructions at:
https://docs.cachix.org/getting-started.html#authenticating
I also read somewhere that it was recommended to use the Cache Auth Tokens instead of the Personal Auth Tokens. So just to understand this, the issue is that using a Cache Auth Token is not working in the same way?
I tried generating a signing key (cachix generate-keypair terlar) with the Cache Auth Token and that failed, are you supposed to be able to do that as well? Because I saw most people were using signing keys in their GitHub Actions (for public projects).
Binary cache doesn't exist or it's a private cache and you don't have access. Please check the name for typos or provide an auth token.
I also read somewhere that it was recommended to use the Cache Auth Tokens instead of the Personal Auth Tokens. So just to understand this, the issue is that using a Cache Auth Token is not working in the same way?
That is correct. The problem is that cache token is tied to a specific cache, but really requesting another cache should just work.
I tried generating a signing key (cachix generate-keypair terlar) with the Cache Auth Token and that failed, are you supposed to be able to do that as well? Because I saw most people were using signing keys in their GitHub Actions (for public projects).
That is if you want to manage signing yourself, which I don't recommend as per https://blog.cachix.org/posts/2020-11-09-write-access-control-for-binary-caches/
Okay, thank you for the clarification, that makes sense. I guess those examples I saw was configured before this feature. Then I will stick to the recommendations and await your changes. In the meantime there is also the work-around to populate the nix configuration with other caches "manually".
Funnily enough if you recommended to use signing key my next question would be about how I rotate/revoke them 😸
Thank you for cachix, apart from this it has been smooth sailing.
@terlar the fix has been deployed, can you confirm?
@domenkozar I can confirm that the fix is working! Thank you very much.
Sorry for the delay, I verified it quite quickly locally, but wanted to verify on CI but had some unrelated issues.
Now my builds are down from 1h 15min to 5-7min. 🐎