nmattia/niv

source.json interpolation with environment variables

yaitskov opened this issue · 5 comments

yaitskov commented

I spent a few days of integrating a cascade of CI for several projects on gitlab.

The issue is ssh authentication. Every git repo is private.
gitlab has a hack - special user with token.

so if sources.json instead of:

{
 "private-dep": {
   "branch": "master",
   "repo": "ssh://git@gitlab.com/company/private-dep.git",
   "rev": "ed73d970fd13b009ca5829f6b8c1c936c7221032",
   "type": "git"
 },

would contain following:

"repo": "ssh://gitlab-ci-token:$CI_JOB_TOKEN@gitlab.com/company/private-dep.git",

then there is no hassle with ssh keys at all.

Some expression language (mustache?) supporting environment variables would help here a lot.

"repo": "ssh://git$([ -n "$CI_JOB_TOKEN ] && echo "lab-ci-token:$CI_JOB_TOKEN)@gitlab.com/company/private-dep.git",
nmattia commented

That sounds like a good idea. Maybe we can even unify this with the "template" feature used in e.g. URL templates (where <foo> in a string is replaced with the value of the foo attribute).

refnil commented

Some expression language (mustache?) supporting environment variables would help here a lot.

I would be hesitant to bring any expression language during the nix evaluation. It would add that language as a dependency to every niv user.

Then the next idea is implement the interpreter for the expression language in nix itself. That seems like a project in itself.

My proposal to solve this problem would be to add a system of mirror to niv and upgrade the template feature to support environment variable.
In you case, the first mirror would be the regular url ssh://git@gitlab.com/company/private-dep.git and the second would be to modified url for the gitlab-ci ssh://gitlab-ci-token:<CI_JOB_TOKEN>@gitlab.com/company/private-dep.git.
The biggest uncertainty of this approach is that we need to change each fetchers to support mirrors since I don't think we can do it generically in nix.

zimbatm commented

Specifically for HTTP fetches, libcurl reads the user's ~/.netcrc file for basic auth type credentials. Maybe Git is also respecting that file?

I'm thinking in terms of addressing the underlying issue.

adrian-gierakowski commented
jh-devv commented

What about other https mirrors that need authentication in the form of cookies?