Authentication failure in running dependabot update on private-repo having pip as package-manager

Question

Authentication failure in running dependabot update on private-repo having pip as package-manager

rahmad-evertz opened this issue a year ago · 9 comments

Hey!

I am facing a private_source_authentication_failure when trying to run dependabot-cli on private repo. I was trying to run dependabot update command. We authenticate with username and password and used an AWS codeartifact repo as source.

Dependabot update command

dependabot update -f test.yaml

Error

........
........
updater | 2023/11/13 18:24:56 INFO Checking if boto3 1.24.84 needs updating
{"data":{"error-type":"private_source_authentication_failure","error-details":{"source":"https://example.com/"}},"type":"record_update_job_error"}
  proxy | 2023/11/13 18:24:56 [008] POST http://host.docker.internal:45341/update_jobs/cli/record_update_job_error
  proxy | 2023/11/13 18:24:56 [008] 200 http://host.docker.internal:45341/update_jobs/cli/record_update_job_error
updater | 2023/11/13 18:24:56 INFO Handled error whilst updating boto3: private_source_authentication_failure {:source=>"https:/example.com/"}
updater | 2023/11/13 18:24:56 INFO Checking if marshmallow 3.18.0 needs updating
  proxy | 2023/11/13 18:24:56 [009] POST http://host.docker.internal:45341/update_jobs/cli/record_update_job_error
{"data":{"error-type":"private_source_authentication_failure","error-details":{"source":"https://example.com/"}},"type":"record_update_job_error"}
.........
........

Sample Job Description

job:
    package-manager: pip
    allowed-updates:
      - update-type: all
    security-advisories:
      - dependency-name: black
        affected-versions:
          - <20.0.0
        patched-versions: []
        unaffected-versions: []
    source:
        provider: github
        repo: example/test
        directory: /      
credentials:
  - type: python-index
    registry: [registry_url]
    token: [token]

What we are really looking for is to authenticate using username/password and using AWS codeartifact repo as source.

Answer 1 · 2023-12-01T14:12:42.000Z

Did you try specifying username and password instead of token?

Answer 2 · 2023-12-01T14:45:28.000Z

Did you try specifying username and password instead of token?

yeah! I have tried that @deivid-rodriguez

Answer 3 · 2023-12-01T15:16:51.000Z

For Python the credentials section should be like so:

credentials:
  - type: python_index # underscore
    index-url: [registry_url] # index-url, not registry
    token: [token]

Answer 4 · 2023-12-01T15:33:10.000Z

Ah yes, I think the token needs to be specified as username:password, did you use that format?

Answer 5 · 2023-12-18T01:19:02.000Z

I encountered this problem as well. Despite setting the token in the format of "username:password", I continued to receive an error.

updater | 2023/12/18 01:11:14 ERROR Error processing pydantic (KeyError)
updater | 2023/12/18 01:11:14 ERROR key not found: "index-url"

My Job definition:

Complete Log

➜  dependabot-core git:(fix-8533-dry-run-failed) ✗ dependabot update -f job.json 
 
    cli | 2023/12/18 01:11:06 Adding missing credentials-metadata into job definition
    cli | 2023/12/18 01:11:06 using image ghcr.io/github/dependabot-update-job-proxy/dependabot-update-job-proxy:latest at sha256:51c2db9ad982aa81cde99979c5bb0f900de092cc37bf65d555c9663bfde053ec
    cli | 2023/12/18 01:11:06 using image ghcr.io/dependabot/dependabot-updater-pip at sha256:6475120f5a2b1174029943b6bd16c2e6ead32b84f31abf6c931c9a9cf2f6091f
updater | Updating certificates in /etc/ssl/certs...
  proxy | 2023/12/18 01:11:07 proxy starting, commit: 6cffd6fae1b2f713f2d837bc45fe916f855c821d
  proxy | 2023/12/18 01:11:07 initializing metrics client: No address passed and autodetection from environment failed
  proxy | 2023/12/18 01:11:07 Listening (:1080)
updater | rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
updater | 1 added, 0 removed; done.
updater | Running hooks in /etc/ca-certificates/update.d...
updater | done.
updater | 2023/12/18 01:11:08 INFO Raven 3.1.2 configured not to capture errors: DSN not set
updater | 2023/12/18 01:11:08 INFO Starting job processing
  proxy | 2023/12/18 01:11:09 [002] GET https://github.com:443/MYORG/MYREPO/info/refs?service=git-upload-pack
  proxy | 2023/12/18 01:11:09 [002] * authenticating git server request (host: github.com)
  proxy | 2023/12/18 01:11:10 [002] 200 https://github.com:443/MYORG/MYREPO/info/refs?service=git-upload-pack
  proxy | 2023/12/18 01:11:10 [003] POST https://github.com:443/MYORG/MYREPO/git-upload-pack
  proxy | 2023/12/18 01:11:10 [003] * authenticating git server request (host: github.com)
  proxy | 2023/12/18 01:11:10 [003] 200 https://github.com:443/MYORG/MYREPO/git-upload-pack
  proxy | 2023/12/18 01:11:10 [004] POST https://github.com:443/MYORG/MYREPO/git-upload-pack
  proxy | 2023/12/18 01:11:10 [004] * authenticating git server request (host: github.com)
  proxy | 2023/12/18 01:11:10 [004] 200 https://github.com:443/MYORG/MYREPO/git-upload-pack
updater | 2023/12/18 01:11:11 INFO Finished job processing
updater | 2023/12/18 01:11:12 INFO Raven 3.1.2 configured not to capture errors: DSN not set
updater | 2023/12/18 01:11:12 INFO Starting job processing
  proxy | 2023/12/18 01:11:14 [005] POST http://host.docker.internal:57960/update_jobs/cli/update_dependency_list
    cli | 2023/12/18 01:11:14 yaml: unmarshal errors:  line 1: cannot unmarshal !!str `g8` into model.RequirementSource  line 1: cannot unmarshal !!str `g8` into model.RequirementSource  line 1: cannot unmarshal !!str `g8` into model.RequirementSource  line 1: cannot unmarshal !!str `g8` into model.RequirementSource  line 1: cannot unmarshal !!str `g8` into model.RequirementSource
{"data":{"dependencies":null,"dependency_files":null},"type":"update_dependency_list"}
  proxy | 2023/12/18 01:11:14 [005] 200 http://host.docker.internal:57960/update_jobs/cli/update_dependency_list
  proxy | 2023/12/18 01:11:14 [006] POST http://host.docker.internal:57960/update_jobs/cli/increment_metric
{"data":{"metric":"updater.started","tags":{"operation":"update_all_versions"}},"type":"increment_metric"}
  proxy | 2023/12/18 01:11:14 [006] 200 http://host.docker.internal:57960/update_jobs/cli/increment_metric
updater | 2023/12/18 01:11:14 INFO Starting update job for MYORG/MYREPO
updater | 2023/12/18 01:11:14 INFO Checking all dependencies for version updates...
updater | 2023/12/18 01:11:14 INFO Checking if pydantic 1.10.13 needs updating
  proxy | 2023/12/18 01:11:14 [007] POST http://host.docker.internal:57960/update_jobs/cli/record_update_job_error
{"data":{"error-type":"unknown_error","error-details":null},"type":"record_update_job_error"}
  proxy | 2023/12/18 01:11:14 [007] 200 http://host.docker.internal:57960/update_jobs/cli/record_update_job_error
updater | 2023/12/18 01:11:14 ERROR Error processing pydantic (KeyError)
updater | 2023/12/18 01:11:14 ERROR key not found: "index-url"
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/authed_url_builder.rb:9:in `fetch'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/authed_url_builder.rb:9:in `authed_url'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/index_finder.rb:156:in `block in config_variable_index_urls'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/index_finder.rb:156:in `map'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/index_finder.rb:156:in `config_variable_index_urls'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/index_finder.rb:23:in `index_urls'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/latest_version_finder.rb:218:in `index_urls'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/latest_version_finder.rb:146:in `available_versions'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/latest_version_finder.rb:53:in `fetch_latest_version'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker/latest_version_finder.rb:34:in `latest_version'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker.rb:243:in `fetch_latest_version'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/python/lib/dependabot/python/update_checker.rb:33:in `latest_version'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:177:in `all_versions_ignored?'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:80:in `check_and_create_pull_request'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:60:in `check_and_create_pr_with_error_handling'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:35:in `block in perform'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:35:in `each'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/update_all_versions.rb:35:in `perform'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:64:in `run'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:39:in `perform_job'
updater | 2023/12/18 01:11:14 ERROR /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:53:in `run'
updater | 2023/12/18 01:11:14 ERROR bin/update_files.rb:24:in `<main>'

Answer 6 · 2024-01-02T14:10:38.000Z

@lucemia I just released v1.44 which should fix that! https://github.com/dependabot/cli/releases/tag/v1.44.0

Answer 7 · 2024-01-02T16:59:13.000Z

@jakecoffman

Thanks, I can confirm that this issue has been resolved with version 1.44.0.

Answer 8 · 2024-01-29T04:03:51.000Z

@lucemia do you mind if I ask if you use pipenv? And if you do, how do you specify the token needed to authenticate against your private repository? We pass it via an env var into our Pipfile, but in that case the CLI is unable to authenticate against it, even when passing the token in the job definition. Similarly, unless we hardcode the token, the github dependabot is creating invalid Pipfile.lock files for us: dependabot/dependabot-core#7936

Thanks!

Answer 9 · 2024-01-29T09:16:24.000Z

@juanitosvq

No, I didn't use pipenv.
My scenario is pretty straightforward.

I ran the command:

dependabot update -f job.json

Here's how my job.json is set up:

# job.yaml
job:
    package-manager: pip
    allowed-updates:
      - update-type: all
    source:
        provider: github
        repo: [ORG/REPO]
        directory: /
        branch: main
credentials:
  - type: python_index
    index-url: PRIVATE_PYPI
    token: _json_key_base64:[PRIVATE_KEY]
  - type: git_source
    host: github.com
    username: x-access-token
    password: [GITHUB_TOKEN]

It started working after PR https://github.com/dependabot/cli/pull/215/files was merged!