Credentials not passed to dependabot-core script

[willpercey-gb]

I've tried using the CLI tool like so
dependabot update --file input.json

My input.json file looks like so

{
  "job": {
    "package-manager": "npm_and_yarn",
    "allowed-updates": [
      {
        "update-type": "all"
      }
    ],
    "source": {
      "provider": "bitbucket",
      "repo": "xxxxx",
      "directory": "\/",
      "branch": "master"
    }
  },
  "credentials": [
    {
      "type": "git_source",
      "host": "github.com",
      "username": "x-access-token",
      "password": "xxxxxx"
    },
    {
      "type": "git_source",
      "host": "bitbucket.org",
      "username": "xxxxx",
      "password": "xxxxx"
    },
    {
      "type": "npm_registry",
      "registry": "https:\/\/npm.pkg.github.com",
      "token": "xxxxx"
    }
  ]
}

I have made custom builds of both CLI and Dependabot Core to add some prints to the console into their code to help me debug where this problem occurs.
Dependabot Core requires credentials for Bitbucket, either username and password or access token.

I can see from these logs that the credentials are available in the CLI, but when the "updater" runs (dependabot-core) the credentials are missing values.

It's unclear to me where or why these credentials are getting dropped.

As a result, the connection to Bitbucket errors and PRs cannot be raised.

[jakecoffman]

The Dependabot CLI has the same architecture that we use in production where the credentials are not passed into the Updater but they're passed to the Proxy which injects the credentials into the HTTPS request. You can read more about that here: https://github.com/dependabot/dependabot-core#private-registry-credential-management

From the logs it looks like it's working correctly, I can see "authenticating git server request (host: bitbucket.org)" and a 200 response.

The CLI doesn't currently have the capability to create PRs. It does output to stdout enough information to create the PRs, we just haven't had the time to create the piece that makes the change on the server.

[willpercey-gb]

This is the part of the logs where I'm experiencing the error, which I assumed was due to the credentials.
I take it this error is because it's not able to raise PRs for Bitbucket?

[jakecoffman]

It's gathering data for the PR but won't actually create one.

Looks like if you change bitbucket.org to api.bitbucket.org in the credentials section it should fix the 404s.

[willpercey-gb]

What would the new credential type be for that?
I need to maintain the git_source as it is - so it can find the repo, but once it switches to API it'd need a different host

[jakecoffman]

Ah, try two entries, one for bitbucket.org the other api.bitbucket.org, both with git_source. I think that should work.

[willpercey-gb]

This doesn't seem to make any difference unfortunately

[jakecoffman]

I dusted off my old bitbucket account and was able to get it to work using this input:

job:
  package-manager: go_modules
  source:
    provider: bitbucket
    repo: dryroasted/simple-go
    directory: /go
  allowed-updates:
    - dependency-type: direct
      update-type: all
credentials:
    - type: git_source
      host: bitbucket.org
      username: dryroasted
      password: $BITBUCKET_APP_PASSWORD
    - type: git_source
      host: api.bitbucket.org
      username: dryroasted
      password: $BITBUCKET_APP_PASSWORD

I started out using a repository access token but was seeing the same error you were, so switched to an app password and it started working. Looks like it is documented here that app password is the only way to access the REST API.

[wblondel]

I spent hours trying to make Bitbucket authentication work, your solution didn't work for me either @jakecoffman

My job is as follows:

job:
  package-manager: composer
  allowed-updates:
    - update-type: all
  source:
    provider: bitbucket
    repo: redacted/business-platform
    directory: /
  credentials:
    - type: git_source
      host: api.bitbucket.org
      username: williamblondel
      password: (app-password)
    - type: git_source
      host: bitbucket.org
      username: williamblondel
      password: (app-password)

Output is as follows:

    cli | 2023/09/01 14:59:25 Inserting $LOCAL_GITHUB_ACCESS_TOKEN into credentials
    cli | 2023/09/01 14:59:25 Adding missing credentials-metadata into job definition
    cli | 2023/09/01 14:59:25 using image ghcr.io/github/dependabot-update-job-proxy/dependabot-update-job-proxy:latest at sha256:699df56da84381c10ea13e537ee8c18c0e00c5e230b13c2be9b44738d2109930
    cli | 2023/09/01 14:59:25 using image ghcr.io/dependabot/dependabot-updater-composer at sha256:ff576fc76eb78f1c2682233f4cdd964700be2f5e1a06b71c5abd4274b9c0edd3
  proxy | 2023/09/01 14:59:26 proxy starting, commit: 93c4a893d794d736d84e940a79420e8d1180c0bd
updater | Updating certificates in /etc/ssl/certs...
  proxy | 2023/09/01 14:59:26 initializing metrics client: No address passed and autodetection from environment failed
  proxy | 2023/09/01 14:59:26 Listening (:1080)
updater | rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
updater | 1 added, 0 removed; done.
updater | Running hooks in /etc/ca-certificates/update.d...
updater | done.
updater | 2023/09/01 14:59:38 INFO Raven 3.1.2 configured not to capture errors: DSN not set
updater | 2023/09/01 14:59:39 INFO Starting job processing
  proxy | 2023/09/01 14:59:39 [002] GET https://api.bitbucket.org:443/2.0/repositories/redacted/business-platform
  proxy | 2023/09/01 14:59:42 [002] 404 https://api.bitbucket.org:443/2.0/repositories/redacted/business-platform
updater | 2023/09/01 14:59:42 ERROR Error during file fetching; aborting
  proxy | 2023/09/01 14:59:42 [003] POST http://host.docker.internal:55611/update_jobs/cli/record_update_job_error
{"data":{"error-type":"job_repo_not_found","error-details":{}},"type":"record_update_job_error"}
  proxy | 2023/09/01 14:59:42 [003] 200 http://host.docker.internal:55611/update_jobs/cli/record_update_job_error
  proxy | 2023/09/01 14:59:42 [004] PATCH http://host.docker.internal:55611/update_jobs/cli/mark_as_processed
{"data":{"base-commit-sha":"unknown"},"type":"mark_as_processed"}
  proxy | 2023/09/01 14:59:42 [004] 200 http://host.docker.internal:55611/update_jobs/cli/mark_as_processed
updater | 2023/09/01 14:59:42 INFO Finished job processing
updater | 2023/09/01 14:59:42 INFO Results:
updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.
updater | +--------------------+
updater | |       Errors       |
updater | +--------------------+
updater | | job_repo_not_found |
updater | +--------------------+
  proxy | 2023/09/01 14:59:42 0/1 calls cached (0%)

I also tried defining the branch, the commit, always the same 404 error (no 401 or 403, always 404).

The app password I created has all permissions.

I really don't know what to try next.

[jakecoffman]

@wblondel The credentials property needs to be at the top level:

job:
  package-manager: composer
  allowed-updates:
    - update-type: all
  source:
    provider: bitbucket
    repo: redacted/business-platform
    directory: /
credentials:
  - type: git_source
    host: api.bitbucket.org
    username: williamblondel
    password: (app-password)
  - type: git_source
    host: bitbucket.org
    username: williamblondel
    password: (app-password)

[wblondel]

Oh... Sorry for that. I feel dumb now 😅
I guess a Friday night is not the right time to try to implement new things 😁
Thank you for your help!

[jakecoffman]

@wblondel No worries! I'm wondering if the CLI should error if there's an unknown property in the input file? 🤔 That would have told you right away what the problem was.