ggshield: protect your secrets with GitGuardian
ggshield is a CLI application that runs in your local environment or in a CI environment to help you detect more than 300 types of secrets, as well as other potential security vulnerabilities or policy breaks.
ggshield uses our public API through py-gitguardian to scan and detect potential secrets on files and other text content.
Only metadata such as call time, request size and scan mode is stored from scans using ggshield, therefore secrets and policy breaks incidents will not be displayed on your dashboard and your files and secrets won't be stored.
- Installation
- Initial setup
- Command reference
- Auth commands
- Config commands
- Secret scan commands
secret scan ci
: scan each commit since the last build in your CIsecret scan commit-range
: scan each commit in the given commit rangesecret scan path
: scan files or directories with the recursive optionsecret scan pre-commit
: scan every changes that have been staged in a git repositorysecret scan repo
: scan all commits in a git repositorysecret scan docker
: scan a Docker image after exporting its filesystem and manifest with thedocker save
commandsecret scan pypi
: scan a pypi packagesecret scan archive
: scan an archive files
secret ignore
commandinstall
commandquota
commandapi-status
command
- Configuration
- Integrations
- Output
- Related open source projects
- License
You can install ggshield using Homebrew by running the following command:
$ brew install gitguardian/tap/ggshield
Deb and RPM packages are available on Cloudsmith.
Setup instructions:
Install and update using pip
:
$ pip install ggshield
ggshield supports Python 3.7 and newer.
The package should run on MacOS, Linux and Windows.
To update ggshield you can add the option -U/--upgrade
to the pip install command.
$ pip install -U ggshield
To use ggshield you need to authenticate against GitGuardian servers. To do so, use the ggshield auth login
command. This command automates the provisioning of a personal access token and its configuration on the local workstation.
You can learn more about it from ggshield auth login
documentation.
Alternatively, you can create your personal access token manually and you can store it in the GITGUARDIAN_API_KEY
environment variable to complete the setup.
Once this is done, you can start scanning a repository with with ggshield secret scan repo /path/to/your/repo
.
Usage: ggshield [OPTIONS] COMMAND [ARGS]...
Options:
-c, --config-path FILE Set a custom config file. Ignores local and global
config files.
-v, --verbose Verbose display mode.
--allow-self-signed Ignore ssl verification.
--version Show the version and exit.
-h, --help Show this message and exit.
Commands:
api-status Show API status.
auth Commands to manage authentication.
install Install a pre-commit or pre-push git hook (local or global).
quota Show quotas overview.
secret Commands to work with secrets.
See the manual for information about the auth commands.
See the manual for information about the config commands.
The ggshield secret scan
command group contains the main ggshield commands, it has a few configuration options that can be used to override output behavior.
Usage: ggshield secret scan [OPTIONS] COMMAND [ARGS]...
Commands to scan various contents.
Options:
--json JSON output results [default: False]
--show-secrets Show secrets in plaintext instead of hiding
them.
--exit-zero Always return a 0 (non-error) status code, even
if incidents are found.The env var
GITGUARDIAN_EXIT_ZERO can also be used to set
this option.
--all-policies Present fails of all policies (Filenames,
FileExtensions, Secret Detection).By default,
only Secret Detection is shown.
-v, --verbose Verbose display mode.
-o, --output PATH Route ggshield output to file.
-b, --banlist-detector TEXT Exclude results from a detector.
--exclude PATH Do not scan the specified path.
--ignore-default-excludes Ignore excluded patterns by default. [default:
False]
-h, --help Show this message and exit.
Commands:
archive scan archive <PATH>.
ci scan in a CI environment.
commit-range scan a defined COMMIT_RANGE in git.
docker scan a docker image <NAME>.
path scan files and directories.
pre-commit scan as a pre-commit git hook.
pre-push scan as a pre-push git hook.
pre-receive scan as a pre-receive git hook.
pypi scan a pypi package <NAME>.
repo scan a REPOSITORY's commits at a given URL or path.
ggshield secret scan
has different subcommands for each type of scan.
Usage: ggshield secret scan ci [OPTIONS]
scan in a CI environment.
Options:
-h, --help Show this message and exit.
Usage: ggshield secret scan commit-range [OPTIONS] COMMIT_RANGE
scan a defined COMMIT_RANGE in git.
git rev-list COMMIT_RANGE to list several commits to scan. example: ggshield
secret scan commit-range HEAD~1...
Options:
-h, --help Show this message and exit.
Usage: ggshield secret scan path [OPTIONS] PATHS...
scan files and directories.
Options:
-r, --recursive Scan directory recursively
-y, --yes Confirm recursive scan
-h, --help Show this message and exit.
Usage: ggshield secret scan pre-commit [OPTIONS] [PRECOMMIT_ARGS]...
scan as a pre-commit git hook.
Options:
-h, --help Show this message and exit.
Usage: ggshield secret scan repo [OPTIONS] REPOSITORY
scan a REPOSITORY's commits at a given URL or path.
REPOSITORY is the clone URI or the path of the repository to scan. Examples:
ggshield secret scan repo git@github.com:GitGuardian/ggshield.git
ggshield secret scan repo /repositories/ggshield
Options:
-h, --help Show this message and exit.
secret scan docker
: scan a Docker image after exporting its filesystem and manifest with the docker save
command
Usage: ggshield secret scan docker [OPTIONS] NAME
scan a docker image <NAME>.
ggshield will try to pull the image if it's not available locally.
Options:
--docker-timeout SECONDS Timeout for Docker commands. [default: 360]
-h, --help Show this message and exit.
Usage: ggshield secret scan pypi [OPTIONS] PACKAGE_NAME
scan a pypi package <NAME>.
Options:
-h, --help Show this message and exit.
Under the hood this command uses the pip download
command to download the python package.
You can use pip
environment variables or configuration files to set pip download
parameters as explained in pip documentation.
For example, you can set pip
--index-url
parameter by setting PIP_INDEX_URL
environment variable.
Usage: ggshield secret scan archive [OPTIONS] PATH
scan archive <PATH>.
Options:
-h, --help Show this message and exit.
The secret ignore
command allows you to ignore some secrets.
For the time being, it only handles the --last-found
option that ignore all secrets found by the last run scan
command.
Under the hood, these secrets are added to the matches-ignore section of your local configuration file (if no local configuration file is found, a .gitguardian.yaml
file is created).
Warning: Using this command will discard any comment present in the configuration file.
Usage: ggshield secret ignore [OPTIONS]
Ignore some secrets.
Options:
--last-found Ignore secrets found in the last ggshield secret scan run
-h, --help Show this message and exit.
The install
command allows you to use ggshield as a pre-commit or pre-push hook
on your machine, either locally or globally for all repositories.
You will find further details in the pre-commit/pre-push part of this documentation.
Usage: ggshield install [OPTIONS]
Command to install a pre-commit or pre-push hook (local or global).
Options:
-m, --mode [local|global] Hook installation mode [required]
-t, --hook-type [pre-commit|pre-push]
Type of hook to install
-f, --force Force override
-a, --append Append to existing script
-h, --help Show this message and exit.
Show remaining quota of the workspace.
Usage: ggshield quota [OPTIONS]
Command to show quotas overview.
Options:
--json JSON output results [default: False]
-h, --help Show this message and exit.
Example:
❯ ggshield quota
Quota available: 9440
Quota used in the last 30 days: 560
Total Quota of the workspace: 10000
Show API status and version.
Usage: ggshield api-status [OPTIONS]
Command to show api status.
Options:
--json JSON output results [default: False]
-h, --help Show this message and exit.
Example:
❯ ggshield api-status
status: healthy
app-version: 1.27.0-rc.1
secrets-engine-version-version: 2.44.0
Configuration in ggshield follows a global>local>CLI
configuration scheme.
Meaning options on local
overwrite or extend global
and options on CLI overwrite or extend local.
ggshield will search for a global
configuration file in the user's home directory (example: ~/.gitguardian.yml
on Linux and %USERPROFILE%\.gitguardian
on Windows).
ggshield will recognize as well a local
configuration file in the user's working directory (example: ./.gitguardian.yml
).
You can also use the option --config-path
on the main command to set another configuration file. In this case, neither local
nor global
configuration files will be evaluated (example: ggshield --config-path=~/Desktop/only_config.yaml scan path -r .
)
A sample configuration file can be found at .gitguardian.example
# Exclude files and paths by globbing
paths-ignore:
- '**/README.md'
- 'doc/*'
- 'LICENSE'
# Ignore security incidents with the SHA256 of the occurrence obtained at output or the secret itself
matches-ignore:
- name:
match: 530e5a4a7ea00814db8845dd0cae5efaa4b974a3ce1c76d0384ba715248a5dc1
- name: credentials
match: MY_TEST_CREDENTIAL
show-secrets: false # default: false
# Set to true if the desired exit code for the CLI is always 0,
# otherwise the exit code will be 1 if incidents are found.
# the environment variable GITGUARDIAN_EXIT_ZERO=true can also be used toggle this behaviour.
exit-zero: false # default: false
# By default only secrets are detected. Use all-policies to toggle this behaviour.
all-policies: false # default: false
instance: https://api.gitguardian.com
verbose: false # default: false
Notes
Old configuration of matches-ignore
with list of secrets is
deprecated but still supported :
# Ignore security incidents with the SHA256 of the occurrence obtained at output or the secret itself
matches-ignore:
- 530e5a4a7ea00814db8845dd0cae5efaa4b974a3ce1c76d0384ba715248a5dc1
- MY_TEST_CREDENTIAL
ggshield can also be configured using environment variables.
Environment variables overrides settings set on your configuration file but are overridden by command line options.
At startup, ggshield attempts to load environment variables from different environment files in the following order:
- path pointed to by the environment variable
GITGUARDIAN_DOTENV_PATH
.env
at your current work directory..env
at the root of the current git directory
Only one file will be loaded of the three.
Reference of current environment variables supported by ggshield:
-
GITGUARDIAN_API_KEY
: API Key for the GitGuardian API. Use this if you don't want to use theggshield auth
commands. -
GITGUARDIAN_INSTANCE
: Custom URL of the GitGuardian dashboard. The API URL will be inferred from it. -
GITGUARDIAN_API_URL
: Custom URL for the scanning API. Deprecated, useGITGUARDIAN_INSTANCE
instead. -
GITGUARDIAN_DONT_LOAD_ENV
: If set to any value, environment variables won't be loaded from a file. -
GITGUARDIAN_DOTENV_PATH
: If set to a path, ggshield will attempt to load the environment from the specified file. -
GITGUARDIAN_TIMEOUT
: If set to a float,ggshield secret scan pre-receive
will timeout after the specified value. Set to 0 to disable the timeout. -
GITGUARDIAN_MAX_COMMITS_FOR_HOOK
: If set to an int,ggshield secret scan pre-receive
andggshield secret scan pre-push
will not scan more than the specified value of commits in a single scan. -
GITGUARDIAN_CRASH_LOG
: If set to True, ggshield will display a full traceback when crashing.
ggshield can be configured to run on your on-premise GitGuardian instance.
First, you need to point ggshield to your instance, by either defining the instance
key in your .gitguardian.yaml
configuration file or by defining the GITGUARDIAN_INSTANCE
environment variable.
Then, you need to authenticate against your instance, by either using the ggshield auth login --instance https://mygitguardianinstance.mycorp.local
command using the --instance
option or by obtaining an API key from your dashboard administrator and storing it in the GITGUARDIAN_API_KEY
environment variable.
By default ggshield ignores certain files and directories.
This list can be found in ggshield/core/utils.py under IGNORED_DEFAULT_PATTERNS
.
You can turn this feature with the flag --ignore-default-excludes
or the ignore-default-excludes
key in your .gitguardian.yaml
configuration file.
#.gitguardian.yml
# Use default excluded vendors folders
ignore-default-excludes: false # default: false
ggshield secret scan --ignore-default-excludes path example_file.md
You can also add custom patterns to ignore by using the --exclude
option or the paths-ignore
key in your .gitguardian.yaml
configuration file.
# .gitguardian.yml
# Exclude files and paths by globbing
paths-ignore:
- '**/README.md'
- 'doc/*'
- 'LICENSE'
ggshield secret scan --exclude dir/subdir/** path -r dir
Useful for ignoring a revoked test credential or a false positive, there are three ways to ignore a secret with ggshield:
⚠ this will also ignore the secret in the GitGuardian dashboard.
Secrets can be ignored in code by suffixing the line with a ggignore
comment.
Examples:
def send_to_notifier() -> int:
return send_slack_message(token="xoxb-23s2js9912ksk120wsjp") # ggignore
func main() {
high_entropy_test := "A@@E*JN#DK@OE@K(JEN@I@#)" // ggignore
}
⚠ Your secret will still show up on the GitGuardian dashboard as potential incident.
You can use the ignore command to ignore the last found secrets in your scan or directly add the ignore SHA that accompanies the incident or one of the secret matches to the configuration file
⚠ A secret ignored on the GitGuardian dashboard will still show as a potential incident on ggshield.
⚠ Your secret will still show up on the GitGuardian dashboard as potential incident.
You can ignore a detector using the CLI option -b
or --banlist-detector
or through the configuration:
Examples:
# .gitguardian.yaml
banlisted-detectors: # default: []
- Generic Password
- Generic High Entropy Secret
ggshield secret scan -b "Generic High Entropy Secret" path example_file.md
In order to use ggshield with the pre-commit framework, you need to do the following steps.
Make sure you have pre-commit installed:
$ pip install pre-commit
Create a .pre-commit-config.yaml
file in your root repository:
repos:
- repo: https://github.com/gitguardian/ggshield
rev: v1.12.0
hooks:
- id: ggshield
language_version: python3
stages: [commit]
Then install the hook with the command:
$ pre-commit install
pre-commit installed at .git/hooks/pre-commit
Now you're good to go!
If you want to skip the pre-commit check, you can add -n
parameter:
$ git commit -m "commit message" -n
Another way is to add SKIP=hook_id
before the command:
$ SKIP=ggshield git commit -m "commit message"
To install pre-commit globally (for all current and future repos), run the following command:
$ ggshield install --mode global
It will do the following:
- check if a global hook folder is defined in the global git configuration
- create the
~/.git/hooks
folder (if needed) - create a
pre-commit
file which will be executed before every commit - give executable access to this file
You can also install the hook locally on desired repositories. To do so, run:
$ ggshield install --mode local
If a pre-commit executable file already exists, it will not be overridden.
You can force override with the --force
option:
$ ggshield install --mode local --force
If you already have a pre-commit executable file and you want to use ggshield, all you need to do is to add this line in the file:
$ ggshield secret scan pre-commit
If you want to try pre-commit scanning through the docker image:
$ docker run -e GITGUARDIAN_API_KEY -v $(pwd):/data --rm gitguardian/ggshield ggshield secret scan pre-commit
Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY
environment variable of your project or development environment.
In case there are more than a 50 commits in a push the hook will be skipped.
The amount of commits to scan before skipping the hook can be configured by the key max-commits-for-hook
in
ggshield configuration file.
Pre-push hooks are executed just before git push
sends data to the remote host. It will pickup and scan the range of commits between the local ref and the origin ref.
If incidents are detected in this range the push will be cancelled.
In order to use ggshield with the pre-commit framework, you need to do the following steps.
Make sure you have pre-commit installed:
$ pip install pre-commit
Create a .pre-commit-config.yaml
file in your root repository:
repos:
- repo: https://github.com/gitguardian/ggshield
rev: main
hooks:
- id: ggshield-push
language_version: python3
stages: [push]
Then install the hook with the command:
$ pre-commit install --hook-type pre-push
pre-commit installed at .git/hooks/pre-push
To install the pre-push hook globally (for all current and future repos), you just need to execute the following command:
$ ggshield install --mode global -t pre-push
It will do the following:
- check if a global hook folder is defined in the global git configuration
- create the
~/.git/hooks
folder (if needed) - create a
pre-push
file which will be executed before every commit - give executable access to this file
You can also install the hook locally on desired repositories. You just need to go in the repository and execute:
$ ggshield install --mode local -t "pre-push"
If a pre-commit executable file already exists, it will not be overridden.
You can force override with the --force
option:
$ ggshield install --mode local --force -t "pre-push"
Or you can append to the existing pre-push
script with the --append
option:
$ ggshield install --mode local --force -t "pre-push"
Now you're good to go!
A pre-receive hook allows you to reject commits from being pushed to a git repository if they do not validate every check. Refer to our learning center for more information.
You can find ggshield's pre-receive hook samples in the doc/pre-receive.sample and doc/pre-receive-docker.sample.
ggshield's pre-receive hook can be skipped if the developer passes the option breakglass
to the git push.
For this setting to work the remote must have push options enabled. (git config receive.advertisePushOptions true
).
Examples:
$ git push -o breakglass
$ git push --push-option=breakglass
-
This pre-receive hook requires the host machine to have python>=3.8 and pip installed
-
Install ggshield from pip:
pip install ggshield
-
Copy
pre-receive.sample
to.git/hooks/pre-receive
or to your provider's git hook directory: -
Do not forget to
chmod +x .git/hooks/pre-receive
-
Either set an environment variable machine wide
GITGUARDIAN_API_KEY
or set it in the.git/hooks/pre-receive
as instructed in the sample file.
How do I add ignored matches and use a custom configuration in this pre-receive hook?
- Create a
gitguardian.yaml
somewhere in the system. An example configuration file is available here - Replace in the pre-receive hook
with:
ggshield secret scan pre-receive
ggshield -c <INSERT path to gitguardian.yaml> scan pre-receive
For the pre-receive hook to work, the directory where the repositories are stored must also be mounted on the container.
- This pre-receive hook requires the host machine to have docker installed.
- Copy pre-receive-docker.sample to
.git/hooks/pre-receive
- Do not forget to
chmod +x .git/hooks/pre-receive
- Either set an environment variable machine wide
GITGUARDIAN_API_KEY
or set it in the.git/hooks/pre-receive
as instructed in the sample file.
The ggshield docker scanning tool (ggshield secret scan docker
) is used to scan local docker images for secrets present in the image's creation process (dockerfile
and build arguments) and in the image's layers' filesystem.
If the image is not available locally on the user's machine, ggshield will attempt to pull the image using docker pull <IMAGE_NAME>
.
You may be interested in using GitGuardian's GitLab integration to ensure full coverage of your GitLab projects as well as full git history scans and reporting.
Configuring GitLab pipelines to use ggshield can be done by adding a step to your project's pipeline:
stages:
- scanning
🦉 gitguardian scan:
image: gitguardian/ggshield:latest
stage: scanning
script: ggshield secret scan ci
variables:
GIT_STRATEGY: clone
GIT_DEPTH: 0
Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY
environment variable in your project settings.
For ggshield to scan every commit in a merge request pipeline the CI must clone the full repository instead of just fetching the branch. The following snippet ensures this behavior.
variables:
GIT_STRATEGY: clone
GIT_DEPTH: 0
You may be interested in using GitGuardian's GitHub integration to ensure full coverage of your GitHub projects as well as full git history scans and reporting.
ggshield support for GitHub comes in the form of GitHub actions.
The action for this repository is hosted at ggshield-action.
Configuring a GitHub workflow to use ggshield can be done by adding a step to your project's workflow:
name: GitGuardian scan
on: [push, pull_request]
jobs:
scanning:
name: GitGuardian scan
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0 # fetch all history so multiple commits can be scanned
- name: GitGuardian scan
uses: GitGuardian/ggshield-action@master
env:
GITHUB_PUSH_BEFORE_SHA: ${{ github.event.before }}
GITHUB_PUSH_BASE_SHA: ${{ github.event.base }}
GITHUB_PULL_BASE_SHA: ${{ github.event.pull_request.base.sha }}
GITHUB_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
GITGUARDIAN_API_KEY: ${{ secrets.GITGUARDIAN_API_KEY }}
Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY
secret in your project settings.
⚠ BitBucket pipelines do not support commit ranges therefore only your latest commit in a pushed group or in a new branch will be scanned.
Configuring a BitBucket pipeline to use ggshield can be done by adding a step to your project's workflow:
pipelines:
default:
- step:
image: gitguardian/ggshield:latest
services:
- docker
script:
- ggshield secret scan ci
Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY
environment variable in your project settings.
Circle CI is supported in ggshield through ggshield-orb.
To add ggshield to your pipelines, configure your .circleci/config.yml
to add the ggshield orb:
orbs:
ggshield: gitguardian/ggshield
workflows:
main:
jobs:
- ggshield/scan:
name: ggshield-scan # best practice is to name each orb job
base_revision: << pipeline.git.base_revision >>
revision: <<pipeline.git.revision>>
Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY
environment variable in your project settings.
To add ggshield to your pipelines, configure your .travis.yml
to add a ggshield secret scanning job:
jobs:
include:
- name: GitGuardian Scan
language: python
python: 3.8
install:
- pip install ggshield
script:
- ggshield secret scan ci
Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY
environment variable in your project settings.
To add ggshield to your pipelines, configure your Jenkinsfile
to add a ggshield stage:
pipeline {
agent none
stages {
stage('GitGuardian Scan') {
agent {
docker { image 'gitguardian/ggshield:latest' }
}
environment {
GITGUARDIAN_API_KEY = credentials('gitguardian-api-key')
}
steps {
sh 'ggshield secret scan ci'
}
}
}
}
Do not forget to add your GitGuardian API Key to the gitguardian-api-key
credential in your project settings.
To add ggshield to your pipelines, configure your .drone.yml
to add a ggshield stage:
kind: pipeline
type: docker
name: default
steps:
- name: ggshield
image: gitguardian/ggshield:latest
commands:
- ggshield secret scan ci
Drone CI integration handles only pull-request or merge-request events, push events are not handled.
Do not forget to add your GitGuardian API Key to the GITGUARDIAN_API_KEY
environment variable for your Drone CI workers.
⚠ Azure Pipelines does not support commit ranges outside of GitHub Pull Requests, therefore on push events in a regular branch only your latest commit will be scanned. This limitation doesn't apply to GitHub Pull Requests where all the commits in the pull request will be scanned.
To add ggshield to your pipelines, configure your azure-pipelines.yml
to add a ggshield secret scanning job:
jobs:
- job: GitGuardianShield
pool:
vmImage: 'ubuntu-latest'
container:
image: gitguardian/ggshield:latest
options: -u 0
steps:
- script: ggshield secret scan ci
env:
GITGUARDIAN_API_KEY: $(gitguardianApiKey)
Do not forget to add your GitGuardian API Key to the gitguardianApiKey
secret variable in your pipeline settings.
If no secrets or policy breaks have been found, the exit code will be 0:
$ ggshield secret scan pre-commit
If a secret or other issue is found in your staged code or in your CI, you will have an alert giving you the type of policy break, the filename where the policy break has been found and a patch giving you the position of the policy break in the file:
$ ggshield secret scan pre-commit
🛡️ ⚔️ 🛡️ 2 policy breaks have been found in file production.rb
11 | config.paperclip_defaults = {
12 | :s3_credentials => {
13 | :bucket => "XXX",
14 | :access_key_id => "XXXXXXXXXXXXXXXXXXXX",
|_____AWS Keys_____|
15 | :secret_access_key => "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
|_______________AWS Keys_______________|
16 | }
17 | }
Lines that are too long are truncated to match the size of the terminal, unless the verbose mode is used (-v
or --verbose
).
ggshield is MIT licensed.