/repository-scanner

Tool to detect secrets in source code management systems.

Primary LanguagePythonMIT LicenseMIT

        ______                     _ _                     _____
        | ___ \                   (_| |                   /  ___|
        | |_/ /___ _ __   ___  ___ _| |_ ___  _ __ _   _  \ `--.  ___ __ _ _ __  _ __   ___ _ __
        |    // _ | '_ \ / _ \/ __| | __/ _ \| '__| | | |  `--. \/ __/ _` | '_ \| '_ \ / _ | '__|
        | |\ |  __| |_) | (_) \__ | | || (_) | |  | |_| | /\__/ | (_| (_| | | | | | | |  __| |
        \_| \_\___| .__/ \___/|___|_|\__\___/|_|   \__, | \____/ \___\__,_|_| |_|_| |_|\___|_|
                  | |                               __/ |
                  |_|                              |___/

Repository Scanner

Maintainer License LaunchedDate LastUpdated Build Version Python TypeScript Vue.js Docker Kubernetes Helm Downloads DockerPulls OpenSSFBestPractices OpenSSF Scorecard SonarCloud

The Repository Scanner (RESC) is a tool used to detect secrets in source code management and version control systems (e.g. GitHub, BitBucket, or Azure DevOps). Among the types of secrets that the Repository Scanner detects are credentials, passwords, tokens, API keys, and certificates. The tool is maintained and updated by the ABN AMRO Bank to match the constantly changing cyber security landscape.

The Repository Scanner was created to prevent that credentials and other sensitive information are left unprotected in code repositories. Exposing sensitive information in such a way can have severe consequences for the security posture of an organization. An attacker can use the data to compromise the organization's network. This can be prevented by scanning a repository with the RESC tool. It marks all the instances of exposed sensitive information in the source code.

RESC-Demo

📒 Table of contents

🔗 Links

Throughout the process of open sourcing this project, the ABN AMRO Bank created a series of articles that describe the capabilities of the Repository Scanner (RESC) tool, the architectural decisions behind it, and the road to open sourcing RESC. With the articles, users can look "behind the scenes" and gain a deeper understanding of the tool.

ABN AMRO Open Source project: Repository Scanner
Open Source Project Update: Repository Scanner
Open Source Project Update: Repository Scanner 2.0.0

Releases

Every notable release of the Repository Scanner tool, the changes that come with the release, and the release date can be found on the Releases page.

🛠️ Technical information

The technologies that the Repository Scanner Tool is built on is listed below. There is also a list with direct links to the individual components of RESC.

  • Python
  • Docker
  • Kubernetes
  • Helm
  • Vue
  • RabbitMQ
  • Redis

RESC high-level overview

The diagram below gives a high-level overview of the Repository Scanner tool. All the different components of the tool and the technologies that it utilizes are explained in detail here. As shown in the diagram, all the components mentioned are run as Docker containers in a Kubernetes ecosystem.

  • RESC-Frontend: The RESC-Frontend is a fully responsive dashboard application developed using TypeScript, Vue 3 and the BootstrapVueNext framework (based on Bootstrap 5). It has screens for Analytics, Repositories, Scan Findings, Rule Analytics, and Rule Pack.
  • RESC-Backend: The RESC-Backend is the backend of the Repository Scanner tool. The RESC-Backend consists of RabbitMQ users and queue creation, Database models, the RESC Web service, and Alembic scripts for database migration. The RESC Web service is created using FASTAPI.
  • RESC-VCS-Scanner: RESC-VCS-Scanner, which runs as a celery worker, gathers repositories from the repositories queue and carries out a secret scan. Gitleaks is used as the scanner to find secrets.
  • RESC-VCS-Scraper: All projects and repositories from supported VCS providers such as Bitbucket, Azure Repos, and GitHub are gathered by the RESC-VCS-SCRAPER. This component contains the VCS-Scraper-Projects and VCS-Scraper-Repositories as its primary modules.

Please visit architecture.md for more information.

🏁 Getting started

Please refer resc-helm-wizard for an interactive and easy way to deploy RESC on a Kubernetes cluster.

🏎️ Key bindings

RESC comes with the following keybindings: keybindings

💁🏽 Contributing guidelines

We believe that innovating together can lead to the most incredible results and developments. Contributions to the Repository Scanner tool are therefore highly encouraged. We have created guidelines that we expect contributors to the project to follow. By contributing to the project you also agree with our Code of Conduct.

📧 Contact

If you need to get in touch with the maintainers of the Repository Scanner tool, please use the following e-mail address: resc@nl.abnamro.com.

⚖️ License

The Repository Scanner (RESC) Tool is licensed under the MIT License.

🎉 Acknowledgements

Since the Repository Scanner (RESC) makes use of GitLeaks, we want to give Zachary Rice credits for creating and maintaining GitLeaks. GitLeaks has helped many organizations in securing their codebases for any leaked secrets.