Abuse Detector

This application is for preventing phishing sites are created on Studio.

Requirements

Go 1.14.9 >=
Docker 2.4.0.0 >=
Docker Compose 1.27.4 >=

Usage

Request verification

This API verifies if the site does not include malicious links, such as fishing.

http://localhost:3000/verify?url=https://www.google.com/

The response would look like below if the site is not malicious.

{
    "strategyName": "TransparencyReportVerifyStrategy",
    "link": ["https://www.google.com/"],
    "malicious": false,
    "statusCode": 200,
    "error": {
        "Name": "google-chrome",
        "Err": {}
    }
}

If it's malicious, the response looks like below.

{
    "strategyName": "TransparencyReportVerifyStrategy",
    "link": ["https://zonabn1segura-pe.com/"],
    "malicious": true,
    "statusCode": 200,
    "error": null
}

How to build

go build *.go

How to run production

Create .env based off from .env.default. For API keys required, please refer documents below in this README.
Set API Keys accordingly. For API keys required, please refer documents below in this README.
Run command below.
```
docker-compose up
```
For the initial start, run as below.
```
docker-compose up --build
```

How to run for development

Create .env based off from .env.default. For API keys required, please refer documents below in this README.
Set API Keys accordingly. For API keys required, please refer documents below in this README.
Comment out CMD ["./app"] and remove comment of CMD [ "realize", "start" ] instead to enable realize for hot reloading.
Run command below.
```
docker-compose up
```
For the initial start, run as below.
```
docker-compose up --build
```

How to run for debugging with IDE, such as Goland

Create .env based off from .env.default. For API keys required, please refer documents below in this README.
Configure COMMON_APP_ENV=, no strings. (Default should be production)
Set API Keys accordingly. For API keys required, please refer documents below in this README.
Comment out CMD ["./app"] and remove comment of CMD [ "realize", "start" ] instead to enable realize for hot reloading.

Spin up servers as below

Spin up chrome headless server

docker run -d -p 9222:9222 --rm --name headless-shell --shm-size 2G chromedp/headless-shell

Then, right click main.go and debug run on Goland IDE.

How to run all tests

go test -v -race -run=. -bench=. ./...

How to build Docker image

This is how to build and confirm the image is built correctly.

docker build -t studio-abuse-detector .
docker run -p 3000:3000 -d --name studio-abuse-detector studio-abuse-detector:latest
curl localhost:3000

Opearation Related

How to remove all images including running

docker rm -f `docker ps -qa`

How to access an image

docker-compose exec app /bin/sh

Appendix

How to get API key for urlscan.io

Go to https://urlscan.io/ and create an account.
Go to Settings & API and create an API Key
Copy the Key and set it to URLSCAN_API_KEY in the .env file

How to get API key for Google Safe Browsing API

Access to Google API Console and create a project
Create API key in the project.
Look for Google Safe Browsing API in Liberary tab and add it for the API Key created.
Copy the Key and set it to GOOGLE_SAFE_BROWSING_API_KEY in the .env file

Caveat

Chrome Headless server in use may need load balancing for a more massive load of access.
Test links are real phishing sites for now. They become offline or removed in the short term, so tests highly likely to fail.

yasuflatland-lf/abuse-detector