hmgle/graftcp

graftcp bash for GitLab CI pipeline .gitlab-ci.yml image Dockerfile ENTRYPOINT

Closed this issue · 14 comments

freidenker commented

my proxy server is a squid proxy server.

failed at graftcp crane pull gcr.io/kaniko-project/executor:v1.9.0-debug kaniko.tar -v
only failed on pulling gcr.io images, works on other registries

root@18f481e9c3e5:/opt# crane version
0.14.0
https://github.com/google/go-containerregistry
root@18f481e9c3e5:/opt# graftcp --version
graftcp v0.4

root@18f481e9c3e5:/opt# cat /etc/graftcp-local/graftcp-local.conf

## graftcp-local configuation

## Listen address (default ":2233")
listen = :2233

## Write logs to file, to stdout if empty
# logfile = graftcp-local.log

## Log level (0-6), 0: debug, 1: info, 2: notice, 3: warn, 4: error,
## 5: critical: 6: fatal
loglevel = 1

## Pipe path for graftcp to send address info (default "/tmp/graftcplocal.fifo")
# pipepath = /tmp/graftcplocal.fifo

## SOCKS5 address (default "127.0.0.1:1080")
# socks5 = 127.0.0.1:1080

## SOCKS5 proxy username (default "")
# socks5_username = SOCKS5USERNAME

## SOCKS5 proxy password (default "")
# socks5_password = SOCKS5PASSWORD

## HTTP proxy address (default "")
### it's a squid http proxy server.
http_proxy = my_proxy:3128
https_proxy = my_proxy:3128
## Set the mode for select a proxy (default "auto")
## "auto": select socks5 if socks5 is reachable, else HTTP proxy if HTTP proxy
##  is rechable, else direct.
## "random": select the reachable proxy randomly.
## "only_http_proxy": only use http proxy.
## "only_socks5": only use socks5 proxy.
## "direct": direct connect.
# select_proxy_mode = only_socks5

## Use the system logger (syslog on Unix, Event Log on Windows)
# use_syslog = true

the error log:
root@18f481e9c3e5:/opt# graftcp crane pull gcr.io/kaniko-project/executor:v1.9.0-debug kaniko.tar -v

2023/05/10 07:53:54 --> GET https://gcr.io/v2/
2023/05/10 07:53:54 GET /v2/ HTTP/1.1
Host: gcr.io
User-Agent: crane/0.14.0 go-containerregistry/0.14.0
Accept-Encoding: gzip


2023/05/10 07:54:04 <-- net/http: TLS handshake timeout GET https://gcr.io/v2/ (10.020513479s)
2023/05/10 07:54:04 retrying net/http: TLS handshake timeout
2023/05/10 07:54:04 --> GET https://gcr.io/v2/
2023/05/10 07:54:04 GET /v2/ HTTP/1.1
Host: gcr.io
User-Agent: crane/0.14.0 go-containerregistry/0.14.0
Accept-Encoding: gzip
hmgle commented

I can't reproduce it on my machine.

You may check the graftcp-local log, or check the proxy server is working.

freidenker commented

I can't reproduce it on my machine.

You may check the graftcp-local log, or check the proxy server is working.

not sure if it was related with the GFW. i have installed a squid http proxy server in aws us-west-2 region, then used graftcp in shanghai local machine to pull gcr images. Also failed pulling gcr images with ss proxy. have any suggestions?

hmgle commented

You may check the proxy server is working: curl -x http://my_proxy:3128 http://www.google.com/.

freidenker commented

You may check the proxy server is working: curl -x http://my_proxy:3128 http://www.google.com/.

hi, looks like find the root cause, actullay i run graftcp-local service in a docker container, in a gitlab runner with a docker container. meanwile the ss5 server didn't get the requests log. but mgraftcp is ok. so another question: does graftcp-local work in container?

hmgle commented

You may check the proxy server is working: curl -x http://my_proxy:3128 http://www.google.com/.

hi, looks like find the root cause, actullay i run graftcp-local service in a docker container, in a gitlab runner with a docker container. meanwile the ss5 server didn't get the requests log. but mgraftcp is ok. so another question: does graftcp-local work in container?

Are graftcp and graftcp-local in the same container? They communicate using named pipes, and graftcp-local needs to access /proc to determine information related to graftcp. They need to be in the same environment to work properly.

freidenker commented

You may check the proxy server is working: curl -x http://my_proxy:3128 http://www.google.com/.

hi, looks like find the root cause, actullay i run graftcp-local service in a docker container, in a gitlab runner with a docker container. meanwile the ss5 server didn't get the requests log. but mgraftcp is ok. so another question: does graftcp-local work in container?

Are graftcp and graftcp-local in the same container? They communicate using named pipes, and graftcp-local needs to access /proc to determine information related to graftcp. They need to be in the same environment to work properly.

yes, they were in same container.

freidenker commented
  1. graftcp-local service issue got fixed, the parameter -config does not work, it always use default configuration in /etc/graftcp-local/graftcp-local.conf.
  2. on ubuntu system systemd-resolved service listen tcp 127.0.0.53:53 port to resolve dns, and aws ec2 also have a internal dns service, i added these 2 IPs into graftcp parameter --blackip-file to ignore them. like 10.16.0.2 127.0.0.53
freidenker commented

and graftcp bash still not work in docker container

hmgle commented

  1. graftcp-local service issue got fixed, the parameter -config does not work, it always use default configuration in /etc/graftcp-local/graftcp-local.conf.

    1. on ubuntu system systemd-resolved service listen tcp 127.0.0.53:53 port to resolve dns, and aws ec2 also have a internal dns service, i added these 2 IPs into graftcp parameter --blackip-file to ignore them. like 10.16.0.2 127.0.0.53
  1. How do you start graftcp-local service? If it run as a service, please check the it's systemd-unit service conf.
  2. This tool does not have the ability to affect the dns.
hmgle commented

and graftcp bash still not work in docker container

Please check if your docker supports ptrace.

freidenker commented

  1. graftcp-local service issue got fixed, the parameter -config does not work, it always use default configuration in /etc/graftcp-local/graftcp-local.conf.

    1. on ubuntu system systemd-resolved service listen tcp 127.0.0.53:53 port to resolve dns, and aws ec2 also have a internal dns service, i added these 2 IPs into graftcp parameter --blackip-file to ignore them. like 10.16.0.2 127.0.0.53
  1. How do you start graftcp-local service? If it run as a service, please check the it's systemd-unit service conf.
  2. This tool does not have the ability to affect the dns.

  1. graftcp-local service issue got fixed, the parameter -config does not work, it always use default configuration in /etc/graftcp-local/graftcp-local.conf.

    1. on ubuntu system systemd-resolved service listen tcp 127.0.0.53:53 port to resolve dns, and aws ec2 also have a internal dns service, i added these 2 IPs into graftcp parameter --blackip-file to ignore them. like 10.16.0.2 127.0.0.53
  1. How do you start graftcp-local service? If it run as a service, please check the it's systemd-unit service conf.
  2. This tool does not have the ability to affect the dns.

graftcp-local -service install & graftcp-local -service start. right, systemd-unit service conf is always /etc/graftcp-local/graftcp-local.conf.

freidenker commented

and graftcp bash still not work in docker container

Please check if your docker supports ptrace.

for GitLab CICD pipeline, we can define that image's Dockerfile to implement graftcp bash.
ENTRYPOINT ["/bin/bash", "-c", "graftcp-local -service start; /opt/graftcp-blackip.sh; graftcp --blackip-file=/opt/blackips bash"]

just changed the issues title, to the users of graftcp with similar issues.

freidenker commented

thanks @hmgle for your help.
the --blackip-file looks does not support CIDR, right?

hmgle commented

thanks @hmgle for your help. the --blackip-file looks does not support CIDR, right?

Not supported.