Integration tests failing behind a http_proxy

[rakrup]

What version of Cloud Foundry and CF CLI are you using? (i.e. What is the output of running cf curl /v2/info && cf version?

v2.65.0

What version of the buildpack you are using?

Python-buildpack v1.5.24

If you were attempting to accomplish a task, what was it you were attempting to do?

Trying to run the machete tests with a offline buildpack to make sure all the tests are passing.
The setup is using a offline buildpack and is behind a http-proxy.

What did you expect to happen?

Expect all the integration tests to pass.

What was the actual behavior?

If I try to run the tests without sourcing the proxy in the env. Some of the tests fail because pip fails to fetch the app dependencies from the internet (pip can't get to the proxy).

If I try to run the tests after sourcing the proxy (and setting the correct no_proxy settings) - some of the test cases which depend of machete's browser emulator fail because it sends all the app GET requests to the proxy as well and proxy fails to resolve the DNS because CF is running internally.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/151057323

The labels on this github issue will be updated when the story is started.

[dgodd]

@rakrup Can I confirm where the proxy is required. I think you are running the buildpack integration tests on a machine which needs to use a http proxy to access the CF cluster, and it is that hop which is failing? Or is http proxy failing somewhere else?

See also: cloudfoundry/python-buildpack#85

[sclevine]

@rakrup Please let us know what your value forno_proxy looks like in this issue.

[rakrup]

The proxy value are in the lines of
http_proxy=myproxy.mydomain.com:8080 https_proxy=myproxy.mydomain.com:443 no_proxy='.mylab.mydomain.com'
My CF has a FQDN of lets say 'mycf.mylab.mydomain.com'.

So, I could disable the proxy as well and run the tests (but thats not documented that way; I could add it once we confirm this). If my apps have dependencies to fetch from the internet they need to go via the proxy which can be set using cf set-env per app basis. But how do we do it for running machete tests which need proxy (add it in text fixtures ?).

Thanks,
Rahul.

[rakrup]

@sclevine @dgodd . Hey Stephen, Dave any help on this one ?
The tests just run fine on my laptop which is not behind a proxy. But in a lab environment which is behind a lab proxy the tests fail for me.

One of my questions is
The failure points can be many - ASGs or env_proxy not being exported by the app, etc . I wanted to understand if machete today supports exporting proxy to the apps that would stage. Tried to look in a bit but this is my first ruby project that I am looking at so might be missing out on few things. Wanted to understand if machete supports proxy for staging apps ?

Also, the initial issue which I mentioned while opening this issue still remains.
like - if I export the http_proxy and no_proxy both for my network. the browser component in machete. reads the http_proxy but doesn't read the no_proxy and hence while reading response from apps it spawns it send the request to the proxy which is unable to resolve the FQDN on the apps. Ideally no_proxy was exported so that for specific requests the request is not forwarded to the proxy.

Appreciate any help on this one. Would love to contribute back to this if you can guide me on how /what to fix.

Thanks,
Rahul.

[sclevine]

Hi Rahul,

We suspect that some gem used by machete fails to properly respect no_proxy. We can address this when priority allows. Possibly worth noting: machete has been deprecated in favor of the libbuildpack/cutlass package. The Python buildpack will switch from machete to cutlass in the near future.

[rakrup]

Thanks @sclevine . Started working with build packs only lately. Since saw machete as the common framework around many of the build packs thought it is the worth fixing it. Didn't know about the deprecation decision around it. Any good reference around build pack/ test/ci roadmap ?

[sclevine]

Since the Python buildpack uses cutlass instead of machete now, and we plan to phase out machete in the long run, do you still need us to investigate this?

[sclevine]

Closing due to inactivity, and because the Python buildpack no longer uses machete.