istio/old_issues_repo

Circuit breaker doesn't work for httpConsecutiveErrors

Opened this issue · 4 comments

jeson114 commented

Y
Is this a BUG or FEATURE REQUEST?:
BUG
Did you review https://istio.io/help/ and existing issues to identify if this is already solved or being worked on?:
Y
Bug:
Y/N
Y
What Version of Istio and Kubernetes are you using, where did you get Istio from, Installation details

istioctl version : Version: 0.7.1
kubectl version : Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.1", GitCommit:"d4ab47518836c750f9949b9e0d387f20fb92260b", GitTreeState:"clean", BuildDate:"2018-04-12T14:26:04Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.1", GitCommit:"d4ab47518836c750f9949b9e0d387f20fb92260b", GitTreeState:"clean", BuildDate:"2018-04-12T14:14:26Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Is Istio Auth enabled or not ?
Did you install the stable istio.yaml, istio-auth.yaml.... or if using the Helm chart please provide full command line input.
Y
What happened:
Circuit breaker doesn't trip on httpConsecutiveErrors: 1 (for 500 response). All requests pass through and give a 500 instead
What you expected to happen:
Circuit breaker should trip and should return 503(Service Unavailable). Inst
How to reproduce it:
Follow the steps Circuit breaker setup
Once httpbin is up you can simulate 500 with it
Request :
kubectl exec -it $FORTIO_POD -c fortio /usr/local/bin/fortio -- load -c 1 -qps 0 -n 20 -loglevel Warning http://httpbin:8000/status/500
Running this will give you simulate 20 requests returning 500 . But if you have applied the circuit breaker if should allow just the one request as 500 and remaining requests should be tripped and a 503 should be returned . This doesn't happen.

jeson114 commented

Can someone at least validate this ?

louiscryan commented

@ldemailly Does fortio recreate connections transparently for HTTP ?

ldemailly commented

@louiscryan fortio reports the exact number of sockets it had to open, if connections get closed in the middle, the number reported is not equal to the input -c connections
(e.g "SocketCount": 48 in https://fortio.istio.io/data/qps_max-s2_to_s1-0.7.1-2018-04-05-22-04.json )
also any http error is reported in status (so 500 vs 503 are reported exactly)

jeson114 commented

This is not fortio specific . I have encountered it with all east-west traffic while applying simplecb

  • 👍2