Cannot stop python client with Ctrl-C
yokoi-h opened this issue · 27 comments
I'm using grpc stream between a server process implemented by Golang and a client process in Python.
The client can receive values from the server with the stream. However when it comes to disconnecting from the client, it doesn't work well.
I made sample programs using the stream like below.
https://gist.github.com/yokoi-h/06ab4d802fa918884d38
Components in the sample program are as follows.
- client1(add_count.py): sends count to server
- server(sample_server.go): receives count and adds it to a variable inside
- client2(monitor_count.py): receives total count from the server
- client2(monitor_count.py) <----- server(sample_server.go) <---- client1(add_count.py)
- these components are running in a different terminal.
If client1 sends count to server, then client2 can receive total count via the server.
But when hit Ctrl-C to quit client2 in the terminal, it's blocked and not possible to quit except using kill.
I would like to know how to finish a client that is reading stream with Ctrl-C.
If you can provide the example to stop with Ctrl-C, it would be great.
Thank you
Thanks for the report. I see you're calling early_adopter_
methods in your code - what happens if you change those to beta_
? You may need to update to gRPC 0.11 or later and re-run protoc
.
Hi,
Thank you for your reply. I should have look at the latest tutorial.
I cloned gRPC git repository and checked out release-0_11_1 tag, and then compiled and installed in an Ubuntu 14.04 VM.
I tried modified script that uses beta_
, but the same problem occurred. I was not able to quit the script with Ctrl-C.
Is there any solution or measure to analyze for this?
Thank you
I'm also seeing this problem with my Python client. Here's my code https://gist.github.com/kyleconroy/9c2ba448d46f2ec93900
We are also seeing this problem (Python2.7, beta API) for both client and server. Client side is easier to trigger. It seems to have something to do with internal broken connection objects that lingered around and causing destructor to block wait forever.
@nathanielmanistaatgoogle what's the status of this issue?
@hsaliak I expect it to be resolved this quarter; likely before mid-May.
still not fixed in 0.14.0.
+1
Definitely still not fixed in HEAD.
@jaytaylor: while working on this we've been finding several causes of it rather than just one. Please go into more detail - platform, Python version, gRPC commit point, and if possible code to reproduce?
hi
I think this is easy to find the code to reproduce, I wrote the code here
GuoJing
http://guojing.me http://guojing.me/
The one who love coding.
--> ret <--
echo null
rm -rf /
在 2016年6月16日,下午11:02,Nathaniel Manista notifications@github.com 写道:
@jaytaylor https://github.com/jaytaylor: while working on this we've been finding several causes of it rather than just one. Please go into more detail - platform, Python version, gRPC commit point, and if possible code to reproduce?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub #3820 (comment), or mute the thread https://github.com/notifications/unsubscribe/AAWKQWtwyo_JuZY4QRCnfEWMp89dnBXVks5qMWV4gaJpZM4GOc70.
@kpayson64 Thanks! I can confirm that the indefinite hang-on-exit I've seen with gcloud-python
+ GAX
+ gRPC
is now gone, leaving in its place a much more benign bunch of logging spew. :)
D0628 16:18:07.661735081 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:08.663269688 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:09.664535323 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:10.665863923 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:11.667386620 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:12.668871835 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:13.670009823 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:14.671078765 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:15.672284821 10465 iomgr.c:99] Waiting for 3 iomgr objects to be destroyed
D0628 16:18:16.673620721 10465 iomgr.c:118] Failed to free 3 iomgr objects before shutdown deadline: memory leaks are likely
D0628 16:18:16.673701411 10465 iomgr.c:80] LEAKED OBJECT: tcp-client:ipv4:216.58.194.106:443 fd=6 0x7fea300045a8
D0628 16:18:16.673723755 10465 iomgr.c:80] LEAKED OBJECT: tcp-client:ipv4:216.58.218.138:443 fd=12 0x7fea30005168
D0628 16:18:16.673746282 10465 iomgr.c:80] LEAKED OBJECT: tcp-client:ipv4:216.58.194.42:443 fd=23 0x7fea30006888
D0628 16:18:16.673830891 10465 metadata.c:238] WARNING: 1 metadata elements were leaked
D0628 16:18:16.673872448 10465 metadata.c:238] WARNING: 3 metadata elements were leaked
D0628 16:18:16.673903482 10465 metadata.c:238] WARNING: 2 metadata elements were leaked
D0628 16:18:16.673927907 10465 metadata.c:238] WARNING: 2 metadata elements were leaked
D0628 16:18:16.673955019 10465 metadata.c:238] WARNING: 1 metadata elements were leaked
D0628 16:18:16.673982805 10465 metadata.c:238] WARNING: 9 metadata elements were leaked
D0628 16:18:16.674006559 10465 metadata.c:238] WARNING: 3 metadata elements were leaked
D0628 16:18:16.674031913 10465 metadata.c:238] WARNING: 3 metadata elements were leaked
D0628 16:18:16.674055336 10465 metadata.c:238] WARNING: 2 metadata elements were leaked
D0628 16:18:16.674124369 10465 metadata.c:238] WARNING: 2 metadata elements were leaked
D0628 16:18:16.674160625 10465 metadata.c:238] WARNING: 2 metadata elements were leaked
D0628 16:18:16.674188014 10465 metadata.c:238] WARNING: 3 metadata elements were leaked
D0628 16:18:16.674212776 10465 metadata.c:238] WARNING: 2 metadata elements were leaked
D0628 16:18:16.674313468 10465 metadata.c:238] WARNING: 7 metadata elements were leaked
D0628 16:18:16.674342804 10465 metadata.c:238] WARNING: 4 metadata elements were leaked
D0628 16:18:16.674363545 10465 metadata.c:238] WARNING: 2 metadata elements were leaked
D0628 16:18:16.674384519 10465 metadata.c:251] WARNING: 2 metadata strings were leaked
D0628 16:18:16.674404965 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:40 GMT
D0628 16:18:16.674425889 10465 metadata.c:255] LEAKED: 3
D0628 16:18:16.674449029 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.674470734 10465 metadata.c:255] LEAKED: 443:quic
D0628 16:18:16.674491956 10465 metadata.c:251] WARNING: 2 metadata strings were leaked
D0628 16:18:16.674518349 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:18:05 GMT
D0628 16:18:16.674541054 10465 metadata.c:255] LEAKED: quic=":443"; ma=2592000; v="34,33,32,31,30,29,28,27,26,25"
D0628 16:18:16.674566538 10465 metadata.c:251] WARNING: 5 metadata strings were leaked
D0628 16:18:16.674587736 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:55 GMT
D0628 16:18:16.674609252 10465 metadata.c:255] LEAKED: /google.logging.v2.ConfigServiceV2/CreateSink
D0628 16:18:16.674633129 10465 metadata.c:255] LEAKED: x-goog-api-client
D0628 16:18:16.674654447 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:35 GMT
D0628 16:18:16.674677346 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:29 GMT
D0628 16:18:16.674699780 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.674719762 10465 metadata.c:255] LEAKED: Requested entity was not found.
D0628 16:18:16.674741234 10465 metadata.c:251] WARNING: 2 metadata strings were leaked
D0628 16:18:16.674763658 10465 metadata.c:255] LEAKED: logging.googleapis.com:443
D0628 16:18:16.674786116 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:18:02 GMT
D0628 16:18:16.674809218 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.674831358 10465 metadata.c:255] LEAKED: 29600m
D0628 16:18:16.674853004 10465 metadata.c:251] WARNING: 3 metadata strings were leaked
D0628 16:18:16.674875729 10465 metadata.c:255] LEAKED: /google.logging.v2.LoggingServiceV2/ListLogEntries
D0628 16:18:16.674933951 10465 metadata.c:255] LEAKED: Bearer ya29.CjAPA_i1yKBinTNZ2b9wS5FgZElPA8fZG4Xg24nF9mkKcyVlTSNZvNGKGLQVKah_X0U
D0628 16:18:16.674954541 10465 metadata.c:255] LEAKED: 5
D0628 16:18:16.674974456 10465 metadata.c:251] WARNING: 2 metadata strings were leaked
D0628 16:18:16.674993382 10465 metadata.c:255] LEAKED: alt-svc
D0628 16:18:16.675010531 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:36 GMT
D0628 16:18:16.675029725 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675048026 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:56 GMT
D0628 16:18:16.675066608 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675084165 10465 metadata.c:255] LEAKED: 30S
D0628 16:18:16.675104658 10465 metadata.c:251] WARNING: 5 metadata strings were leaked
D0628 16:18:16.675123718 10465 metadata.c:255] LEAKED: alternate-protocol
D0628 16:18:16.675142083 10465 metadata.c:255] LEAKED: attachment
D0628 16:18:16.675159526 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:59 GMT
D0628 16:18:16.675177156 10465 metadata.c:255] LEAKED: Invalid resource: id is missing
D0628 16:18:16.675194886 10465 metadata.c:255] LEAKED: /google.logging.v2.MetricsServiceV2/ListLogMetrics
D0628 16:18:16.675217420 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675236510 10465 metadata.c:255] LEAKED: /google.logging.v2.MetricsServiceV2/CreateLogMetric
D0628 16:18:16.675259646 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675278980 10465 metadata.c:255] LEAKED: /google.logging.v2.LoggingServiceV2/WriteLogEntries
D0628 16:18:16.675306325 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675325605 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:58 GMT
D0628 16:18:16.675345407 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675364507 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:48 GMT
D0628 16:18:16.675384439 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675402924 10465 metadata.c:255] LEAKED: Requested metric was not found
D0628 16:18:16.675422288 10465 metadata.c:251] WARNING: 4 metadata strings were leaked
D0628 16:18:16.675440743 10465 metadata.c:255] LEAKED: /google.logging.v2.LoggingServiceV2/DeleteLog
D0628 16:18:16.675520524 10465 metadata.c:255] LEAKED: /google.logging.v2.MetricsServiceV2/UpdateLogMetric
D0628 16:18:16.675546900 10465 metadata.c:255] LEAKED: Bearer ya29.CjAPAzw3OKeovYMPda0Dx5OOoAtN8irv6v_WXnxPZqHU0Ryo2cs_gyxXbnVdnrUaGh8
D0628 16:18:16.675567006 10465 metadata.c:255] LEAKED: /google.logging.v2.ConfigServiceV2/GetSink
D0628 16:18:16.675588591 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675608092 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:18:00 GMT
D0628 16:18:16.675626798 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675644204 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:18:06 GMT
D0628 16:18:16.675663752 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675682391 10465 metadata.c:255] LEAKED: Python-gRPC-0.15.0 grpc-c/0.15.0 (manylinux; chttp2)
D0628 16:18:16.675703338 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675722238 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:31 GMT
D0628 16:18:16.675740482 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675758867 10465 metadata.c:255] LEAKED: /google.logging.v2.ConfigServiceV2/DeleteSink
D0628 16:18:16.675778098 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675794892 10465 metadata.c:255] LEAKED: /google.logging.v2.ConfigServiceV2/UpdateSink
D0628 16:18:16.675813321 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.675831682 10465 metadata.c:255] LEAKED: gax/0.12.1 gapic/0.1.0 gax/0.12.1 python/2.7.11
D0628 16:18:16.675852964 10465 metadata.c:251] WARNING: 6 metadata strings were leaked
D0628 16:18:16.675870778 10465 metadata.c:255] LEAKED: /google.logging.v2.ConfigServiceV2/ListSinks
D0628 16:18:16.675891221 10465 metadata.c:255] LEAKED: Requested sink was not found
D0628 16:18:16.675909202 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:18:01 GMT
D0628 16:18:16.675927607 10465 metadata.c:255] LEAKED: Bearer ya29.CjAPA5-BwfB1hVHRXhtfoCZPg6D-7EYUUxcVTSPm3hvnTOd3Z25uwnFbd1MSyH_Tz58
D0628 16:18:16.675945394 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:27 GMT
D0628 16:18:16.675963916 10465 metadata.c:255] LEAKED: /google.logging.v2.MetricsServiceV2/DeleteLogMetric
D0628 16:18:16.675984275 10465 metadata.c:251] WARNING: 2 metadata strings were leaked
D0628 16:18:16.676002156 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:53 GMT
D0628 16:18:16.676017818 10465 metadata.c:255] LEAKED: Tue, 28 Jun 2016 20:17:39 GMT
D0628 16:18:16.676033360 10465 metadata.c:251] WARNING: 1 metadata strings were leaked
D0628 16:18:16.676048260 10465 metadata.c:255] LEAKED: /google.logging.v2.MetricsServiceV2/GetLogMetric
Ctrl + C can not make the client to exit with grpcio 0.15.0
and it is not to appear all the time
Traceback (most recent call last):
File "greeter_client.py", line 70, in <module>
run()
File "greeter_client.py", line 61, in run
response = stub.SayHello.future(helloworld_pb2.HelloRequest(name='you'))
File "/home/xxxx/.virtualenvs/xxxx/local/lib/python2.7/site-packages/grpc/_channel.py", line 477, in future
request, timeout, metadata)
File "/home/xxx/.virtualenvs/xxxx/local/lib/python2.7/site-packages/grpc/_channel.py", line 440, in _prepare
state = _RPCState(_UNARY_UNARY_INITIAL_DUE, None, None, None, None)
File "/home/xxxx/.virtualenvs/xxxx/local/lib/python2.7/site-packages/grpc/_channel.py", line 109, in __init__
self.due = set(due)
KeyboardInterrupt
^C^CE0704 17:57:48.988460597 29195 chttp2_transport.c:1810] close_transport: {"created":"@1467626268.959884871","description":"Failed parsing HTTP/2","file":"src/core/ext/transport/chttp2/transport/chttp2_transport.c","file_line":1726,"referenced_errors":[{"created":"@1467626268.959881826","description":"EOF","file":"src/core/lib/iomgr/tcp_posix.c","file_line":233}]}
^C^C^C
@kpayson64 Is it still a issue on the master branch?
I'm still having this issue with grpcio-1.0.0rc1
.
@kpayson64 that actually looks exactly like the problem I'm having.
@minhaz1: as the person who wrote that leaving the problem in place is "highly unlikely to have any effect in the real world", I'd be curious to hear more about how significant an obstacle it is for you. Also what is it that has you using Python 2.7?
@nathanielmanistaatgoogle Just a friendly aside, which versions of Python is gRPC supported for? I thought 2.7 was in the set of supported versions.
@hsaliak Fair enough, but it would be nice to add another column to the Repository Structure & Status table, and listing the supported versions for each language.
@nathanielmanistaatgoogle what I am working on is going to be an internal library and consumers of the library will most likely be running as python 2.x. It isn't really a huge issue, as it does still handle other signals like SIGTERM correctly. It just made it inconvenient for testing things locally because I would have to Ctrl-Z + and kill %%. I just added a signal handler for my test client code to handle SIGINT and then send SIGTERM to the current process so I can Ctrl-C for local testing.
@minhaz1: good to know and thank you for sharing.
@minhaz1: you can just ctrl+\ to kill a process. If that doesnt work, try
ctrl+|
On Tue, 2 Aug 2016, 09:16 Paul Grosu, notifications@github.com wrote:
@hsaliak https://github.com/hsaliak Thank you - very appreciated.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#3820 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAkbBzgf2hbDAKqwvatJVoKHS0RpsJqOks5qbvzxgaJpZM4GOc70
.