dmlc/ps-lite

When Finalize(), the server sent a ADD_NODE message again

formath opened this issue · 0 comments

When Finalize(), the server node sent a nessage of ADD_NODE again.
@mli

[[libprotobuf FATAL google/protobuf/message_lite.cc:224] CHECK failed: !coded_out.HadError():
19:55:28] src/van.cc:136: [1 => 9. Meta: request=0, timestamp=0, customer_id=0, simple_app=1, push=0, head=019:55:28
[19:55:28] ] src/van.cc:src/van.cc:161: 161: 8 => 1. Meta: request=1, timestamp=0, customer_id=0, simple_app=1, push=0, head=3
1 => 9. Meta: request=0, timestamp=0, customer_id=0, simple_app=1, push=0, head=0
[19:55:28] src/scheduler/scheduler.cc:29: scheduler received request type: 3
[19:55:28] src/van.cc:136: 1 => 8. Meta: request=0, timestamp=0, customer_id=0, simple_app=1, push=0, head=3
[19:55:28] src/scheduler/scheduler.cc:57: scheduler received SERVER_COMPLETE message. one server completed.
[19:55:28] src/van.cc:136: [19:55:28] src/van.cc:1 => 1. Meta: request=1, timestamp=6, control={ cmd=BARRIER, barrier_group=7 }161
: 1 => 8. Meta: request=0, timestamp=0, customer_id=0, simple_app=1, push=0, head=3
[19:55:28] src/van.cc:161: 1 => 1. Meta: request=1, timestamp=6, control={ cmd=BARRIER, barrier_group=7 }
[19:55:28] src/van.cc:294: Barrier count for 7 : 2
0 main 0x00000001023a0708 _Z7handlerv + 40
1 libc++abi.dylib 0x00007fff89578d69 _ZSt11__terminatePFvvE + 8
2 libc++abi.dylib 0x00007fff895787de _ZN10__cxxabiv1L22exception_cleanup_funcE19_Unwind_Reason_CodeP17_Unwind_Exception + 0
3 libprotobuf.8.dylib 0x00000001024dfa76 _ZN6google8protobuf8internal10LogMessage6FinishEv + 214
4 libprotobuf.8.dylib 0x00000001024ea886 _ZNK6google8protobuf11MessageLite31SerializeWithCachedSizesToArrayEPh + 150
5 libprotobuf.8.dylib 0x00000001024ead63 _ZNK6google8protobuf11MessageLite16SerializeToArrayEPvi + 51
6 main 0x00000001023abf34 _ZN2ps3Van8PackMetaERKNS_4MetaEPPcPi + 996
7 main 0x00000001023ade03 _ZN2ps6ZMQVan7SendMsgERKNS_7MessageE + 275
8 main 0x00000001023aa48f _ZN2ps3Van4SendERKNS_7MessageE + 47
9 main 0x00000001023a4049 _ZN2ps10Postoffice7BarrierEi + 601
10 main 0x00000001023a41db _ZN2ps10Postoffice8FinalizeEb + 27
11 main 0x00000001023a0af8 main + 872
12 libdyld.dylib 0x00007fff8a967255 start + 1
13 ??? 0x0000000000000002 0x0 + 2
0 main 0x00000001023a0708 _Z7handlerv + 40
1 libc++abi.dylib 0x00007fff89578d69 _ZSt11__terminatePFvvE + 8
2 libc++abi.dylib 0x00007fff89578de3 _ZSt9terminatev + 51
3 main 0x00000001023ae3c8 _ZN2ps3VanD2Ev + 104
4 main 0x00000001023ac5c4 _ZN2ps6ZMQVanD0Ev + 100
5 main 0x000000010237fb0b _ZN2ps10PostofficeD2Ev + 27
6 libsystem_c.dylib 0x00007fff8a9fc178 __cxa_finalize_ranges + 332
7 libsystem_c.dylib 0x00007fff8a9fc4b2 exit + 55
8 main 0x00000001023a0788 _Z7handlerv + 168
9 libc++abi.dylib 0x00007fff89578d69 _ZSt11__terminatePFvvE + 8
10 libc++abi.dylib 0x00007fff895787de _ZN10__cxxabiv1L22exception_cleanup_funcE19_Unwind_Reason_CodeP17_Unwind_Exception + 0
11 libprotobuf.8.dylib 0x00000001024dfa76 _ZN6google8protobuf8internal10LogMessage6FinishEv + 214
12 libprotobuf.8.dylib 0x00000001024ea886 _ZNK6google8protobuf11MessageLite31SerializeWithCachedSizesToArrayEPh + 150
13 libprotobuf.8.dylib 0x00000001024ead63 _ZNK6google8protobuf11MessageLite16SerializeToArrayEPvi + 51
14 main 0x00000001023abf34 _ZN2ps3Van8PackMetaERKNS_4MetaEPPcPi + 996
15 main 0x00000001023ade03 _ZN2ps6ZMQVan7SendMsgERKNS_7MessageE + 275
16 main 0x00000001023aa48f _ZN2ps3Van4SendERKNS_7MessageE + 47
17 main 0x00000001023a4049 _ZN2ps10Postoffice7BarrierEi + 601
18 main 0x00000001023a41db _ZN2ps10Postoffice8FinalizeEb + 27
19 main 0x00000001023a0af8 main + 872
[19:55:28] src/van.cc:74: Bind to role=server, ip=xxx, port=56225, is_recovery=0
[19:55:28] src/van.cc:136: ? => 1. Meta: request=0, timestamp=0, control={ cmd=ADD_NODE, node={ role=server, ip=xxx, port=56225, is_recovery=0 } }
[19:55:28] src/van.cc:161: ? => 1. Meta: request=0, timestamp=0, control={ cmd=ADD_NODE, node={ role=server, ip=xxx, port=56225, is_recovery=0 } }