[Bug]: kudu写Doris丢数据
Closed this issue · 14 comments
What happened?
将kudu写入Doris丢数据
Version
4.2.0 (Default)
OS Type
No response
Java JDK Version
Oracle JDK 1.8.0
Relevant log output
2024-11-01 13:53:52.071 [ main] INFO Engine -
___ _ _
/ _ \ | | | |
/ /_\ \ __| | __| | __ ___ __
| _ |/ _` |/ _` |/ _` \ \/ /
| | | | (_| | (_| | (_| |> <
\_| |_/\__,_|\__,_|\__,_/_/\_\
:: Addax version :: (v4.2.2-SNAPSHOT)
2024-11-01 13:53:52.323 [ main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2024-11-01 13:53:52.339 [ main] INFO Engine -
{
"setting":{
"speed":{
"byte":-1,
"channel":1
},
"errorLimit":{
"record":0,
"percentage":0
}
},
"content":{
"reader":{
"name":"kudureader",
"parameter":{
"masterAddress":"idc-cdh-119-73:7051,idc-cdh-119-74:7051,idc-cdh-119-75:7051,idc-cdh-119-80:7051,idc-cdh-119-81:7051",
"table":"impala::kududb.b2b_uorders_orderdetail",
"splitPk":"id",
"lowerBound":0,
"upperBound":100000000,
"column":[
"aquantity",
"asalepaymoney",
"cactivities",
"cbizname",
"ccreator",
"cerpcode",
"cmasterunitmeasure",
"cmemo",
"cmodifier",
"cmutualactivities",
"corderno",
"corderproducttype",
"coutsyskey",
"cparentsuitekey",
"cproductauxunitname",
"cproductcode",
"cproductname",
"cproductoutsyskey",
"cproductunitname",
"cprojectid",
"cskucode",
"cskuoutsyskey",
"cspecdescription",
"cstockname",
"csuitekey",
"ctaxid",
"dconsigntime",
"dcreated",
"dcreatedtime",
"dmodifytime",
"fauxexchangerate",
"fmainexchangerate",
"fmarketpricie",
"fmastermeasurequantity",
"fordrealmoney",
"foutboundqty",
"fparticularlymoney",
"fpointsmoney",
"fpromotionmoney",
"frebatemoney",
"freturnpresendqty",
"freturnqty",
"fsalecost",
"fsalepaymoney",
"fsaleprice",
"ftaxrate",
"ftransactionprice",
"iauxunitquantity",
"ibizid",
"ibizproductid",
"ibizskuid",
"iconversionrate",
"icorpid",
"id",
"ideleted",
"idkey",
"igroupid",
"igroupindex",
"imasterunitid",
"iorderdetailpriceid",
"iorderid",
"iorgid",
"iproductauxunitid",
"iproductid",
"iproductunitid",
"iquantity",
"irebatereturnproductid",
"isaledepartmentid",
"isaleorgid",
"isalesorgid",
"isendquantity",
"isettlementorgid",
"ishoppingcartid",
"iskuid",
"istockid",
"istockorgid",
"itransactiontypeid",
"pubuts",
"tquantity",
"tsalepaymoney",
"version",
"ytenant_id"
],
"readTimeout":30,
"scanTimeout":60,
"where":[
"dcreated > 2020-05-01 00:00:00",
"dcreated < 2020-06-01 00:00:00"
]
}
},
"writer":{
"name":"doriswriter",
"parameter":{
"loadUrl":[
"10.96.119.181:8001",
"10.96.119.183:8001",
"10.96.119.184:8001"
],
"writeMode":"insert",
"username":"bigdata_job_load",
"password":"*****",
"loadProps":{
"format":"json",
"strip_outer_array":"true",
"read_json_by_line":"true"
},
"batchSize":100000,
"column":[
"aquantity",
"asalepaymoney",
"cactivities",
"cbizname",
"ccreator",
"cerpcode",
"cmasterunitmeasure",
"cmemo",
"cmodifier",
"cmutualactivities",
"corderno",
"corderproducttype",
"coutsyskey",
"cparentsuitekey",
"cproductauxunitname",
"cproductcode",
"cproductname",
"cproductoutsyskey",
"cproductunitname",
"cprojectid",
"cskucode",
"cskuoutsyskey",
"cspecdescription",
"cstockname",
"csuitekey",
"ctaxid",
"dconsigntime",
"dcreated",
"dcreatedtime",
"dmodifytime",
"fauxexchangerate",
"fmainexchangerate",
"fmarketpricie",
"fmastermeasurequantity",
"fordrealmoney",
"foutboundqty",
"fparticularlymoney",
"fpointsmoney",
"fpromotionmoney",
"frebatemoney",
"freturnpresendqty",
"freturnqty",
"fsalecost",
"fsalepaymoney",
"fsaleprice",
"ftaxrate",
"ftransactionprice",
"iauxunitquantity",
"ibizid",
"ibizproductid",
"ibizskuid",
"iconversionrate",
"icorpid",
"id",
"ideleted",
"idkey",
"igroupid",
"igroupindex",
"imasterunitid",
"iorderdetailpriceid",
"iorderid",
"iorgid",
"iproductauxunitid",
"iproductid",
"iproductunitid",
"iquantity",
"irebatereturnproductid",
"isaledepartmentid",
"isaleorgid",
"isalesorgid",
"isendquantity",
"isettlementorgid",
"ishoppingcartid",
"iskuid",
"istockid",
"istockorgid",
"itransactiontypeid",
"pubuts",
"tquantity",
"tsalepaymoney",
"version",
"ytenant_id"
],
"session":[
"set session sql_mode='ANSI'"
],
"connection":{
"selectedDatabase":"dshisdatadb",
"database":"dshisdatadb",
"jdbcUrl":"jdbc:mysql://bd-doris-test.pec.com.cn:9030/dstestdb?useUnicode=true&useSSL=false",
"table":[
"b2b_uorders_orderdetail"
]
}
}
}
}
}
2024-11-01 13:53:52.362 [ main] INFO JobContainer - The jobContainer begins to process the job.
2024-11-01 13:53:52.380 [ job-0] INFO JobContainer - The Reader.Job [kudureader] perform prepare work .
2024-11-01 13:53:52.397 [ job-0] INFO JobContainer - The Writer.Job [doriswriter] perform prepare work .
2024-11-01 13:53:52.816 [ job-0] INFO JobContainer - Job set Channel-Number to 1 channel(s).
2024-11-01 13:53:52.817 [ job-0] INFO JobContainer - The Reader.Job [kudureader] is divided into [1] task(s).
2024-11-01 13:53:52.817 [ job-0] INFO JobContainer - The Writer.Job [doriswriter] is divided into [1] task(s).
2024-11-01 13:53:52.866 [ job-0] INFO JobContainer - The Scheduler launches [1] taskGroup(s).
2024-11-01 13:53:52.874 [ taskGroup-0] INFO TaskGroupContainer - The taskGroupId=[0] started [1] channels for [1] tasks.
2024-11-01 13:53:52.877 [ taskGroup-0] INFO Channel - The Channel set byte_speed_limit to -1, No bps activated.
2024-11-01 13:53:52.877 [ taskGroup-0] INFO Channel - The Channel set record_speed_limit to -1, No tps activated.
2024-11-01 13:53:55.946 [Doris-interval-flush] INFO DorisWriterManager - Doris interval Sinking triggered: label[addax_doris_writer_a29c631f-31ad-4b24-9a5d-9ca4db19712f].
2024-11-01 13:53:56.015 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[15328] bytes[20742816] label[addax_doris_writer_a29c631f-31ad-4b24-9a5d-9ca4db19712f].
2024-11-01 13:53:56.033 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '20758145'
2024-11-01 13:53:57.341 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_a29c631f-31ad-4b24-9a5d-9ca4db19712f].
2024-11-01 13:54:00.342 [Doris-interval-flush] INFO DorisWriterManager - Doris interval Sinking triggered: label[addax_doris_writer_4855c4a6-47a9-48c1-a059-601447146eb1].
2024-11-01 13:54:00.344 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[78310] bytes[106962410] label[addax_doris_writer_4855c4a6-47a9-48c1-a059-601447146eb1].
2024-11-01 13:54:00.828 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '107040721'
2024-11-01 13:54:04.136 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_4855c4a6-47a9-48c1-a059-601447146eb1].
2024-11-01 13:54:05.020 [ job-0] INFO StandAloneJobContainerCommunicator - Total 120800 records, 40781252 bytes | Speed 3.24MB/s, 10066 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 6.486s | Percentage 0.00%
2024-11-01 13:54:06.715 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136293291] label[addax_doris_writer_5e609a2f-53be-458c-98e3-8d153494b7e0].
2024-11-01 13:54:06.817 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136393292'
2024-11-01 13:54:12.363 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_5e609a2f-53be-458c-98e3-8d153494b7e0].
2024-11-01 13:54:12.364 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136352720] label[addax_doris_writer_b643dd52-2756-4d2c-86dd-f9be3adfb8b5].
2024-11-01 13:54:12.454 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136452721'
2024-11-01 13:54:16.118 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_b643dd52-2756-4d2c-86dd-f9be3adfb8b5].
2024-11-01 13:54:16.964 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136840276] label[addax_doris_writer_a5e94622-bcbb-4125-a0e8-8c671b319660].
2024-11-01 13:54:17.091 [ job-0] INFO StandAloneJobContainerCommunicator - Total 310848 records, 105015704 bytes | Speed 5.10MB/s, 15837 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 12.090s | Percentage 0.00%
2024-11-01 13:54:17.116 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136940277'
2024-11-01 13:54:22.210 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_a5e94622-bcbb-4125-a0e8-8c671b319660].
2024-11-01 13:54:24.233 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136479105] label[addax_doris_writer_1ca725b1-acf6-45ce-9651-6c86b1a75488].
2024-11-01 13:54:24.332 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136579106'
2024-11-01 13:54:28.506 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_1ca725b1-acf6-45ce-9651-6c86b1a75488].
2024-11-01 13:54:29.095 [ job-0] INFO StandAloneJobContainerCommunicator - Total 475168 records, 160615326 bytes | Speed 4.42MB/s, 13693 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 18.256s | Percentage 0.00%
2024-11-01 13:54:30.277 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136441189] label[addax_doris_writer_c43e9f22-bdf8-4368-855e-efe5d6a527d2].
2024-11-01 13:54:30.354 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136541190'
2024-11-01 13:54:33.490 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_c43e9f22-bdf8-4368-855e-efe5d6a527d2].
2024-11-01 13:54:34.940 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136455770] label[addax_doris_writer_c428a868-d7dd-4354-aab3-7a33c55d306e].
2024-11-01 13:54:35.140 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136555771'
2024-11-01 13:54:38.130 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_c428a868-d7dd-4354-aab3-7a33c55d306e].
2024-11-01 13:54:39.581 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136689051] label[addax_doris_writer_8ab07a3a-9bc0-4b2b-a30a-399d8fd29c4d].
2024-11-01 13:54:39.657 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136789052'
2024-11-01 13:54:41.100 [ job-0] INFO StandAloneJobContainerCommunicator - Total 654400 records, 221253456 bytes | Speed 4.82MB/s, 14936 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 24.194s | Percentage 0.00%
2024-11-01 13:54:42.646 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_8ab07a3a-9bc0-4b2b-a30a-399d8fd29c4d].
2024-11-01 13:54:44.430 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136015297] label[addax_doris_writer_da32b20c-b1d5-4a2e-b60d-5a4ef27ba195].
2024-11-01 13:54:44.504 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136115298'
2024-11-01 13:54:48.314 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_da32b20c-b1d5-4a2e-b60d-5a4ef27ba195].
2024-11-01 13:54:49.530 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[137452067] label[addax_doris_writer_36d80e57-f536-4da6-a79e-5895388bf4fa].
2024-11-01 13:54:49.619 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '137552068'
2024-11-01 13:54:52.705 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_36d80e57-f536-4da6-a79e-5895388bf4fa].
2024-11-01 13:54:53.103 [ job-0] INFO StandAloneJobContainerCommunicator - Total 867392 records, 293099021 bytes | Speed 5.71MB/s, 17749 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 30.138s | Percentage 0.00%
2024-11-01 13:54:55.017 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136384699] label[addax_doris_writer_7caa92fd-f0de-4b9c-827c-037032901d2b].
2024-11-01 13:54:55.109 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136484700'
2024-11-01 13:54:58.720 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_7caa92fd-f0de-4b9c-827c-037032901d2b].
2024-11-01 13:55:01.313 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[137248054] label[addax_doris_writer_c8c00f4c-4e73-47c1-a0fa-5afb8ba3576b].
2024-11-01 13:55:01.387 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '137348055'
2024-11-01 13:55:04.513 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_c8c00f4c-4e73-47c1-a0fa-5afb8ba3576b].
2024-11-01 13:55:05.416 [ job-0] INFO StandAloneJobContainerCommunicator - Total 1228704 records, 415490000 bytes | Speed 9.73MB/s, 30109 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 42.348s | Percentage 0.00%
2024-11-01 13:55:06.335 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136655725] label[addax_doris_writer_5b9f6d73-48e3-4bbf-9535-44cea7e550ca].
2024-11-01 13:55:06.532 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136755726'
2024-11-01 13:55:09.618 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_5b9f6d73-48e3-4bbf-9535-44cea7e550ca].
2024-11-01 13:55:10.911 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[135919890] label[addax_doris_writer_18d59689-8f07-49af-801a-c5d4caed1040].
2024-11-01 13:55:10.995 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136019891'
2024-11-01 13:55:15.999 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_18d59689-8f07-49af-801a-c5d4caed1040].
2024-11-01 13:55:16.001 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[139436171] label[addax_doris_writer_157651dc-63a3-403a-8e59-d105ad19ae49].
2024-11-01 13:55:16.079 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '139536172'
2024-11-01 13:55:17.419 [ job-0] INFO StandAloneJobContainerCommunicator - Total 1442272 records, 487717153 bytes | Speed 5.74MB/s, 17797 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 48.248s | Percentage 0.00%
2024-11-01 13:55:20.002 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_157651dc-63a3-403a-8e59-d105ad19ae49].
2024-11-01 13:55:20.463 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[149979779] label[addax_doris_writer_66de9c5e-75c3-480c-b406-86d7cadc6e25].
2024-11-01 13:55:20.550 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '150079780'
2024-11-01 13:55:23.826 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_66de9c5e-75c3-480c-b406-86d7cadc6e25].
2024-11-01 13:55:25.102 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[151233251] label[addax_doris_writer_fdec9c04-6623-4a64-a680-2addfa99519f].
2024-11-01 13:55:25.184 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '151333252'
2024-11-01 13:55:29.096 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_fdec9c04-6623-4a64-a680-2addfa99519f].
2024-11-01 13:55:29.423 [ job-0] INFO StandAloneJobContainerCommunicator - Total 1657504 records, 567567117 bytes | Speed 6.35MB/s, 17936 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 53.903s | Percentage 0.00%
2024-11-01 13:55:29.975 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[141789613] label[addax_doris_writer_5609d9cd-d04d-4d7b-8c4b-384d1c429441].
2024-11-01 13:55:30.059 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '141889614'
2024-11-01 13:55:33.454 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_5609d9cd-d04d-4d7b-8c4b-384d1c429441].
2024-11-01 13:55:34.605 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[144505092] label[addax_doris_writer_797378d2-df95-4005-9631-71ae92152e7b].
2024-11-01 13:55:34.684 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '144605093'
2024-11-01 13:55:38.486 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_797378d2-df95-4005-9631-71ae92152e7b].
2024-11-01 13:55:39.293 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[149613006] label[addax_doris_writer_b46e0cce-d96f-45a3-a5c0-2e5c3258730d].
2024-11-01 13:55:39.497 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '149713007'
2024-11-01 13:55:41.425 [ job-0] INFO StandAloneJobContainerCommunicator - Total 1870592 records, 643981065 bytes | Speed 6.07MB/s, 17757 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 59.607s | Percentage 0.00%
2024-11-01 13:55:43.163 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_b46e0cce-d96f-45a3-a5c0-2e5c3258730d].
2024-11-01 13:55:43.894 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[145376428] label[addax_doris_writer_a84c9511-036b-4f22-affa-5ac0f32c6494].
2024-11-01 13:55:43.990 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '145476429'
2024-11-01 13:55:47.104 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_a84c9511-036b-4f22-affa-5ac0f32c6494].
2024-11-01 13:55:49.032 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136757894] label[addax_doris_writer_7371e68b-fb79-40ff-9ea6-47fcd944a8c9].
2024-11-01 13:55:49.106 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136857895'
2024-11-01 13:55:52.788 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_7371e68b-fb79-40ff-9ea6-47fcd944a8c9].
2024-11-01 13:55:53.429 [ job-0] INFO StandAloneJobContainerCommunicator - Total 2088416 records, 724722564 bytes | Speed 6.42MB/s, 18152 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 65.164s | Percentage 0.00%
2024-11-01 13:55:54.541 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136635781] label[addax_doris_writer_6d484d58-bde1-46d2-9cbe-ec0660a4c3b5].
2024-11-01 13:55:54.631 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136735782'
2024-11-01 13:55:59.428 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_6d484d58-bde1-46d2-9cbe-ec0660a4c3b5].
2024-11-01 13:55:59.430 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136786231] label[addax_doris_writer_8ef00410-5fcc-4f93-82bd-268c2a5ba4e8].
2024-11-01 13:55:59.512 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136886232'
2024-11-01 13:56:03.308 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_8ef00410-5fcc-4f93-82bd-268c2a5ba4e8].
2024-11-01 13:56:04.086 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136440213] label[addax_doris_writer_73a7a567-2a58-4065-b0c4-9e3a51e88c9f].
2024-11-01 13:56:04.202 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136540214'
2024-11-01 13:56:05.434 [ job-0] INFO StandAloneJobContainerCommunicator - Total 2487008 records, 859649371 bytes | Speed 10.72MB/s, 33216 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 77.234s | Percentage 0.00%
2024-11-01 13:56:07.182 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_73a7a567-2a58-4065-b0c4-9e3a51e88c9f].
2024-11-01 13:56:08.604 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136295115] label[addax_doris_writer_361e4dea-cad9-4c2c-b66f-dd6f591e1992].
2024-11-01 13:56:08.693 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136395116'
2024-11-01 13:56:11.742 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_361e4dea-cad9-4c2c-b66f-dd6f591e1992].
2024-11-01 13:56:13.356 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136225528] label[addax_doris_writer_8a68f7fa-25a0-491d-ac6c-b096be03abee].
2024-11-01 13:56:13.479 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136325529'
2024-11-01 13:56:17.380 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_8a68f7fa-25a0-491d-ac6c-b096be03abee].
2024-11-01 13:56:17.436 [ job-0] INFO StandAloneJobContainerCommunicator - Total 2703200 records, 932666506 bytes | Speed 5.80MB/s, 18016 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 83.035s | Percentage 0.00%
2024-11-01 13:56:17.858 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136723048] label[addax_doris_writer_dd333b0a-be60-464b-8513-392ddaf0e590].
2024-11-01 13:56:17.948 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136823049'
2024-11-01 13:56:21.351 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_dd333b0a-be60-464b-8513-392ddaf0e590].
2024-11-01 13:56:22.687 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136553415] label[addax_doris_writer_500bcdf5-99b8-430c-8d2a-8d14a567f79d].
2024-11-01 13:56:22.774 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136653416'
2024-11-01 13:56:25.863 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_500bcdf5-99b8-430c-8d2a-8d14a567f79d].
2024-11-01 13:56:27.407 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136008227] label[addax_doris_writer_13a680af-d7aa-40ff-8fba-c09aa74b261f].
2024-11-01 13:56:27.496 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136108228'
2024-11-01 13:56:29.440 [ job-0] INFO StandAloneJobContainerCommunicator - Total 2916640 records, 1004714404 bytes | Speed 5.73MB/s, 17786 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 89.011s | Percentage 0.00%
2024-11-01 13:56:31.153 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_13a680af-d7aa-40ff-8fba-c09aa74b261f].
2024-11-01 13:56:32.618 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[137068462] label[addax_doris_writer_ed54f1bd-4216-4ee1-83eb-0170fdc69645].
2024-11-01 13:56:32.706 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '137168463'
2024-11-01 13:56:35.772 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_ed54f1bd-4216-4ee1-83eb-0170fdc69645].
2024-11-01 13:56:37.256 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136784112] label[addax_doris_writer_4b2652e8-c462-46c8-8d76-0d9c4bcd056e].
2024-11-01 13:56:37.375 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136884113'
2024-11-01 13:56:40.373 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_4b2652e8-c462-46c8-8d76-0d9c4bcd056e].
2024-11-01 13:56:41.442 [ job-0] INFO StandAloneJobContainerCommunicator - Total 3119072 records, 1073329891 bytes | Speed 5.45MB/s, 16869 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 94.995s | Percentage 0.00%
2024-11-01 13:56:41.860 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136537637] label[addax_doris_writer_7f8e7a17-439f-470b-bb29-97bc54270857].
2024-11-01 13:56:41.939 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136637638'
2024-11-01 13:56:48.319 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_7f8e7a17-439f-470b-bb29-97bc54270857].
2024-11-01 13:56:48.321 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136477462] label[addax_doris_writer_0c733c2d-6840-4029-8321-c504c68b480e].
2024-11-01 13:56:48.439 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136577463'
2024-11-01 13:56:53.451 [ job-0] INFO StandAloneJobContainerCommunicator - Total 3336288 records, 1146762505 bytes | Speed 5.84MB/s, 18101 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 101.006s | Percentage 0.00%
2024-11-01 13:56:53.456 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_0c733c2d-6840-4029-8321-c504c68b480e].
2024-11-01 13:56:53.457 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136013331] label[addax_doris_writer_7d6645af-7110-49c2-b669-80e68ca30445].
2024-11-01 13:56:53.538 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136113332'
2024-11-01 13:56:56.658 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_7d6645af-7110-49c2-b669-80e68ca30445].
2024-11-01 13:56:56.659 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[144223545] label[addax_doris_writer_0caa46e2-dd76-4028-baf2-2163d3895b50].
2024-11-01 13:56:56.742 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '144323546'
2024-11-01 13:57:00.803 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_0caa46e2-dd76-4028-baf2-2163d3895b50].
2024-11-01 13:57:01.061 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[149886564] label[addax_doris_writer_dca33c7c-746f-46ec-b78e-0dc25b0b5732].
2024-11-01 13:57:01.143 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '149986565'
2024-11-01 13:57:04.330 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_dca33c7c-746f-46ec-b78e-0dc25b0b5732].
2024-11-01 13:57:05.464 [ job-0] INFO StandAloneJobContainerCommunicator - Total 3746880 records, 1293729266 bytes | Speed 11.68MB/s, 34216 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 111.990s | Percentage 0.00%
2024-11-01 13:57:06.433 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[151340896] label[addax_doris_writer_dc3dec28-3664-43e4-ac53-4f734ca4805f].
2024-11-01 13:57:06.520 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '151440897'
2024-11-01 13:57:10.027 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_dc3dec28-3664-43e4-ac53-4f734ca4805f].
2024-11-01 13:57:10.935 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[137213109] label[addax_doris_writer_5296f297-72cf-4304-96b9-f68fb0fcbd36].
2024-11-01 13:57:11.009 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '137313110'
2024-11-01 13:57:14.022 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_5296f297-72cf-4304-96b9-f68fb0fcbd36].
2024-11-01 13:57:15.867 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[148956652] label[addax_doris_writer_c5e4cccd-7220-46a9-b2e5-c4458cda49b4].
2024-11-01 13:57:15.949 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '149056653'
2024-11-01 13:57:17.773 [ job-0] INFO StandAloneJobContainerCommunicator - Total 3956512 records, 1368655386 bytes | Speed 5.95MB/s, 17469 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 117.933s | Percentage 0.00%
2024-11-01 13:57:19.423 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_c5e4cccd-7220-46a9-b2e5-c4458cda49b4].
2024-11-01 13:57:20.399 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[149848963] label[addax_doris_writer_54b8c2bd-1779-4bdb-8199-948fcec7a09a].
2024-11-01 13:57:20.481 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '149948964'
2024-11-01 13:57:24.022 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_54b8c2bd-1779-4bdb-8199-948fcec7a09a].
2024-11-01 13:57:25.091 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[141392516] label[addax_doris_writer_f478b49b-1755-4948-ba99-47e0f1779f7e].
2024-11-01 13:57:25.172 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '141492517'
2024-11-01 13:57:28.395 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_f478b49b-1755-4948-ba99-47e0f1779f7e].
2024-11-01 13:57:29.787 [ job-0] INFO StandAloneJobContainerCommunicator - Total 4174784 records, 1448826444 bytes | Speed 6.37MB/s, 18189 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 123.584s | Percentage 0.00%
2024-11-01 13:57:29.792 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136335964] label[addax_doris_writer_bd7cb5d1-1f19-46d0-b434-2a539f11a9bf].
2024-11-01 13:57:29.872 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136435965'
2024-11-01 13:57:33.782 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_bd7cb5d1-1f19-46d0-b434-2a539f11a9bf].
2024-11-01 13:57:34.760 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136268211] label[addax_doris_writer_8f03e6d5-3cee-4b2c-a019-951600db7517].
2024-11-01 13:57:34.849 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136368212'
2024-11-01 13:57:38.962 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_8f03e6d5-3cee-4b2c-a019-951600db7517].
2024-11-01 13:57:39.499 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136982905] label[addax_doris_writer_8ec702ab-f2b9-4d20-90ea-e3ada0792427].
2024-11-01 13:57:39.573 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '137082906'
2024-11-01 13:57:41.800 [ job-0] INFO StandAloneJobContainerCommunicator - Total 4383680 records, 1519452406 bytes | Speed 5.61MB/s, 17408 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 129.775s | Percentage 0.00%
2024-11-01 13:57:42.943 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_8ec702ab-f2b9-4d20-90ea-e3ada0792427].
2024-11-01 13:57:44.265 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136574799] label[addax_doris_writer_e661d8a9-feaa-4388-a157-d1f9ffb687e4].
2024-11-01 13:57:44.343 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136674800'
2024-11-01 13:57:47.401 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_e661d8a9-feaa-4388-a157-d1f9ffb687e4].
2024-11-01 13:57:49.414 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136271958] label[addax_doris_writer_4b2b2c9b-e3ba-4ee3-a626-6ac1e5fe54a4].
2024-11-01 13:57:49.494 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136371959'
2024-11-01 13:57:52.518 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_4b2b2c9b-e3ba-4ee3-a626-6ac1e5fe54a4].
2024-11-01 13:57:54.028 [ job-0] INFO StandAloneJobContainerCommunicator - Total 4595648 records, 1591222357 bytes | Speed 5.70MB/s, 17664 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 135.885s | Percentage 0.00%
2024-11-01 13:57:54.352 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136608503] label[addax_doris_writer_231f931d-c4ec-4b41-8e6e-5dca4038686e].
2024-11-01 13:57:54.477 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136708504'
2024-11-01 13:57:58.034 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_231f931d-c4ec-4b41-8e6e-5dca4038686e].
2024-11-01 13:58:00.290 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136751961] label[addax_doris_writer_1f93f4c4-f00d-44b6-adae-d58a060ac982].
2024-11-01 13:58:00.371 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136851962'
2024-11-01 13:58:05.829 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_1f93f4c4-f00d-44b6-adae-d58a060ac982].
2024-11-01 13:58:05.831 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136059508] label[addax_doris_writer_2e5019fa-4cb5-47be-b5f5-8b3256077827].
2024-11-01 13:58:05.910 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136159509'
2024-11-01 13:58:06.032 [ job-0] INFO StandAloneJobContainerCommunicator - Total 4977632 records, 1720255239 bytes | Speed 10.25MB/s, 31832 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 148.012s | Percentage 0.00%
2024-11-01 13:58:09.334 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_2e5019fa-4cb5-47be-b5f5-8b3256077827].
2024-11-01 13:58:10.156 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136966538] label[addax_doris_writer_f84bc8fc-6db2-4d3c-a894-1ca56ac03375].
2024-11-01 13:58:10.245 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '137066539'
2024-11-01 13:58:13.453 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_f84bc8fc-6db2-4d3c-a894-1ca56ac03375].
2024-11-01 13:58:15.418 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136627553] label[addax_doris_writer_290f7776-5372-4f87-9d2d-ad9e34ebd820].
2024-11-01 13:58:15.534 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136727554'
2024-11-01 13:58:18.035 [ job-0] INFO StandAloneJobContainerCommunicator - Total 5182592 records, 1789664599 bytes | Speed 5.52MB/s, 17080 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 154.088s | Percentage 0.00%
2024-11-01 13:58:18.609 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_290f7776-5372-4f87-9d2d-ad9e34ebd820].
2024-11-01 13:58:20.281 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136621146] label[addax_doris_writer_bec754a8-d195-4b4c-a277-61985a37bff7].
2024-11-01 13:58:20.358 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136721147'
2024-11-01 13:58:25.617 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_bec754a8-d195-4b4c-a277-61985a37bff7].
2024-11-01 13:58:25.619 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136505121] label[addax_doris_writer_bda14de3-0efa-49ea-ba0d-f13d9d676bf6].
2024-11-01 13:58:25.699 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136605122'
2024-11-01 13:58:28.828 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_bda14de3-0efa-49ea-ba0d-f13d9d676bf6].
2024-11-01 13:58:29.252 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136229726] label[addax_doris_writer_ca1352f4-e60a-41a7-95ff-9feeb0033392].
2024-11-01 13:58:29.338 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136329727'
2024-11-01 13:58:30.037 [ job-0] INFO StandAloneJobContainerCommunicator - Total 5394016 records, 1861125618 bytes | Speed 5.68MB/s, 17618 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 160.301s | Percentage 0.00%
2024-11-01 13:58:32.539 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_ca1352f4-e60a-41a7-95ff-9feeb0033392].
2024-11-01 13:58:33.882 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136393561] label[addax_doris_writer_92660f11-878d-4fe8-ab0a-ec8e6b755a50].
2024-11-01 13:58:33.978 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136493562'
2024-11-01 13:58:36.974 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_92660f11-878d-4fe8-ab0a-ec8e6b755a50].
2024-11-01 13:58:38.409 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[149002733] label[addax_doris_writer_966f99e8-e0d1-4da3-82ea-e639701cbda7].
2024-11-01 13:58:38.504 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '149102734'
2024-11-01 13:58:41.694 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_966f99e8-e0d1-4da3-82ea-e639701cbda7].
2024-11-01 13:58:42.039 [ job-0] INFO StandAloneJobContainerCommunicator - Total 5611744 records, 1935150866 bytes | Speed 5.88MB/s, 18144 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 165.943s | Percentage 0.00%
2024-11-01 13:58:43.007 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[151312919] label[addax_doris_writer_99a42602-5f2a-4809-b198-5f5d19ead8b5].
2024-11-01 13:58:43.089 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '151412920'
2024-11-01 13:58:46.359 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_99a42602-5f2a-4809-b198-5f5d19ead8b5].
2024-11-01 13:58:47.442 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[145701239] label[addax_doris_writer_e8628809-0607-4de5-89d3-1703135b742d].
2024-11-01 13:58:47.525 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '145801240'
2024-11-01 13:58:50.470 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_e8628809-0607-4de5-89d3-1703135b742d].
2024-11-01 13:58:52.008 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[140762719] label[addax_doris_writer_c61afadc-590d-4cd2-a356-7aae05be595f].
2024-11-01 13:58:52.087 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '140862720'
2024-11-01 13:58:54.042 [ job-0] INFO StandAloneJobContainerCommunicator - Total 5836000 records, 2019799317 bytes | Speed 6.73MB/s, 18688 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 171.498s | Percentage 0.00%
2024-11-01 13:58:54.043 [ job-0] INFO VMInfo -
[delta cpu info] =>
curDeltaCPU | averageCPU | maxDeltaCPU | minDeltaCPU
-1.00% | -1.00% | -1.00% | -1.00%
[delta memory info] =>
NAME | used(MB) | used_pct(%) | max_used(MB) | max_pct(%)
PS Eden Space | 369.01 | 58.81 | 369.01 | 58.81
Code Cache | 16.36 | 99.13 | 16.36 | 99.13
Compressed Class Space | 4.12 | 91.48 | 4.12 | 91.48
PS Survivor Space | 24.91 | 92.27 | 24.91 | 92.27
PS Old Gen | 763.72 | 55.93 | 763.72 | 55.93
Metaspace | 37.31 | 96.90 | 37.31 | 96.90
[delta gc info] =>
NAME | curDeltaGCCount | totalGCCount | maxDeltaGCCount | minDeltaGCCount | curDeltaGCTime | totalGCTime | maxDeltaGCTime | minDeltaGCTime
PS MarkSweep | 46 | 46 | 46 | 46 | 5.335 | 5.335 | 5.335 | 5.335
PS Scavenge | 501 | 501 | 501 | 501 | 12.036 | 12.036 | 12.036 | 12.036
2024-11-01 13:58:55.509 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_c61afadc-590d-4cd2-a356-7aae05be595f].
2024-11-01 13:58:56.503 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[150060087] label[addax_doris_writer_6bc6bb81-a0f7-4e56-9ad9-bb4a2676a9c7].
2024-11-01 13:58:56.591 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '150160088'
2024-11-01 13:59:00.779 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_6bc6bb81-a0f7-4e56-9ad9-bb4a2676a9c7].
2024-11-01 13:59:01.131 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[149270060] label[addax_doris_writer_15bcaf14-4652-43c5-a5c3-337a452ef1df].
2024-11-01 13:59:01.217 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '149370061'
2024-11-01 13:59:04.919 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_15bcaf14-4652-43c5-a5c3-337a452ef1df].
2024-11-01 13:59:06.047 [ job-0] INFO StandAloneJobContainerCommunicator - Total 6269120 records, 2175713110 bytes | Speed 12.39MB/s, 36093 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 182.767s | Percentage 0.00%
2024-11-01 13:59:06.117 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136303165] label[addax_doris_writer_0086fc0b-d91b-4de7-8754-3aedac995fdd].
2024-11-01 13:59:06.196 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136403166'
2024-11-01 13:59:09.839 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_0086fc0b-d91b-4de7-8754-3aedac995fdd].
2024-11-01 13:59:10.893 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136287594] label[addax_doris_writer_0754888c-7b30-4ffc-a8b0-4815199272a1].
2024-11-01 13:59:10.969 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136387595'
2024-11-01 13:59:15.955 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_0754888c-7b30-4ffc-a8b0-4815199272a1].
2024-11-01 13:59:15.957 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136578698] label[addax_doris_writer_c3f60891-6648-4e16-b00d-fe3b2f0fd6e3].
2024-11-01 13:59:16.042 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136678699'
2024-11-01 13:59:18.050 [ job-0] INFO StandAloneJobContainerCommunicator - Total 6486432 records, 2249055001 bytes | Speed 5.83MB/s, 18109 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 188.872s | Percentage 0.00%
2024-11-01 13:59:20.908 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_c3f60891-6648-4e16-b00d-fe3b2f0fd6e3].
2024-11-01 13:59:20.909 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136690801] label[addax_doris_writer_d57dbe74-57a2-48c4-b0b7-b3333787c153].
2024-11-01 13:59:20.990 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136790802'
2024-11-01 13:59:24.490 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_d57dbe74-57a2-48c4-b0b7-b3333787c153].
2024-11-01 13:59:25.070 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136786885] label[addax_doris_writer_8e3fa188-4526-4dfe-8727-01c03c040367].
2024-11-01 13:59:25.146 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136886886'
2024-11-01 13:59:27.993 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_8e3fa188-4526-4dfe-8727-01c03c040367].
2024-11-01 13:59:29.522 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136006178] label[addax_doris_writer_4043103c-bb18-46c2-841c-fb25022f9324].
2024-11-01 13:59:29.600 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136106179'
2024-11-01 13:59:30.053 [ job-0] INFO StandAloneJobContainerCommunicator - Total 6693088 records, 2319051069 bytes | Speed 5.56MB/s, 17221 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 194.805s | Percentage 0.00%
2024-11-01 13:59:32.748 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_4043103c-bb18-46c2-841c-fb25022f9324].
2024-11-01 13:59:34.065 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136496152] label[addax_doris_writer_a9cc6f90-c3f1-4e92-a688-9cad9d6522c9].
2024-11-01 13:59:34.139 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136596153'
2024-11-01 13:59:37.293 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_a9cc6f90-c3f1-4e92-a688-9cad9d6522c9].
2024-11-01 13:59:38.577 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136549177] label[addax_doris_writer_0ddef391-83b8-43df-8a58-479c20eda6db].
2024-11-01 13:59:38.778 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136649178'
2024-11-01 13:59:42.115 [ job-0] INFO StandAloneJobContainerCommunicator - Total 6916128 records, 2394309904 bytes | Speed 5.98MB/s, 18586 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 200.697s | Percentage 0.00%
2024-11-01 13:59:42.179 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_0ddef391-83b8-43df-8a58-479c20eda6db].
2024-11-01 13:59:44.005 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136114123] label[addax_doris_writer_d799f132-7f90-48df-80d3-343fef28a56c].
2024-11-01 13:59:44.082 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136214124'
2024-11-01 13:59:47.481 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_d799f132-7f90-48df-80d3-343fef28a56c].
2024-11-01 13:59:48.756 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136905216] label[addax_doris_writer_57330ec7-3a2f-4bb4-afc8-f7ddc7a1d075].
2024-11-01 13:59:48.830 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '137005217'
2024-11-01 13:59:51.898 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_57330ec7-3a2f-4bb4-afc8-f7ddc7a1d075].
2024-11-01 13:59:53.328 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136497838] label[addax_doris_writer_39db7a3c-e755-4549-a549-434cb8b5e6bc].
2024-11-01 13:59:53.409 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136597839'
2024-11-01 13:59:54.117 [ job-0] INFO StandAloneJobContainerCommunicator - Total 7117120 records, 2462111763 bytes | Speed 5.39MB/s, 16749 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 207.058s | Percentage 0.00%
2024-11-01 13:59:56.332 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_39db7a3c-e755-4549-a549-434cb8b5e6bc].
2024-11-01 13:59:59.015 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[137147325] label[addax_doris_writer_11c902af-7088-4881-b564-b7aab0018d6f].
2024-11-01 13:59:59.418 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '137247326'
2024-11-01 14:00:03.478 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_11c902af-7088-4881-b564-b7aab0018d6f].
2024-11-01 14:00:05.073 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[136521756] label[addax_doris_writer_c76f6b34-d65f-42c2-95c4-7bcd4d7443c3].
2024-11-01 14:00:05.152 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '136621757'
2024-11-01 14:00:06.126 [ job-0] INFO StandAloneJobContainerCommunicator - Total 7497344 records, 2590870991 bytes | Speed 10.23MB/s, 31685 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 218.957s | Percentage 0.00%
2024-11-01 14:00:08.541 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_c76f6b34-d65f-42c2-95c4-7bcd4d7443c3].
2024-11-01 14:00:09.990 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[135742429] label[addax_doris_writer_6efefdc6-aeba-4e47-aecd-e91921fceb6f].
2024-11-01 14:00:10.071 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '135842430'
2024-11-01 14:00:14.168 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_6efefdc6-aeba-4e47-aecd-e91921fceb6f].
2024-11-01 14:00:16.073 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[141387553] label[addax_doris_writer_5cd70429-7a58-47b2-b7bd-09d661bb44f9].
2024-11-01 14:00:16.153 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '141487554'
2024-11-01 14:00:18.380 [ job-0] INFO StandAloneJobContainerCommunicator - Total 7684480 records, 2654953333 bytes | Speed 5.09MB/s, 15594 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 225.150s | Percentage 0.00%
2024-11-01 14:00:20.034 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_5cd70429-7a58-47b2-b7bd-09d661bb44f9].
2024-11-01 14:00:21.368 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[150014344] label[addax_doris_writer_9200c08c-e3a8-4d76-bdd7-b6a66ed5b22c].
2024-11-01 14:00:21.451 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '150114345'
2024-11-01 14:00:26.258 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_9200c08c-e3a8-4d76-bdd7-b6a66ed5b22c].
2024-11-01 14:00:27.237 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[151335830] label[addax_doris_writer_0ab872a2-5053-4827-b383-74f30c20031b].
2024-11-01 14:00:27.319 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '151435831'
2024-11-01 14:00:30.519 [ job-0] INFO StandAloneJobContainerCommunicator - Total 7858560 records, 2720761063 bytes | Speed 5.23MB/s, 14506 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 231.031s | Percentage 0.00%
2024-11-01 14:00:30.551 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_0ab872a2-5053-4827-b383-74f30c20031b].
2024-11-01 14:00:31.789 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[140286674] label[addax_doris_writer_2237d22c-0d34-4b76-8026-f72b111709aa].
2024-11-01 14:00:31.871 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '140386675'
2024-11-01 14:00:35.021 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_2237d22c-0d34-4b76-8026-f72b111709aa].
2024-11-01 14:00:36.366 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[146240194] label[addax_doris_writer_2b9d4413-9eba-45a1-a9e6-72a70ce2478d].
2024-11-01 14:00:36.450 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.184:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '146340195'
2024-11-01 14:00:41.943 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_2b9d4413-9eba-45a1-a9e6-72a70ce2478d].
2024-11-01 14:00:41.945 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[100000] bytes[149560939] label[addax_doris_writer_94e1c842-5b23-4698-9a35-8e45fb09637e].
2024-11-01 14:00:42.032 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.181:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '149660940'
2024-11-01 14:00:42.522 [ job-0] INFO StandAloneJobContainerCommunicator - Total 8074720 records, 2798256585 bytes | Speed 6.16MB/s, 18013 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 236.772s | Percentage 0.00%
2024-11-01 14:00:45.294 [ Thread-1] INFO DorisWriterManager - Async stream load finished: label[addax_doris_writer_94e1c842-5b23-4698-9a35-8e45fb09637e].
2024-11-01 14:00:45.296 [ Thread-1] INFO DorisStreamLoadObserver - Start to join batch data: rows[52820] bytes[79049955] label[addax_doris_writer_f9476650-2fbf-41c5-b13c-361484479f5f].
2024-11-01 14:00:45.340 [ Thread-1] INFO DorisStreamLoadObserver - Executing stream load to: 'http://10.96.119.183:8001/api/dshisdatadb/b2b_uorders_orderdetail/_stream_load', size: '79102776'
2024-11-01 14:00:45.523 [ job-0] INFO AbstractScheduler - The scheduler has completed all tasks.
2024-11-01 14:00:45.523 [ job-0] INFO JobContainer - The Writer.Job [doriswriter] perform post work.
2024-11-01 14:00:45.524 [ job-0] INFO JobContainer - The Reader.Job [kudureader] perform post work.
2024-11-01 14:00:45.524 [ job-0] INFO StandAloneJobContainerCommunicator - Total 8246458 records, 2862649860 bytes | Speed 6.63MB/s, 20015 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 240.936s | Percentage 100.00%
2024-11-01 14:00:45.525 [ job-0] INFO JobContainer -
Job start at : 2024-11-01 13:53:52
Job end at : 2024-11-01 14:00:45
Job took secs : 413s
Average bps : 6.63MB/s
Average rps : 20015rec/s
Number of rec : 8246458
Failed record : 0
Addax: 4.2.2-fix_kudu_timestamp
Kudu: 1.10.0-cdh6.3.2
Impala: 3.2.0-cdh6.3.2
你的 doris 表是不是用的 merge-on-write 模式表,这种情况下,主键或者唯一索引相同的行会合并写入的。如果 Addax
最后统计的结果是读取了 8246458 行了,那肯定也写入了这么多行,不相同的情况,最大的可能就是 doris 表模式的问题
- addax比源头多了四条数据
- Doris使用的是明细模型 不应该会少数据
- 源头问题 和 写入问题
你看下源表里是不是有 dcreated
, id
相同的行
id没有重复
以下是我的猜测
1.这边还有一个问题就是 id如果超过上界和下界的范围会导致数据不准确嘛 因为档数据量足够大时是超过这个范围的
2.切换不同的切片会导致读取的数据量变少
一个是任务完成的数据量和源头对应不上
一个是写入的数据量和完成的对应不上
但是数据在几万级别时却又不会出现这种情况 数据量大时却又出现丢数据
没有用ID范围条件 只是使用"splitPk":"id"
如果使用"splitPk":"id", 然后"lowerBound":0, "upperBound":100000000, 如果id本身的值超过下界会不会导致丢数据
没有用ID范围条件 只是使用"splitPk":"id" 如果使用"splitPk":"id", 然后"lowerBound":0, "upperBound":100000000, 如果id本身的值超过下界会不会导致丢数据
我意思是你在 kudu 上查询的记录数和在 doris 上查询的记录数不相同,这些查询都带上了 id >=0 and id <=100000000
的条件了吗?
id不是一个切片字段嘛 为什么要带上这个呢
都没有带上 id >=0 and id <=100000000