Agora-wireless/Agora

Failed to setup in bigstation mode, and question about bigstation_mode performance

Opened this issue · 1 comments

Hello, I try to test Agora in bigstation mode but failed, and I can't fix it. :(
I used sim_ul.json with a little modification:

{
    "bs_radio_num": 8,
    "ue_radio_num": 8,
    "frame_schedule": [
        "PUUUUUUUUUUUUU"
    ],
    "modulation": "64QAM",
    "Zc": 104,
    "bs_server_addr": "127.0.0.1",
    "bs_rru_addr": "127.0.0.1",
    "fft_size": 2048,
    "ofdm_data_num": 1200,
    "demul_block_size": 64,
    "freq_orthogonal_pilot": true,
    /* Compute configuration */
    "core_offset": 4,
    "exclude_cores": [
        0
    ],
    "bigstation_mode": true,
    "fft_thread_num": 8,
    "demul_thread_num": 4,
    "decode_thread_num": 4,
    "worker_thread_num": 16,
    "socket_thread_num": 1
}

And I got the below result (I attached it at the last)

(It works fine without bigstation mode.)
Could you please help find the reason? Thank you very much!

Also, I am curious about Agora's performance w/o bigstation mode. In fact, I am confused about how bigstation mode can improve PHY performance, since we just bind the different tasks to different CPUs without any other optimization, maybe we can benefit from cache? But it seems not such a big deal. Is there any theory or experiment result that can tell us that it is really useful?

Thank you very much!

Agora return:

FlexRAN SDK bblib_lte_ldpc_encoder version sdk-19.04-ea1-1-g3be2380
FlexRAN SDK bblib_lte_ldpc_decoder version sdk-19.04-ea1-1-g3be2380
FlexRAN SDK bblib_common version sdk-19.04-ea1-1-g3be2380
System CPU count 48
NUMA node 0 111111111111000000000000000000000000000000000000 CPUs: 0 1 2 3 4 5 6 7 8 9 10 11 
NUMA node 1 000000000000111111111111000000000000000000000000 CPUs: 12 13 14 15 16 17 18 19 20 21 22 23 
Usable Cpu count 23
Unable to open config file 
35:616805 INFOR: Config: Frame schedule PUUUUUUUUUUUUU (14 symbols)
35:616843 INFOR: Config: LDPC: Zc: 104, 1 code blocks per symbol, 2288 information bits per encoding, 6864 bits per encoded code word, decoder iterations: 5, code rate 0.333 (nRows = 46)
35:616853 INFOR: Config: 8 BS antennas, 8 UE antennas, 1 pilot symbols per frame,
        13 uplink data symbols per frame, 0 downlink data symbols per frame,
        2048 OFDM subcarriers (1200 data subcarriers), modulation 64QAM,
        1 codeblocks per symbol, 286 bytes per code block,
        3588 UL MAC data bytes per frame, 3718 UL MAC bytes per frame, 
        0 DL MAC data bytes per frame, 0 DL MAC bytes per frame, frame time 5734.400 usec 
Uplink Max Mac data tp (Mbps) 5.006 
Downlink Max Mac data tp (Mbps) 0.000 
Agora: project directory [.], RDTSC frequency = 2.19 GHz
DPDK 20.02.1 initialized
Number of ports: 1 used (offset: 0), 2 available, socket: 0
NIC 0 Socket: 0, MAC: 40 a6 b7 2c 9d b0 
Adding steering rule for src IP 127.0.0.1, dest IP 127.0.0.1, src port: 9000, dst port: 8000, DPDK port 0, queue: 0
Number of DPDK cores: 2
36:534863 INFOR: Master thread core 4, TX/RX thread cores 5--5, worker thread cores 6--21
DPDK TXRX thread 0: pinned to core 6
Main [frame 0 + 1316556875.98 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 0 + 0.10 ms]: Received all pilots
Main [frame 0 + 0.40 ms]: FFT-ed all pilots
Frame 0 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 0 + 185.57 ms]: Received all packets
Main [frame 1 + 199.78 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 1 + 0.06 ms]: Received all pilots
Main [frame 1 + 185.78 ms]: Received all packets
   +++ Deferring encoding of frame 2
Main [frame 2 + 200.01 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 2 + 0.19 ms]: Received all pilots
Main [frame 2 + 0.29 ms]: FFT-ed all pilots
Frame 2 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 2 + 185.79 ms]: Received all packets
   +++ Deferring encoding of frame 3
Main [frame 3 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 3 + 0.06 ms]: Received all pilots
Main [frame 3 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 4
Main [frame 4 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 4 + 0.17 ms]: Received all pilots
Main [frame 4 + 0.32 ms]: FFT-ed all pilots
Frame 4 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 4 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 5
Main [frame 5 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 5 + 0.05 ms]: Received all pilots
Main [frame 5 + 185.75 ms]: Received all packets
   +++ Deferring encoding of frame 6
Main [frame 6 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 6 + 0.05 ms]: Received all pilots
Main [frame 6 + 0.17 ms]: FFT-ed all pilots
Frame 6 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 6 + 185.78 ms]: Received all packets
   +++ Deferring encoding of frame 7
Main [frame 7 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 7 + 0.05 ms]: Received all pilots
Main [frame 7 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 8
Main [frame 8 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 8 + 0.06 ms]: Received all pilots
Main [frame 8 + 0.17 ms]: FFT-ed all pilots
Frame 8 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 8 + 185.78 ms]: Received all packets
   +++ Deferring encoding of frame 9
Main [frame 9 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 9 + 0.05 ms]: Received all pilots
Main [frame 9 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 10
Main [frame 10 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 10 + 0.10 ms]: Received all pilots
Main [frame 10 + 0.22 ms]: FFT-ed all pilots
Frame 10 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 10 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 11
Main [frame 11 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 11 + 0.04 ms]: Received all pilots
Main [frame 11 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 12
Main [frame 12 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 12 + 0.06 ms]: Received all pilots
Main [frame 12 + 0.17 ms]: FFT-ed all pilots
Frame 12 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 12 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 13
Main [frame 13 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 13 + 0.05 ms]: Received all pilots
Main [frame 13 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 14
Main [frame 14 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 14 + 0.05 ms]: Received all pilots
Main [frame 14 + 0.17 ms]: FFT-ed all pilots
Frame 14 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 14 + 185.78 ms]: Received all packets
   +++ Deferring encoding of frame 15
Main [frame 15 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 15 + 0.06 ms]: Received all pilots
Main [frame 15 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 16
Main [frame 16 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 16 + 0.05 ms]: Received all pilots
Main [frame 16 + 0.22 ms]: FFT-ed all pilots
Frame 16 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 16 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 17
Main [frame 17 + 200.02 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 17 + 0.04 ms]: Received all pilots
Main [frame 17 + 185.74 ms]: Received all packets
   +++ Deferring encoding of frame 18
Main [frame 18 + 199.99 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 18 + 0.12 ms]: Received all pilots
Main [frame 18 + 0.17 ms]: FFT-ed all pilots
Frame 18 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 18 + 185.79 ms]: Received all packets
   +++ Deferring encoding of frame 19
Main [frame 19 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 19 + 0.05 ms]: Received all pilots
Main [frame 19 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 20
Main [frame 20 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 20 + 0.08 ms]: Received all pilots
Main [frame 20 + 0.18 ms]: FFT-ed all pilots
Frame 20 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 20 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 21
Main [frame 21 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 21 + 0.05 ms]: Received all pilots
Main [frame 21 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 22
Main [frame 22 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 22 + 0.06 ms]: Received all pilots
Main [frame 22 + 0.12 ms]: FFT-ed all pilots
Frame 22 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 22 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 23
Main [frame 23 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 23 + 0.06 ms]: Received all pilots
Main [frame 23 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 24
Main [frame 24 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 24 + 0.05 ms]: Received all pilots
Main [frame 24 + 0.14 ms]: FFT-ed all pilots
Frame 24 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 24 + 185.78 ms]: Received all packets
   +++ Deferring encoding of frame 25
Main [frame 25 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 25 + 0.05 ms]: Received all pilots
Main [frame 25 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 26
Main [frame 26 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 26 + 0.05 ms]: Received all pilots
Main [frame 26 + 0.14 ms]: FFT-ed all pilots
Frame 26 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 26 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 27
Main [frame 27 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 27 + 0.06 ms]: Received all pilots
Main [frame 27 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 28
Main [frame 28 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 28 + 0.06 ms]: Received all pilots
Main [frame 28 + 0.14 ms]: FFT-ed all pilots
Frame 28 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 28 + 185.78 ms]: Received all packets
   +++ Deferring encoding of frame 29
Main [frame 29 + 200.01 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 29 + 0.05 ms]: Received all pilots
Main [frame 29 + 185.75 ms]: Received all packets
   +++ Deferring encoding of frame 30
Main [frame 30 + 199.99 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 30 + 0.05 ms]: Received all pilots
Main [frame 30 + 0.12 ms]: FFT-ed all pilots
Frame 30 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 30 + 185.78 ms]: Received all packets
   +++ Deferring encoding of frame 31
Main [frame 31 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 31 + 0.05 ms]: Received all pilots
Main [frame 31 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 32
Main [frame 32 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 32 + 0.06 ms]: Received all pilots
Main [frame 32 + 0.14 ms]: FFT-ed all pilots
Frame 32 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 32 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 33
Main [frame 33 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 33 + 0.05 ms]: Received all pilots
Main [frame 33 + 185.75 ms]: Received all packets
   +++ Deferring encoding of frame 34
Main [frame 34 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 34 + 0.06 ms]: Received all pilots
Main [frame 34 + 0.13 ms]: FFT-ed all pilots
Frame 34 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 34 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 35
Main [frame 35 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 35 + 0.05 ms]: Received all pilots
Main [frame 35 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 36
Main [frame 36 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 36 + 0.07 ms]: Received all pilots
Main [frame 36 + 0.16 ms]: FFT-ed all pilots
Frame 36 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 36 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 37
Main [frame 37 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 37 + 0.05 ms]: Received all pilots
Main [frame 37 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 38
Main [frame 38 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 38 + 0.05 ms]: Received all pilots
Main [frame 38 + 0.12 ms]: FFT-ed all pilots
Frame 38 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 38 + 185.77 ms]: Received all packets
   +++ Deferring encoding of frame 39
Main [frame 39 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 39 + 0.06 ms]: Received all pilots
Main [frame 39 + 185.76 ms]: Received all packets
   +++ Deferring encoding of frame 40
Main [frame 40 + 200.00 ms since last frame]: Received first packet. Remaining packets in prev frame: 0
Main [frame 40 + 0.03 ms]: Received all pilots
Main [frame 40 + 0.13 ms]: FFT-ed all pilots
Frame 40 Pilot Signal SNR (dB) Range at BS Antennas: User 0: [26.3,28.5] User 1: [0.0,-100.0] User 2: [0.0,-100.0] User 3: [0.0,-100.0] User 4: [0.0,-100.0] User 5: [0.0,-100.0] User 6: [0.0,-100.0] User 7: [0.0,-100.0] 
Main [frame 40 + 69.66 ms]: Received all packets
TXRX thread [0] DpdkRecv rx_buffer full, slot: 112
47:449324 INFOR: Agora: printing stats and saving to file
Stats: total processed frames 1
Uplink totals (tasks, frames): CSI (168, 21.00), ZF (0, 0.00), FFT (2184, 21.00), Demul (0, 0.00), Decode (0, 0.00)
Thread 0 performed (tasks, fraction of tasks): FFT (273, 12.50%), CSI (23, 13.69%), 
Thread 1 performed (tasks, fraction of tasks): FFT (272, 12.45%), CSI (21, 12.50%), 
Thread 2 performed (tasks, fraction of tasks): FFT (274, 12.55%), CSI (21, 12.50%), 
Thread 3 performed (tasks, fraction of tasks): FFT (279, 12.77%), CSI (20, 11.90%), 
Thread 4 performed (tasks, fraction of tasks): FFT (253, 11.58%), CSI (19, 11.31%), 
Thread 5 performed (tasks, fraction of tasks): FFT (285, 13.05%), CSI (21, 12.50%), 
Thread 6 performed (tasks, fraction of tasks): FFT (280, 12.82%), CSI (20, 11.90%), 
Thread 7 performed (tasks, fraction of tasks): FFT (268, 12.27%), CSI (23, 13.69%), 
Thread 8 performed (tasks, fraction of tasks): 
Thread 9 performed (tasks, fraction of tasks): 
Thread 10 performed (tasks, fraction of tasks): 
Thread 11 performed (tasks, fraction of tasks): 
Thread 12 performed (tasks, fraction of tasks): 
Thread 13 performed (tasks, fraction of tasks): 
Thread 14 performed (tasks, fraction of tasks): 
Thread 15 performed (tasks, fraction of tasks): 
Stats: Saving master timestamps to ./data/timeresult.txt
Stats: Printing detailed results to ./data/timeresult_detail.txt
UE 0: Uplink bit errors (BER) 0/0(-nan), block errors (BLER) 0/0 (-nan)
UE 1: Uplink bit errors (BER) 0/0(-nan), block errors (BLER) 0/0 (-nan)
UE 2: Uplink bit errors (BER) 0/0(-nan), block errors (BLER) 0/0 (-nan)
UE 3: Uplink bit errors (BER) 0/0(-nan), block errors (BLER) 0/0 (-nan)
UE 4: Uplink bit errors (BER) 0/0(-nan), block errors (BLER) 0/0 (-nan)
UE 5: Uplink bit errors (BER) 0/0(-nan), block errors (BLER) 0/0 (-nan)
UE 6: Uplink bit errors (BER) 0/0(-nan), block errors (BLER) 0/0 (-nan)
UE 7: Uplink bit errors (BER) 0/0(-nan), block errors (BLER) 0/0 (-nan)
47:450207 INFOR: Agora: terminating
=================================
          CORE LIST SUMMARY      
=================================
Total Number of Cores: 48 : 24 
|| Core ID:  5 || Requested:  4 || ThreadType: Master           || ThreadId: 139958922388480 
|| Core ID:  5 || Requested:  4 || ThreadType: Master           || ThreadId: 139958922388480 
|| Core ID:  7 || Requested:  6 || ThreadType: Worker (FFT)     || ThreadId: 139958573908992 
|| Core ID:  8 || Requested:  7 || ThreadType: Worker (FFT)     || ThreadId: 139958565516288 
|| Core ID:  9 || Requested:  8 || ThreadType: Worker (FFT)     || ThreadId: 139958488658944 
|| Core ID: 10 || Requested:  9 || ThreadType: Worker (FFT)     || ThreadId: 139958480266240 
|| Core ID: 11 || Requested: 10 || ThreadType: Worker (FFT)     || ThreadId: 139958471873536 
|| Core ID: 12 || Requested: 11 || ThreadType: Worker (FFT)     || ThreadId: 139958463480832 
|| Core ID: 13 || Requested: 12 || ThreadType: Worker (FFT)     || ThreadId: 139958455088128 
|| Core ID: 14 || Requested: 13 || ThreadType: Worker (FFT)     || ThreadId: 139958446695424 
|| Core ID: 15 || Requested: 14 || ThreadType: Worker (Demul)   || ThreadId: 139958438302720 
|| Core ID: 16 || Requested: 15 || ThreadType: Worker (Demul)   || ThreadId: 139958354441216 
|| Core ID: 17 || Requested: 16 || ThreadType: Worker (Demul)   || ThreadId: 139958346048512 
|| Core ID: 18 || Requested: 17 || ThreadType: Worker (Demul)   || ThreadId: 139958337655808 
|| Core ID: 19 || Requested: 18 || ThreadType: Worker (Decode)  || ThreadId: 139958329263104 
|| Core ID: 20 || Requested: 19 || ThreadType: Worker (Decode)  || ThreadId: 139958320870400 
|| Core ID: 21 || Requested: 20 || ThreadType: Worker (Decode)  || ThreadId: 139958312477696 
|| Core ID: 22 || Requested: 21 || ThreadType: Worker (Decode)  || ThreadId: 139958304084992 
=================================

There seems to be a bug with big station mode in Agora. Big station mode was designed for comparison vs Agora "normal" mode. Feel free to reference the result ( https://www.yecl.org/publications/ding2020conext.pdf ) but BigStation mode is only preferable in highly distributed systems where the Agora data-parallel design has increased cost of synchronization.