JANA4ML4FPGA
EIC R&D supported project developing ML on FPGA for streaming readout systems
How to run
Docker images
There are two images:
- eicdev/ml4fpga-pre - with all dependencies but without JANA4ML4FPGA built. This image is used for CI or development reasons and is not designed for running JANA4ML4FPGA
- eicdev/ml4fpga - The image can be used to run JANA4ML4FPGA
Running in docker
(extended information about options of running this docker containers)
To download/update the container
docker pull eicdev/ml4fpga:latest
Running docker:
docker run -it --rm eicdev/ml4fpga:latest bash
--rm
flag Docker automatically cleans up the container and remove the file system when the container exits.-it
flag enables interactive session. Without this flagctrl+c
may not work. In general-it
is used to run e.g. bash session (see below)
Extremely shallow docker crash course
Docker utilizes the pull command to download a package called an image.
When Docker executes the image, it runs within a container.
Each time the docker run
command is called, a new container is generated.
Docker creates a modifiable layer over the image and starts the container with the given command.
If the --rm
flag is not used, stopping the container does not remove it.
A stopped container can be restarted with all its previous changes intact using the docker start command.
Docker functions similarly to tmux or screen, allowing you to reconnect to the running image, attach multiple bash shells, and even reconnect if the container is stopped. This feature facilitates easier debugging and data retention. However, be cautious when using the --rm flag, as it removes the container upon stopping.
Docker documentation: docker run, docker start, --rm flag.
Add your system directory inside docker
You can bind any directory on your system to docker image by using -v flag:
-v <your/directory>:<docker/directory>
# /mnt is a good place to mount directories inside a container
docker run -it --rm -v /host/dir:/mnt/data eicdev/ml4fpga:latest
More information on docker bind, There are also other mechanisms of how to manage data in docker. the official documentation on managing data in docker
Debugging : To do C++ debugging (run GDB or so) one has to specify additional flags (the debugging is switched off by docker by default for security reasons):
--cap-add=SYS_PTRACE --security-opt seccomp=unconfined
Data
All EVIO DATA is here:
/gluonraid3/data4/rawdata/trd/DATA/hd_rawdata_*.evio
Physics runs:
===> 2 crates , 3 detectors : CAL/FA250, GEMTRD/FA125 , GEM/SRS
CODA:: Run_2531 GEMTRD:ok; CAL:ok; SRS:del=0x41; ROSYBC=0x90 3bin; 10APV;OK; TRDpos=100; 5.1M ev *PHYS* !!!!
cDAQ:: Run_2543 GEMTRD:ok; CAL:ok; SRS:del=0x41; ROSYBC=0x90 3bin; 10APV;OK; TRDpos=100; 1.1M ev *PHYS* !!!!
===> 1 crate ; 2 detectors : GEMTRD/FA125 ; GEM/SRS
CODA:: Run_2548 (CODA) GEMTRD:ok; CAL:No; SRS:del=0x40; ROSYBC=0x70 9bin; 10APV;OK; TRDpos=155; 1.5M ev *PHYS* !!!!
cDAQ:: Run_2567 (cDAQ) GEMTRD:ok; CAL:No; SRS:del=0x40; ROSYBC=0x70 9bin; 10APV;OK; TRDpos=155; 3.2M ev *PHYS* !!!!
===> new files Mode8 (RAW) and Mode5 (short)
cDAQ / rawmode (8) ; trd_ti_fp.conf ; 300 nA; 650-700Hz ; 16 MB/s
14:09 Run_2633 (cDAQ) GEMTRD:ok; CAL:on; SRS:del=0x41; ROSYBC=0x70 3bin; 10APV;OK; TRDpos=150; Mode8; 250K evt;
cDAQ / Mode5; thr300 / trd_ti_fp_m5.conf ; 300 nA; 650-700Hz ; 7.5 MB/s
14:48 Run_2635 (cDAQ) GEMTRD:ok; CAL:on; SRS:del=0x41; ROSYBC=0x70 3bin; 10APV;OK; TRDpos=50; Mode5; 250 K evt;
Test setup configuration:
- rocFMWPC1 is a TI master (no TSG), with single fa250 board, the last 3 channels contain data from the calorimeter.
- rocTRD1 is a slave with 4 fa125 boards reading GEMTRD (bank 16) also provides SRS/GEM data - bank 17.
Example of reading evio file:
/mnt/c/eic/data/2023-02_ml4fpga_trd_data/hd_rawdata_002539_000.evio
TCP test sender and receiver
There are two test executables built with the project that could be used for testing
- tcp_sender - can send dummy evio events through tcp
- tcp_receiver - can receive them
To make test loop: sender sends data to receiver - drinks coffee (or what it does):
# In one terminal/process
tcp_receiver
# In another terminal/process
tcp_sender -req=ex -cmd=send -host=localhost:20249
Run flags examples
CDAQ evio example analysis
Run parameters:
jana4ml4fpga
-Pplugins=log,root_output,CDAQfile,example_evio_analysis
-Pjana:debug_plugin_loading=1
-PCDAQEVIOFileSource:LogLevel=trace
-Pexample_evio_analysis:LogLevel=trace
-Pjana:timeout=0
-Pjana:nevents=10
-Pnthreads=1
-Phistsfile=/home/romanov/eic/JANA4ML4FPGA/cmake-build-debug/test.root
/mnt/work/data/2023-03-03-trd-data/hd_rawdata_002633_000.evio
Read EVIO and save to flat root file
jana4ml4fpga
-Pplugins=log,root_output,CDAQfile,flat_tree
-Pnthreads=1
-Phistsfile=output_file_name.root
hd_rawdata_002633_000.evio # <= input file
Useful flags
# Write extended info where plugins are loaded from
-Pjana:debug_plugin_loading=1
# <plugin>:LogLevel usually controls verbosity level of a plugin
-Pflat_tree:LogLevel=trace
# Switches off jana internal watchdog timer
# Setting timeout=0 is needed if processing threads stops,
# i.e. when pausing on debug breakpoint or waiting for connection, etc.
-Pjana:timeout=0
# Set number of events to process
-Pjana:nevents=10000
# Set number of events to skip
-Pjana:nskip=10000