adaptive mpi: Number of worker threads stuck to one

[lucaparisi91]

Hello,
I am unable to launch with more than one working thread per processor and I was wandering if anyone knew how to set the number of worker threads . Thanks in advance.

Background

I compiled a toy code running independent tasks on different ranks. Each task takes a different amount of time. I compiled and run with adaptive mpi but found no improvement on load balancing

Issue

I run the application with

 charmrun  +p36 ./toy.ampi +vp 360 +balancer RefineLB

However it seems only one working thread is spawned per process, with 10 MPI ranks each. Is there a way to increase the number of working threads ? I could node find anything about this in the documentation

Additional details

I built adaptive mpi with

./build AMPI-only ucx-linux-x86_64  --with-production  --enable-tracing -j8

The charm output I get is

Charm++> Running in non-SMP mode: 36 processes (PEs)
Converse/Charm++ Commit ID: v70000
Charm++: Tracemode Projections enabled.
Trace: traceroot: /mnt/lustre/indy2lfs/work/dc150/dc150/lparisi-answ/monk/charm_test_bed/toy_imbalance/./toy.ampi
Charm++> Using STL-based msgQ:
Charm++> Message priorities have been turned off and will not be respected.
Isomalloc> Synchronized global address space.
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 hosts (2 sockets x 18 cores x 2 PUs = 72-way SMP)
Charm++> cpu topology info is gathered in 0.037 seconds.
[0] TreeLB in LEGACY MODE support
[0] TreeLB: Using PE_Root tree with: RefineA 
 Using          360  MPI ranks.
 Time:    12.354672083999999     
[Partition 0][Node 0] End of program

[stwhite91]

The program output here says that you have launched 36 processes across 36 cores (2 sockets with 18 cores each) using 360 ranks (10 per process or core). If you want to launch multiple worker threads per process, you need to build Charm/AMPI in SMP mode, as in ./build AMPI-only ucx-linux-x86_64 smp --with-production --enable-tracing -j8. Then you will need to use charmrun's ppn or other options described here: https://charm.readthedocs.io/en/latest/charm++/manual.html#smp-options

[lucaparisi91]

That worked, thanks for the fast reply.
I misunderstood the concept of SMP mode.