/corentin

dispatch task on nodes with make

Primary LanguageMakefile

Table of Contents

What's new

  • The fibs-rr.mk variant, now with round-robin locked allocation
  • Generated 99 random invocation of fib (max 33) on my 5 test nodes
  • Use outupt-sync make option
  • Display sequence of task, time of run, name of node, arg of fib and result

A priori

  • If the number of processe (-j) is less than the number of nodes no nodes will run more than a task at a time
  • If the number of processe (-j) is more than the number of nodes all nodes may run more than a task at a time

Sequentially on all nodes

thy@tde-ws:~/usr/hub/work/ext/corentin$ fibs-rr.mk clean; time fibs-rr.mk
01 1638307399 tst2 18 2584
02 1638307401 tst4 31 1346269
03 1638307402 tst5 16 987
04 1638307402 tst6 1 1
05 1638307404 tst8 19 4181
06 1638307403 tst2 23 28657
07 1638307406 tst4 0 1
08 1638307404 tst5 0 1
09 1638307405 tst6 8 21
10 1638307407 tst8 5 5
11 1638307406 tst2 17 1597
12 1638307408 tst4 23 28657
13 1638307407 tst5 26 121393
14 1638307408 tst6 30 832040
15 1638307411 tst8 22 17711
16 1638307410 tst2 10 55
17 1638307412 tst4 29 514229
18 1638307411 tst5 29 514229
19 1638307412 tst6 19 4181
20 1638307415 tst8 5 5
21 1638307413 tst2 25 75025
22 1638307416 tst4 8 21
23 1638307414 tst5 1 1
24 1638307415 tst6 8 21
25 1638307417 tst8 28 317811
26 1638307416 tst2 4 3
27 1638307419 tst4 28 317811
28 1638307418 tst5 5 5
29 1638307418 tst6 16 987
30 1638307421 tst8 1 1
31 1638307419 tst2 4 3
32 1638307422 tst4 3 2
33 1638307421 tst5 17 1597
34 1638307421 tst6 20 6765
35 1638307423 tst8 16 987
36 1638307422 tst2 30 832040
37 1638307425 tst4 6 8
38 1638307424 tst5 12 144
39 1638307425 tst6 31 1346269
40 1638307428 tst8 10 55
41 1638307427 tst2 27 196418
42 1638307430 tst4 30 832040
43 1638307429 tst5 25 75025
44 1638307430 tst6 26 121393
45 1638307432 tst8 23 28657
46 1638307431 tst2 15 610
47 1638307433 tst4 32 2178309
48 1638307434 tst5 3 2
49 1638307435 tst6 7 13
50 1638307437 tst8 8 21
51 1638307436 tst2 13 233
52 1638307438 tst4 27 196418
53 1638307437 tst5 25 75025
54 1638307438 tst6 11 89
55 1638307440 tst8 21 10946
56 1638307439 tst2 30 832040
57 1638307442 tst4 23 28657
58 1638307441 tst5 31 1346269
59 1638307442 tst6 8 21
60 1638307445 tst8 21 10946
61 1638307443 tst2 16 987
62 1638307446 tst4 10 55
63 1638307445 tst5 16 987
64 1638307445 tst6 16 987
65 1638307448 tst8 23 28657
66 1638307446 tst2 8 21
67 1638307449 tst4 32 2178309
68 1638307450 tst5 0 1
69 1638307450 tst6 29 514229
70 1638307453 tst8 26 121393
71 1638307452 tst2 2 1
72 1638307454 tst4 18 2584
73 1638307453 tst5 18 2584
74 1638307453 tst6 6 8
75 1638307456 tst8 30 832040
76 1638307455 tst2 4 3
77 1638307458 tst4 27 196418
78 1638307457 tst5 19 4181
79 1638307457 tst6 15 610
80 1638307460 tst8 12 144
81 1638307458 tst2 12 144
82 1638307461 tst4 2 1
83 1638307459 tst5 19 4181
84 1638307460 tst6 30 832040
85 1638307463 tst8 25 75025
86 1638307462 tst2 15 610
87 1638307464 tst4 28 317811
88 1638307463 tst5 13 233
89 1638307464 tst6 25 75025
90 1638307466 tst8 32 2178309
91 1638307467 tst2 18 2584
92 1638307470 tst4 8 21
93 1638307468 tst5 22 17711
94 1638307469 tst6 2 1
95 1638307471 tst8 3 2
96 1638307470 tst2 1 1
97 1638307472 tst4 8 21
98 1638307471 tst5 18 2584
99 1638307472 tst6 0 1

real	1m12.966s
user	0m6.852s
sys	0m3.496s

Concurently on all nodes

thy@tde-ws:~/usr/hub/work/ext/corentin$ fibs-rr.mk clean; time fibs-rr.mk -j10
01 1638307497 tst2 18 2584
05 1638307497 tst5 19 4181
08 1638307497 tst6 0 1
06 1638307497 tst2 23 28657
04 1638307499 tst4 1 1
03 1638307497 tst5 16 987
07 1638307497 tst6 0 1
09 1638307499 tst8 8 21
10 1638307499 tst4 5 5
11 1638307498 tst2 17 1597
12 1638307499 tst4 23 28657
16 1638307498 tst2 10 55
15 1638307499 tst8 22 17711
13 1638307498 tst5 26 121393
19 1638307498 tst6 19 4181
20 1638307500 tst8 5 5
22 1638307498 tst2 8 21
23 1638307498 tst5 1 1
24 1638307498 tst6 8 21
18 1638307498 tst5 29 514229
17 1638307500 tst4 29 514229
21 1638307500 tst4 25 75025
14 1638307498 tst6 30 832040
26 1638307499 tst2 4 3
29 1638307499 tst5 16 987
28 1638307499 tst6 5 5
30 1638307501 tst8 1 1
31 1638307499 tst2 4 3
25 1638307500 tst8 28 317811
32 1638307501 tst4 3 2
33 1638307499 tst5 17 1597
27 1638307500 tst4 28 317811
02 1638307499 tst8 31 1346269
34 1638307499 tst6 20 6765
35 1638307501 tst8 16 987
38 1638307499 tst5 12 144
37 1638307501 tst4 6 8
40 1638307501 tst8 10 55
43 1638307500 tst5 25 75025
41 1638307499 tst2 27 196418
45 1638307502 tst8 23 28657
46 1638307500 tst2 15 610
44 1638307500 tst6 26 121393
48 1638307500 tst5 3 2
49 1638307500 tst6 7 13
36 1638307499 tst2 30 832040
51 1638307500 tst2 13 233
53 1638307500 tst5 25 75025
50 1638307502 tst8 8 21
54 1638307500 tst6 11 89
55 1638307502 tst8 21 10946
59 1638307501 tst6 8 21
39 1638307499 tst6 31 1346269
60 1638307503 tst8 21 10946
57 1638307503 tst4 23 28657
52 1638307502 tst4 27 196418
61 1638307501 tst2 16 987
63 1638307501 tst5 16 987
62 1638307503 tst4 10 55
64 1638307501 tst6 16 987
42 1638307501 tst4 30 832040
65 1638307503 tst8 23 28657
66 1638307501 tst2 8 21
56 1638307501 tst2 30 832040
68 1638307502 tst5 0 1
71 1638307502 tst2 2 1
73 1638307502 tst5 18 2584
74 1638307502 tst6 6 8
72 1638307504 tst4 18 2584
70 1638307504 tst8 26 121393
58 1638307501 tst5 31 1346269
69 1638307502 tst6 29 514229
76 1638307502 tst2 4 3
78 1638307502 tst5 19 4181
80 1638307502 tst6 12 144
81 1638307502 tst2 12 144
79 1638307504 tst8 15 610
82 1638307505 tst4 2 1
83 1638307503 tst5 19 4181
86 1638307503 tst2 15 610
85 1638307505 tst8 25 75025
77 1638307504 tst4 27 196418
75 1638307504 tst8 30 832040
88 1638307503 tst5 13 233
89 1638307503 tst6 25 75025
91 1638307504 tst2 18 2584
93 1638307504 tst5 22 17711
84 1638307503 tst6 30 832040
92 1638307506 tst4 8 21
94 1638307504 tst6 2 1
95 1638307506 tst8 3 2
96 1638307504 tst2 1 1
87 1638307505 tst4 28 317811
98 1638307504 tst5 18 2584
99 1638307504 tst6 0 1
97 1638307506 tst4 8 21
47 1638307502 tst4 32 2178309
90 1638307505 tst8 32 2178309
67 1638307503 tst4 32 2178309

real	0m9.459s
user	0m5.499s
sys	0m1.938s

The simpler version

fibs.mk Dispatch task on nodes with make

Test on a handful test VM

thy@tdews2:~$ time fibs.mk
mkdir tmp/fib
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 1346269 2178309 3524578
real	0m29.881s
user	0m3.233s
sys	0m2.003s
thy@tdews2:~$ fibs.mk clean
rm -r tmp/fib
thy@tdews2:~$ time fibs.mk -j 4
mkdir tmp/fib
02 01 04 03 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 1346269 2178309 3524578
real	0m11.799s
user	0m2.855s
sys	0m1.525s
thy@tdews2:~$ fibs.mk clean
rm -r tmp/fib
thy@tdews2:~$ time fibs.mk -j 8
mkdir tmp/fib
01 02 04 03 05 06 07 08 09 10 11 13 12 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 1346269 2178309 3524578
real	0m7.270s
user	0m2.340s
sys	0m1.483s