Can we optimize the compilation time of hwinfo?
Closed this issue · 3 comments
the check_items, join_items_by_value and join_items_by_key functions be executed for a long time during hwinfo compilation, Can this be optimized? Two loops are used to remove duplicate information, it's algorithm complexity is O(n2).
Can be optimized by using hash to search for keys or multi-thread parallel computing?
for(item0 = hd->first; item0; item0 = item0->next) { if(item0->remove) continue; for(item1 = item0->next; item1 && !item0->remove; item1 = item1->next) { if(item1->remove) continue;
1 now:
`[root@localhost ids]# time ./check_hd --check --sort --cfile hd_ids.h src/bus src/class src/extra src/special src/scanner src/network src/usb src/usb2 src/isapnp src/monitor src/camera src/tv2 src/tv src/dvb2 src/dvb src/chipcard src/modem src/pcmcia src/s390 src/sdio src/pci src/storage src/sound src/mouse src/braille
data written to "hd.ids"
log written to "hd.log"
statistics:
601 inconsistencies fixed
20 errors, 20 resolved
59755 items in
45309 items out
real 18m34.980s
user 18m31.784s
sys 0m0.295s`
2 after use thread pool:
`time ./check_hd --check --sort --cfile hd_ids.h src/bus src/class src/extra src/special src/scanner src/network src/usb src/usb2 src/isapnp src/monitor src/camera src/tv2 src/tv src/dvb2 src/dvb src/chipcard src/modem src/pcmcia src/s390 src/sdio src/pci src/storage src/sound src/mouse src/braille
data written to "hd.ids"
log written to "hd.log"
statistics:
606 inconsistencies fixed
20 errors, 20 resolved
59755 items in
45309 items out
real 1m19.455s
user 5m14.164s
sys 0m0.414s`
exection time reduced by N times.
We compared the hd_ids.h result, there was no difference.
localhost:/home/c # diff -r multi-thread/hd_ids.h now/hd_ids.h localhost:/home/# md5sum multi-thread/hd_ids.h now/hd_ids.h 43555839e718eed94260ac57af23d249 multi-thread/hd_ids.h 43555839e718eed94260ac57af23d249 now/hd_ids.h localhost:/home/ # md5sum multi-thread/hd_ids_tiny.h now/hd_ids_tiny.h eed2aadf1806e77818caeef37e278366 multi-thread/hd_ids_tiny.h eed2aadf1806e77818caeef37e278366 now/hd_ids_tiny.h
environment Information: virtual machine 16 cpus
[root@localhost ~]# lscpu Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: HiSilicon BIOS Vendor ID: QEMU Model name: Kunpeng-920 BIOS Model name: virt-4.1 Model: 0 Thread(s) per core: 1 Core(s) per socket: 16 Socket(s): 1 Stepping: 0x1 Frequency boost: disabled CPU max MHz: 2600.0000 CPU min MHz: 2600.0000 BogoMIPS: 200.00
Even on a multi-core machine, it takes 15 to 20 minutes to compile hwinfo, which will affects the whole version building time, so we want to optimize it, Please the maintainer review.
No One's Here， Close