Parallelism

[SalOne22]

Add parallelism to bulk optimization

Description:

Our current implementation of bulk optimization is processing images sequentially, which can lead to longer processing times, especially for larger batches of images. We need to add parallelism to improve performance and decrease processing times.

Required Libraries:

• num_cpus (https://crates.io/crates/num_cpus)
• threadpool (https://crates.io/crates/threadpool)

Steps to Implement:

1. Install the num_cpus and threadpool crates using Cargo.
2. Use num_cpus to determine the number of available CPU cores.
3. Use threadpool to create a pool of worker threads equal to the number of available CPU cores.
4. Split the list of images to be optimized into equal-sized batches.
5. Assign each batch of images to a worker thread from the thread pool.
6. Wait for all worker threads to finish processing the assigned batches of images.
7. Collect the optimized images from each worker thread and combine them into a single output folder.

Expected Behavior:

With the addition of parallelism, the bulk optimization process should complete much faster, especially for larger batches of images.

Actual Behavior:

Currently, the bulk optimization process is sequential, leading to longer processing times, especially for larger batches of images.

Note:

Parallelism can significantly improve the performance of the bulk optimization process, but it may require more system resources. It's essential to test the new implementation on different systems to ensure it works well without causing resource contention or other issues.

[SalOne22]

Already implemented