Enhance Performance of get_firewall_info with Multi-Threading
Closed this issue · 0 comments
Summary
The current implementation of the get_firewall_info function fetches system information for each firewall sequentially. This approach can be time-consuming, especially when dealing with a large number of firewalls managed by Panorama. To improve performance and efficiency, it's proposed to refactor this function to use multi-threading, allowing system information for multiple firewalls to be fetched concurrently.
Current Behavior
The get_firewall_info function iterates over a list of Firewall objects, fetching system information for each firewall one at a time.
This sequential processing leads to increased execution time, especially noticeable when the number of firewalls is large.
Expected Behavior
System information for multiple firewalls should be fetched concurrently, leveraging multi-threading.
A significant reduction in the total execution time of the get_firewall_info function.
Proposed Solution
- Introduce a helper function fetch_firewall_info that handles fetching system information for a single firewall.
- Utilize concurrent.futures.ThreadPoolExecutor within get_firewall_info to execute fetch_firewall_info concurrently for each firewall in the input list.
- Aggregate the results from each thread into a list of dictionaries, maintaining the current return structure of get_firewall_info.
Benefits
- Performance Improvement: By fetching firewall information concurrently, the overall execution time of get_firewall_info will be significantly reduced.
- Scalability: This enhancement will make the function more scalable, handling a larger number of firewalls more efficiently.
- Resource Utilization: Improved resource utilization by leveraging I/O wait times to perform other fetches concurrently.
Considerations
- Error Handling: Ensure robust error handling within fetch_firewall_info to manage partial failures or timeouts gracefully.
- Concurrency Limit: Carefully choose the max_workers parameter for ThreadPoolExecutor to balance concurrency and system resource utilization.
- This enhancement is expected to provide a noticeable performance improvement in environments with a large number of firewalls managed by Panorama, enhancing the overall user experience and efficiency of firewall management tasks.