Implement critical system log capture before and after upgrades

[cdot65]

Is your feature request related to a problem? Please describe.
When upgrading PAN-OS on firewalls using the pan-os-upgrade utility, it is crucial to capture and analyze critical system logs before and after the upgrade process. These logs can provide valuable insights into the health and stability of the firewall, as well as help identify any potential issues or anomalies that may occur during or after the upgrade. Currently, the utility does not have a built-in mechanism to capture and store critical system logs, which may make it difficult to troubleshoot or investigate upgrade-related problems.

Describe the solution you'd like
Enhance the pan-os-upgrade utility to capture critical system logs on the firewalls before and after the upgrade process. The utility should:

1. Before initiating the upgrade, identify the critical system logs that need to be captured based on predefined criteria or user-specified configuration.
2. Execute the appropriate CLI commands (e.g., show log system) or API calls to retrieve the specified system logs from the firewall.
3. Store the captured logs in a structured format (e.g., JSON or XML) along with metadata such as the timestamp, device information, and log type.
4. Proceed with the normal upgrade process.
5. After the upgrade is completed and the firewall is back online, re-capture the same set of critical system logs from the upgraded firewall.
6. Store the post-upgrade logs in a similar structured format as the pre-upgrade logs.
7. Perform a comparison between the pre-upgrade and post-upgrade logs to identify any significant changes, error messages, or anomalies.
8. Generate a report or display the comparison results to the user, highlighting any potential issues or areas that require further investigation.
9. Provide an option to export or archive the captured logs for future reference or deeper analysis.

Describe alternatives you've considered
An alternative approach could be to rely on external log management solutions or SIEMs to capture and analyze the system logs. However, this would require additional integrations and may not provide a seamless experience within the pan-os-upgrade utility itself. Moreover, capturing logs directly through the utility ensures that the relevant logs are available even if external logging systems are not accessible or properly configured.

Additional context
Here are a few additional points to consider:

• Allow users to customize the list of critical system logs to capture based on their specific requirements or organizational policies.
• Handle scenarios where the log capture commands may fail or return unexpected output, and provide appropriate error handling and logging.
• Implement log rotation or purging mechanisms to prevent excessive storage consumption, especially for large-scale deployments with multiple firewalls.
• Provide guidelines and best practices on interpreting the log comparison results and identifying potential issues.
• Consider integrating with external log analysis tools or machine learning algorithms to automatically detect anomalies or patterns in the captured logs.
• Update the project's documentation to include information about this new feature, explaining how it helps monitor and troubleshoot the upgrade process through system log analysis.

By implementing this feature, the pan-os-upgrade utility will provide a comprehensive approach to capturing and analyzing critical system logs before and after the upgrade process. This will enable users to proactively identify and address any potential issues, ensure the stability and reliability of the upgraded firewalls, and facilitate effective troubleshooting and root cause analysis.