Implement optional counter checks for PAN-OS features after upgrade

[cdot65]

Is your feature request related to a problem? Please describe.
After upgrading PAN-OS on a firewall using the pan-os-upgrade utility, it is important to ensure that various PAN-OS features and functionalities are working correctly. One way to verify this is by checking if the counters associated with specific features are incrementing as expected. Currently, the utility does not provide an option to check these counters, which may lead to scenarios where the upgrade is considered successful, but certain features are not functioning properly.

Describe the solution you'd like
Enhance the pan-os-upgrade utility to include an optional, opt-in feature that allows users to specify counters for different PAN-OS features and check if those counters are incrementing after the upgrade. The utility should:

1. Provide a configuration option or command-line argument that enables users to opt-in to the counter check feature.
2. Allow users to declare a list of counters they want to monitor, such as GP user count, user-IP mapping, TCP sessions, or any other relevant counters.
3. After the upgrade is completed and the firewall is back online, retrieve the specified counter values using the appropriate CLI commands or API calls.
4. Implement a mechanism to monitor the counters over a configurable period of time (e.g., every 5 minutes for 30 minutes) to observe if they are incrementing.
5. Compare the initial counter values with the values obtained during the monitoring period to determine if there is a steady increase.
6. If the counters are incrementing as expected, consider the feature as functioning correctly and proceed with the upgrade process.
7. If the counters are not incrementing or are showing unexpected behavior, display a warning message to the user and log the details for further investigation.

Describe alternatives you've considered
An alternative approach could be to perform a more comprehensive health check of the firewall after the upgrade, including checking various logs, system resources, and running a series of diagnostic commands. However, this might be more complex and time-consuming, and it may not specifically focus on the incrementing behavior of counters associated with specific features.

Additional context
Here are a few additional points to consider:

• The list of counters to monitor should be customizable and easily extensible to accommodate different PAN-OS features and customer requirements.
• The utility should provide clear and informative messages about the counter check process, including the counters being monitored, the monitoring interval, and the results of the check.
• In case of any counters not incrementing as expected, the utility should provide troubleshooting suggestions or recommend contacting support for further assistance.
• Consider adding a threshold or percentage-based approach to determine if the counter increment is sufficient, as the increment rate may vary depending on the network traffic and usage patterns.
• Update the project's documentation to include information about this new feature, explaining how to enable and configure the counter checks and interpret the results.

By implementing this feature, the pan-os-upgrade utility will provide an additional level of assurance by verifying that specific PAN-OS features are functioning correctly after the upgrade. This will help identify potential issues early and ensure that the upgraded firewall is operating as expected, enhancing the overall reliability and stability of the system.