Full integration for JSON Compliance

[mzbroch]

Proposed Functionality

Full integration for ComplianceRule objects of JSON type

• allows to define keys intended for compliance
• allows to read intended/actual configurations stored in JSON format

In the initial phase, JSON ComplianceRule would be enabled automatically per host if all host's platform ComplianceRules are type of JSON.

Use Case

Structured data support
API Based controllers / vendors

[jeffkala]

Probably should migrate the bones to jdiff.

[itdependsnetworks]

Can jdiff introspect two models to compare?

[mzbroch]

Can we get into agreement :

• (*) One of the Platform categories could be "JSON_BASED"
• If all of the compliance rules for the given platform are type of JSON, platform is treated as JSON_BASED
• JSON_BASED relevant intended / backup configuration files are interpreted as .json files and loaded as python dictionaries
• compliance checks for the JSON_BASED understands match_config attribute and interpretes as a list of keys to be compared between backup / intended

@jeffkala @itdependsnetworks This is the prototype that implements above :
develop...mzb-json

JDiff vs DeepDiff is the discussion about the diff engine, I would like to have it but would prefer to focus on the ☝️ first

[pato23arg]

Can we get into agreement :

* (*) One of the Platform categories could be "`JSON_BASED`"

* If all of the compliance rules for the given platform are type of `JSON`, platform is treated as `JSON_BASED`

* `JSON_BASED` relevant intended / backup configuration files are interpreted as `.json` files and loaded as python dictionaries

* compliance checks for the `JSON_BASED` understands `match_config` attribute and interpretes as a list of keys to be compared between backup / intended

@jeffkala @itdependsnetworks This is the prototype that implements above : develop...mzb-json

JDiff vs DeepDiff is the discussion about the diff engine, I would like to have it but would prefer to focus on the ☝️ first

@jeffkala @itdependsnetworks any chance I can get collaborator access to this branch? develop...mzb-json

[jeffkala]

Can we get into agreement :

• (*) One of the Platform categories could be "JSON_BASED"
• If all of the compliance rules for the given platform are type of JSON, platform is treated as JSON_BASED
• JSON_BASED relevant intended / backup configuration files are interpreted as .json files and loaded as python dictionaries
• compliance checks for the JSON_BASED understands match_config attribute and interpretes as a list of keys to be compared between backup / intended

@jeffkala @itdependsnetworks This is the prototype that implements above : develop...mzb-json

JDiff vs DeepDiff is the discussion about the diff engine, I would like to have it but would prefer to focus on the ☝️ first

What does "platform category" mean?
Is this to avoid having arista_eos aaa rule of type CLI and JSON? I'm not following this fully. The definition of the "type" is already defined at the compliance rule level.

[mzbroch]

What does "platform category" mean? Is this to avoid having arista_eos aaa rule of type CLI and JSON? I'm not following this fully. The definition of the "type" is already defined at the compliance rule level.

As of today, the rule type is defined on ComplianceRule object.

Having mixed types of ComplianceRule (json & cli), would lead to a problem how to get a cli or json config per each rule to perform compliance.

I propose to officially support two configuration standards: .json and .cfg (cli). The json config could be automatically detected and parsed - perhaps we could try/except json.loads(configuration) and determine if format is JSON.

[itdependsnetworks]

Perhaps we require the files to be have a suffix of .json?

Though, I am not still getting it, what does that exactly buy us? We still need to delineate between cli and custom.

[mzbroch]

Perhaps we require the files to be have a suffix of .json?

Though, I am not still getting it, what does that exactly buy us? We still need to delineate between cli and custom.

It gives us ability to automatically execute JSON compliance based on detected actual / intended configs :

    for rule in rules[obj.platform.slug]:
        # using update_or_create() method to conveniently update actual obj or create new one.
        if backup_json and intended_json:
            _actual = {k: backup_json.get(k) for k in rule.config_to_match.splitlines() if rule.config_to_match}
            _intended = {k: intended_json.get(k) for k in rule.config_to_match.splitlines() if rule.config_to_match}
        else:
            _actual = section_config(rule, backup_cfg, get_platform(platform))
            _intended = section_config(rule, intended_cfg, get_platform(platform))

        ConfigCompliance.objects.update_or_create(
            device=obj,
            rule=rule["obj"],
            defaults={
                "actual": _actual,
                "intended": _intended,
                "missing": "",
                "extra": "",
            },

[itdependsnetworks]

But how do we handle custom then?

[pato23arg]

But how do we handle custom then?

@itdependsnetworks I believe Custom should remain as it is: https://docs.nautobot.com/projects/golden-config/en/latest/user/app_feature_compliancecustom/

[itdependsnetworks]

But I don't understand how does auto-detection plus this would work.

[pato23arg]

I think this is the auto-detection criteria:

"Applying a match_config presumes it is CLI type and not having one presumes it is JSON type."

Would change to:

Feature-Based JSON: match_config set to API endpoint(s)
Config-Based JSON: match_config not set
Feature-Based CLI: match_config set to config line(s) to match

It would still auto detect, but with one additional condition (Feature-Based JSON).

@mzbroch do you agree?

[mzbroch]

But I don't understand how does auto-detection plus this would work.

auto detection:

1. Attempt to safely json.loads(configuration) to determine if configuration is stored in JSON format
2. config_to_match would work in similar to regex - pickup tier-1 keys from json dictionary, if those keys are defined via rule.config_to_match

Custom compliance is not affected.
What is indeed changed, is the ability to recognize JSON structures automatically and bypass netutils with parser checks. If JSON config is recognized, it is saved under ConfigCompliance object, then when this object is .saved() actual compliance of type JSON/Regex/Custom happens.

[jeffkala]

fixed in #485 and #487