Idea: display only `Failure`/`Success`/`Controlled`

[gmiu]

I think it would be useful (especially for testing) to show, using kwargs, just some of the results. Something like:

• -f / --failure: shows only the Failure list
• -s / --success: shows only the Success list
• -c / --controlled: shows only the Controlled list

[cedwards]

I'm open to the idea, but unsure the best implementation. @basepi ?

[basepi]

Right now we already have show_success. I see no reason anyone would ever want only the successes and not the failures. Failures are why we're here.

Controlled is a little bit weirder. Right now we treat it as success for compliance computation, but it's not really the same as success....

I'm thinking we add a new config value, hubblestack.nova.treat_controlled_as_success, which will default to True. And we'll remove Controlled and Success when show_success is False. Thoughts?

[gmiu]

I came with the idea because I had to scroll up to the top of the results just to see just one result I was looking for. BTW, is targeting just some of the checks available anymore (as we did it at the beginning, with regular expressions: hubble.audit CIS-4.2.\*)?

[gmiu]

And, if hubblestack.nova.treat_controlled_as_success = False, they will be counted as failures?

[basepi]

Yes, targeting is still available, both in hubble.audit, and in topfiles.

No, controlled will never be counted as failures. This is a confusing configuration value. Maybe instead we should just add show_controlled as well.

I understand that right now you're interested in what's at the top (because Failures are generally at the bottom). But in the wild, I think people will only be interested in failures.

[basepi]

And I should also note that in the wild, people will actually control checks and/or update their boxes until there are no failures. Then the output will be much shorter (at least with show_success and show_controlled set to False).

[dgmorrisjr]

so, currently, are inspections that include the control tag only organized into the control section of the output? i.e. not grouped together with successes if the test would normally be a success?

[cedwards]

control checks are simply skipped and put into the "Controlled" section of the output. They are not listed at all under Success or Failure.

[dgmorrisjr]

Should they be executed, and if successful, then go under success. but if they fail, then go under control?

[cedwards]

I'm curious @grimamethyst opinion here.

[gmiu]

@cedwards / @dgmorrisjr, I think it's better just to skip the control checks. Why wasting time on making a check that doesn't matter anyway? Even if it's a small amount of time, it's still wasted (I usually try to follow the perfection). In the current design, for a tag_data that contains the control key, we just continue the for loop that iterates through the tags to be checked. No time is wasted on doing that check. If we'd really want to check the control ones, we could add a new option that signals hubble to do so.

[dgmorrisjr]

IF the inspection with a control option were processed, here is a situation where I think the tag is a bit helpful:

    ip6tables_should_be_running:
      data:
        Red Hat Enterprise Linux Server-6:
        - ip6tables:      V-38551
      description: |
        (Medium) The os must connect to external networks or information systems
        only through managed IPv6 interfaces consisting of boundary protection
        devices arranged in accordance with an organizational security architeture.
      control: |
        If the system is a cross-domain system, this is not applicable.
        If IPv6 is disabled, this is not applicable.

for the STIGS, in most systems, ip6tables must be running. however, there are two situations where the control is not applicable. So ideally here is the logic I think would be helpful:

if ip6tables service is running then = success
else list it under "control" so that it can manually be determined if it's a failure or success.

if it's under the failure group, then you have to sift through all of those to find these weird situations.

[basepi]

@dgmorrisjr What would this logic look like? Would it be a bash command (like with unless and onlyif in salt)? I've thought about having specific logic-based control but haven't thought of a really good way to implement it.

I should note that for the separate control configs, the audits will likely be run, and only failures will end up in the Controlled list. This is because separate control configs will be processed by hubble.py, post-audit.

[dgmorrisjr]

I think supporting command requisites could be one way of doing that. maybe something like this:

ip6tables_should_be_running:
      data:
        Red Hat Enterprise Linux Server-6:
        - ip6tables:      V-38551
          unless:
            - grep "install ipv6 /bin/true" /etc/modprobe.d/modprobe.conf
            - grep -i "NETWORKING_IPV6=no" /etc/sysconfig/network-scripts/ifcfg-eth*
            - grep -i "IPV6INIT=no" /etc/sysconfig/network-scripts/ifcfg-eth*

but within the unless construct, is it possible to specify an OR or an AND logic?

[basepi]

In salt, unless doesn't support a list, only a single command, so AND and OR aren't a thing there. I'll have to think if there's a way to make it happen for us.

[dgmorrisjr]

Got it, more than anything I was trying to provide a working example of the types of checks that might need to be done. I think there are simpler situations where there is just one value you want to validate against... but the condition (e.g. the unless) could be:

• value in a file(s) (e.g. setting in sysctl),
• the presence of a file(s) (e.g. file in /etc/modprobe.d),
• thx output of a command (e.g. chkconfig, rpm, etc),
• combinations using AND / OR logic

[basepi]

Yeah, I understand where you're coming from. I'll have to think on this.

[basepi]

So, focusing back on the core of this issue, I think we should just add a show_controlled argument and configuration setting like we did for show_success. As far as requisites and whatnot, we should move the discussion about those into new issues.

[basepi]

This issue was moved to hubblestack/hubble-salt#33