`hdat show suite` displays results for cases that no longer exist

[johndgiese]

i.e., if an HDAT suite S has cases A, B, C

and we run hdat run S, then we delete case B, then run hdat run S, then run hdat show S, hdat will still try to show B even though its case was deleted.

If a case is deleted, we should not display old results for it.

[johndgiese]

@russellkan I don't understand how this commit fixes the bug described in this issue.

Could you confirm that:

1. You can reproduce the bug in question prior to the commit (e.g. maybe use git revert locally to try this out)
2. After re-apply this commit, the bug goes away?

[russellkan]

1: Recreated in locally reverted state at e91cd56:

2: After reapplying the commit:

This works by eliminating cases that no longer exist before passing the cases to resolve_resultspecs. This way, if you remove a specific case, the result remains but show_results will not be called for that case.

[russellkan]

Update:
hdat show <suite> should show all existing cases within the suite specified.

If the case has results but no longer exists, it should not be shown.

If the suite has a new case that has not been run, an error message should be printed to stderr and the command should exit with code 1 but not until the rest of the cases have been printed.

To do this: suites will be passed to resolve_resultspecs and it should be handled within there.

[russellkan]

Update:
hdat show should raise an error the errors as they are encountered, aggregate the error messages, then raise a final AbortError with a consolidated message of all errors to print.

To do:

• Rename resolve_resultspecs to resolve_resultspec
• Create new resolve_resultspecs to call resolve_resultspec on each result spec given as arguments
• Create custom errors and messages to be thrown within resolve_resultspec
• Consolidate errors from resolve_resultspec in resolve_resultspecs and raise an AbortError

Related new issue:

• Add try except to each function that utilizes user-specific code (run, show, diff)