RobertKrawitz/OpenShift4-tools

Json error when passing custom analyze path

Closed this issue · 4 comments

ebattat commented

When running ClusterBuster as following:

./run-kata-perf-suite --run_type=test_ci --client-pin-node=master-0 --server-pin-node=master-1 --sync-pin-node=master-1 --basename=clusterbuster --artifactdir=/tmp/benchmark-runner-run-artifacts/clusterbuster-2022-07-05-18-55-59 --analyze=/tmp/benchmark-runner-run-artifacts/clusterbuster-2022-07-05-18-55-59/clusterbuster-report.json

Get the following result:

Run times:
      1:16 cpusoaker-runc-0000-10
      1:14 uperf-runc-0000-8192B-1i-1P-stream
      1:15 uperf-runc-0001-8192B-1i-1P-rr
      1:30 fio-runc-0000-1P
      2:35 files-runc-0000-1P-32D-32F-4096B-4096S-0T
      1:20 cpusoaker-kata-0000-10
      1:20 uperf-kata-0000-8192B-1i-1P-stream
      1:20 uperf-kata-0001-8192B-1i-1P-rr
      1:41 fio-kata-0000-1P
      2:36 files-kata-0000-1P-32D-32F-4096B-4096S-0T
Run took 16:07 (Passed)
Traceback (most recent call last):
  File "/tmp/OpenShift4-tools/analyze-clusterbuster-report", line 28, in <module>
    data = ClusterBusterLoader(args.files).Load()
  File "/tmp/OpenShift4-tools/lib/clusterbuster/loader/ClusterBusterLoader.py", line 81, in __init__
    self.reports = ClusterBusterReporter.report(dirs_and_files, format="json-summary")
  File "/tmp/OpenShift4-tools/lib/clusterbuster/reporter/ClusterBusterReporter.py", line 97, in report
    answers.append(ClusterBusterReporter.report_one(json.load(f), format, **kwargs))
  File "/usr/lib64/python3.6/json/__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This is not happening when running:
./run-kata-perf-suite --run_type=test_ci --client-pin-node=master-0 --server-pin-node=master-1 --sync-pin-node=master-1 --basename=clusterbuster --artifactdir=/tmp/benchmark-runner-run-artifacts/clusterbuster-2022-07-05-18-55-59 --analyze=/tmp/clusterbuster-report.json

Why analyze path must be /tmp/clusterbuster-report.json ?

RobertKrawitz commented

Thanks for opening the issue. I will look into it. This is absolutely not correct behavior; it should be possible to save the report inside the artifact directory.

RobertKrawitz commented

Issue is reproduced.

RobertKrawitz commented

The issue is specifically when the analyze file is named ${artifactdir}/clusterbuster-report.json. This file is being incorrectly recognized as a job report, which the analyzer attempts to process. For example, this fails:

run-kata-perf-suite --run_type=test_ci --client-pin-node=worker-0 --server-pin-node=worker-1 --sync-pin-node=worker-1 --basename=clusterbuster --artifactdir=/tmp/benchmark-runner-run-artifacts/clusterbuster-2022-07-05-18-55-59 --analyze=/tmp/benchmark-runner-run-artifacts/clusterbuster-2022-07-05-18-55-59/clusterbuster-report.json --runtimeclass=kata --runtime=1 cpusoaker

but this works:

run-kata-perf-suite --run_type=test_ci --client-pin-node=worker-0 --server-pin-node=worker-1 --sync-pin-node=worker-1 --basename=clusterbuster --artifactdir=/tmp/benchmark-runner-run-artifacts/clusterbuster-2022-07-05-18-55-59 --analyze=/tmp/benchmark-runner-run-artifacts/clusterbuster-2022-07-05-18-55-59/clusterbuster-report-x.json --runtimeclass=kata --runtime=1 cpusoaker

I will investigate a fix.