Samsung/Universum

Static_analysis_report.json has incorrect format if there is more than one code report step

i-keliukh opened this issue · 4 comments

Description

If multiple config steps are marked with code_report=True, the Static_analysis_report.json file created by universum is not valid json.

Steps to reproduce

  1. Create config with at least two code report steps, for example:
from universum.configuration_support import Configuration, Step

configs = Configuration([Step(name='Run pylint', command=['python', '-m', 'universum.analyzers.pylint', '--result-file', '${CODE_REPORT_FILE}', '--files', 'source_file.py'], code_report=True)])
configs += Configuration([Step(name='Run mypy', command=['python', '-m', 'universum.analyzers.mypy', '--result-file', '${CODE_REPORT_FILE}', '--files', 'source_file.py'], code_report=True)])
  1. Create source file that fails checks of both analyzers, for example:
"Docstring."

s: int = "world"
print(f"Hello {s}.")
  1. Run the above config against the source file passing --no-finalize option to universum or check it on real code review system, such as Github or Gerrit
  2. If running locally, check the Static_analysis_report.json file in artifacts directory

Expected result

  1. The Static_analysis_report.json must be the correct json file

Actual result

The Static_analysis_report.json is not valid json: it is a concatenation of two json files with arrays, which is not valid.

Environment

  • OS: Ubuntu 22.0.4
  • Python version 3.10
  • Universum version: 91afad6

Additional context

The bug is located here:

for report_file in reports:
with open(report_file, "r", encoding="utf-8") as f:
text: str = f.read()
report: Optional[List[Dict[str, str]]] = None
if text:
report = json.loads(text)
json_file: TextIO = self.artifacts.create_text_file("Static_analysis_report.json")
json_file.write(json.dumps(report, indent=4))

shouldn't code reports be delivered via

self.reporter.code_report(path, message)

that is called later?

The issue is than write doesn't clear the file and in the second and further iterations the file is not valid json

as far as I know, "Static_analysis_report.json" is not used and can be safely deleted. Actually, I would prefer if we removed it from Universum logic to prevent future confusion

we have reproduced an issue with multiple code reports not working together, trying to narrow down the issue