Different attributes used in different representations of the test results
pevogam opened this issue · 1 comments
Describe the bug
It seems that dictionaries for test results retrieved via job.result.tests have different attributes for the same data than test results retrieved via the results.json file of a previous job.
Steps to reproduce
For instance, try to compare test duration from the two sources:
# current job
{'result': 'PASS',
'fail_reason': '',
'class_name': 'VirtTest',
'fail_class': '',
'traceback': '',
'status': 'PASS',
'time': 71943.550614783,
'job_logdir': '/root/avocado/job-results/job-2023-12-06T21.26-e7a19e8',
'job_unique_id': 'e7a19e83552bf98d65d72c51293ec714ae414921',
'base_path': '/root/avocado/job-results/job-2023-12-06T21.26-e7a19e8/test-results',
'logfile': '/root/avocado/job-results/job-2023-12-06T21.26-e7a19e8/test-results/01-c101vm1-normal.nongui.quicktest.tutorial1.vms.vm1.qemu_kvm_centos.default_bios.no_9p_export.smallpages.no_pci_assignable.qcow2.virtio_blk.smp2.virtio_net.Linux.CentOS.8.0.x86_64.nets.net1.cluster1/debug.log',
'task_path': '/root/avocado/job-results/job-2023-12-06T21.26-e7a19e8/test-results/01-c101vm1-normal.nongui.quicktest.tutorial1.vms.vm1.qemu_kvm_centos.default_bios.no_9p_export.smallpages.no_pci_assignable.qcow2.virtio_blk.smp2.virtio_net.Linux.CentOS.8.0.x86_64.nets.net1.cluster1',
'time_start': 71920.510717456,
'actual_time_start': 1701869222.4892325,
'name': '01-c101vm1-normal.nongui.quicktest.tutorial1.vms.vm1.qemu_kvm_centos.default_bios.no_9p_export.smallpages.no_pci_assignable.qcow2.virtio_blk.smp2.virtio_net.Linux.CentOS.8.0.x86_64.nets.net1.cluster1',
'params': [],
'whiteboard': "{'kvm_version': '6.2.7-100.fc36.x86_64',
'qemu_version': '5.1.0 (qemu-5.1.0-9.fc36)'}",
'time_end': 71943.550614783,
'actual_time_end': 1701869245.5677369,
'time_elapsed': 23.039897327005747,
'logdir': '/root/avocado/job-results/job-2023-12-06T21.26-e7a19e8/test-results/01-c101vm1-normal.nongui.quicktest.tutorial1.vms.vm1.qemu_kvm_centos.default_bios.no_9p_export.smallpages.no_pci_assignable.qcow2.virtio_blk.smp2.virtio_net.Linux.CentOS.8.0.x86_64.nets.net1.cluster1',
'tags': {}}
# previous job
{'actual_end': 1701869245.5677369,
'actual_start': 1701869222.4892325,
'end': 71943.550614783,
'fail_reason': '',
'id': '01-c101vm1-normal.nongui.quicktest.tutorial1.vms.vm1.qemu_kvm_centos.default_bios.no_9p_export.smallpages.no_pci_assignable.qcow2.virtio_blk.smp2.virtio_net.Linux.CentOS.8.0.x86_64.nets.net1.cluster1',
'logdir': '/root/avocado/job-results/job-2023-12-06T21.26-e7a19e8/test-results/01-c101vm1-normal.nongui.quicktest.tutorial1.vms.vm1.qemu_kvm_centos.default_bios.no_9p_export.smallpages.no_pci_assignable.qcow2.virtio_blk.smp2.virtio_net.Linux.CentOS.8.0.x86_64.nets.net1.cluster1',
'logfile': '/root/avocado/job-results/job-2023-12-06T21.26-e7a19e8/test-results/01-c101vm1-normal.nongui.quicktest.tutorial1.vms.vm1.qemu_kvm_centos.default_bios.no_9p_export.smallpages.no_pci_assignable.qcow2.virtio_blk.smp2.virtio_net.Linux.CentOS.8.0.x86_64.nets.net1.cluster1/debug.log',
'name': 'normal.nongui.quicktest.tutorial1.vms.vm1.qemu_kvm_centos.default_bios.no_9p_export.smallpages.no_pci_assignable.qcow2.virtio_blk.smp2.virtio_net.Linux.CentOS.8.0.x86_64.nets.net1.cluster1',
'start': 71920.510717456,
'status': 'PASS',
'tags': {},
'time': 23.039897327005747,
'whiteboard': "{'kvm_version': '6.2.7-100.fc36.x86_64',
'qemu_version': '5.1.0 (qemu-5.1.0-9.fc36)'}"}]
The first one uses time_elapsed and the second uses time. To make matters more ambiguous, the first also uses time but in a difference sense that doesn't coincide neither with time_start nor time_end.
Expected behavior
Job replay functionality as well as general functionality needing to compare currently run tests with reference tests from previous jobs should be able to use unique and easy to map if not fully equivalent attributes per test result.
Current behavior
Confusing use of the same keywords for different things that are meant in the end.
System information (please complete the following information):
N.A.
Additional information
N.A.