viash-io/viash

[BUG] Adding an `organization` to the package config causes tests to fail silently.

DriesSchaumont opened this issue · 1 comments

What happened?

Adding a organization to the _viash.yaml file causes tests to silently fail when the test docker container does not exist yet.

Steps to reproduce

LOOK OUT! THIS EXAMPLE RUNS docker system prune -a -f and will remove all containers, volumes and caches from your system
Let's setup a test case:

rm -r /tmp/test_bug
mkdir -p /tmp/test_bug
cd /tmp/test_bug
cat > _viash.yaml << 'HERE'
viash_version: 0.9.0-RC6

source: src
target: target
organization: "openpipelines-bio"
HERE

mkdir -p src/dummycomponent

cat > src/dummycomponent/config.vsh.yaml << 'HERE'

name: myscript
namespace: "dummycomponent"
test_resources:
  - type: python_script
    path: test_script.py
    text: |
      raise ValueError("TEST FAIL")

engines:
  - type: docker
    image: 'python:latest'

runners:
  - type: executable
HERE

And run this test component:

docker system prune -a -f && viash test ./src/dummycomponent/config.vsh.yaml

This gives:

====================================================================
+/tmp/viash_test_myscript_6536994418194033747/test_test_script/test_executable
[notice] Checking if Docker image is available at 'dummycomponent/myscript:test'
[warning] Could not pull from 'dummycomponent/myscript:test'. Docker image doesn't exist or is not accessible.
[notice] Building container 'dummycomponent/myscript:test' with Dockerfile
Traceback (most recent call last):
  File "/viash_automount/tmp/viash_test_myscript_6536994418194033747/test_test_script/tmp/viash-run-myscript-QmFUxD.py", line 31, in <module>
    raise ValueError("TEST FAIL")
ValueError: TEST FAIL
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory

Running the test again, does produce the correct error:

viash test ./src/dummycomponent/config.vsh.yaml
====================================================================
+/tmp/viash_test_myscript_10126328146746982995/test_test_script/test_executable
Traceback (most recent call last):
  File "/viash_automount/tmp/viash_test_myscript_10126328146746982995/test_test_script/tmp/viash-run-myscript-4xWUSP.py", line 31, in <module>
    raise ValueError("TEST FAIL")
ValueError: TEST FAIL
====================================================================
ERROR! Only 0 out of 1 test scripts succeeded!
Unexpected error occurred! If you think this is a bug, please post
create an issue at https://github.com/viash-io/viash/issues containing
a reproducible example and the stack trace below.

viash - 0.9.0-RC6
Stacktrace:
java.lang.RuntimeException: Only 0 out of 1 test scripts succeeded!
	at io.viash.ViashTest$.apply(ViashTest.scala:137)
	at io.viash.Main$.mainCLI(Main.scala:256)
	at io.viash.Main$.mainCLIOrVersioned(Main.scala:124)
	at io.viash.Main$.main(Main.scala:59)
	at io.viash.Main.main(Main.scala)

But removing the container produces the erroneous behavior again:

docker system prune -a -f && viash test ./src/dummycomponent/config.vsh.yaml
====================================================================
+/tmp/viash_test_myscript_10345737742027411938/test_test_script/test_executable
[notice] Checking if Docker image is available at 'dummycomponent/myscript:test'
[warning] Could not pull from 'dummycomponent/myscript:test'. Docker image doesn't exist or is not accessible.
[notice] Building container 'dummycomponent/myscript:test' with Dockerfile
Traceback (most recent call last):
  File "/viash_automount/tmp/viash_test_myscript_10345737742027411938/test_test_script/tmp/viash-run-myscript-J7Ka4f.py", line 31, in <module>
    raise ValueError("TEST FAIL")
ValueError: TEST FAIL
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory

Expected behavior

I would expect viash to capture and return the correct exit code and not silently fail

Relevant log output

No response

Version

  • OS: Ubuntu 24.04 LTS
  • Java: openjdk 21.0.3 2024-04-16
  • Viash 0.9.0-RC6

Possible solution

No response

Confirmation

  • I have searched the existing issues to make sure this is not a duplicate.
  • I have provided clear and concise information about the bug.

Additional context

No response

Turned out it wasn't specific to the usage of organization, but was an overal issue. This could also occur with e.g. viash run.
Fixed in Viash 0.9.0-RC7